Email Subscription Form

Saturday, August 11, 2018

Mobile Testing Part III: Seven Automated Mobile Testing Tips (and Five Great Tools)

Walk into any mobile carrier store and you will see a wide range of mobile devices for sale.  Of course you want to make sure that your application works well on all of those devices, in addition to the older devices that some users have.  But running even the simplest of manual tests on a phone or tablet takes time.  Multiply that time by the number of devices you want to support, and you've got a huge testing burden!

This is where automated mobile testing comes in.  We are fortunate to be testing at a time where there are a whole range of products and services to help us automate our mobile tests.  Later in this article, I will discuss five of them.  But first, let's take a look at seven tips to help you be successful with mobile automated testing.



Tip 1: Don't test things on mobile that could be more easily tested elsewhere

Mobile automation is not the place to test your back-end services.  It's also not the place to test the general logic of the application, unless your application is mobile only.  Mobile testing should be used for verifying that elements appear correctly on the device and function correctly when used.  For example, let's say you have a sign-up form in your application.  In your mobile testing, you'll want to make sure that the form renders correctly, that all fields can be filled in, that error messages display appropriately, and that the Save button submits the form when clicked on.  But you don't want to test that the error message has the correct text, or that the fields have all saved correctly.  You can save those tests for standard Web browser or API automation.

Tip 2: Decide whether you want to run your tests on real devices or emulators

The advantage of running your tests on real devices is that the devices will behave like the devices your users own, with the possibility of having a low battery, connectivity issues, or other applications running.  But because of this, it's more likely that your tests will fail because a phone in the device farm froze up or was being used by another tester.  Annoyances like these can be avoided by using emulators, but emulators can't completely mimic the real user experience.  It's up to you decide which choice is more appropriate for your application.  

Tip 3: Test only one thing at a time

Mobile tests can be flaky, due to the issues found in real devices discussed above and other issues such as the variations found in different phones and tablets.  You may find yourself spending a fair amount of time taking a look at your failed tests and diagnosing why they failed.  Because of this, it's a good strategy to keep your tests small.  For example, if you were testing a login screen, you could have one test for a successful login and a second test for an unsuccessful login, instead of putting both scenarios into the same test.

Tip 4: Be prepared to re-run tests

As mentioned in Tip 3, you will probably encounter some flakiness in your mobile tests.  A test can fail simply because the service that is hosting the emulator loses connectivity for a moment.  Because of this, you may want to set up a system where your tests run once and then re-run the failed tests automatically.  You can then set up an alert that will notify you only if a test has failed twice.

Tip 5: Don't feel like you have to test every device in existence

As testers, we love to be thorough.  We love to come up with every possible permutation in testing and run through them all.  But in the mobile space, this can quickly drive you crazy!  The more devices you are running your automated tests on, the more failures you will have.  The more failures you have, the more time you have to spend diagnosing those issues.  This is time taken away from new feature testing or exploratory testing.  Do some research on which devices your users own and come up with a list of devices to test with that covers most, but not all, of those devices.  

Tip 6: Take screenshots

Nothing is more frustrating than seeing that a test failed and not being able to figure out why.  Screenshots can help you determine if you were on the correct screen during a test step and if all the elements are visible.  Some mobile testing companies take a screenshot of every test step as the test progresses.  Others automatically take a screenshot of the last view before a test fails.  You can also code your test to take screenshots of specific test steps.  

Tip 7: Use visual validation

Visual validation is essential in mobile testing.  Many of the bugs you will encounter will be elements not rendering correctly on the screen.  You can test for the presence of an element, but unless you have a way to compare a screenshot with one you have on file, you won't really be verifying that your elements are visible to the user.  Applitools makes an excellent product for visual comparison.  It integrates with common test software such as Selenium, Appium, and Protractor.  With Applitools, you can build visual verification right into your tests and save a collection of screenshots from every device you test with to use for image comparison. 

Now let's discuss some good tools for test automation.  I've already mentioned Applitools; below are four other tools that are great for mobile test automation.  The mobile landscape is filled with products for automated testing, both open-source and paid.  In this post, I am discussing only the products that I have used; there are many other great products out there. 

Visual Studio App Center:  A Microsoft product that allows you to test Android and iOS applications on real devices.  A screenshot is taken of every test step, which makes it easy to figure out where a test started to go wrong. 

Appium:  An open-source product that integrates with Selenium and provides the capability to test on device emulators (or real devices if you integrate with a device farm). 

Sauce Labs:  Sauce Labs is great for testing on both mobile devices and web browsers on all kinds of operating systems.  You can run tests on real devices or emulators, and you can run tests in parallel.  They integrate well with Selenium and Appium.  A screenshot is taken whenever a test fails, and you can even watch a video of your test execution.

Perfecto: Uses real devices and integrates with Visual Studio, Appium, and Selenium.  They can simulate real-world user conditions such as network availability and location.

Whatever automated test tools you choose to use, remember the tips above, and you will ensure that you are comprehensively testing your application on mobile without spending a lot of time debugging. 

I initially said this series on Mobile Testing was going to be three blog posts long.  On reflection, I've realized that we need a fourth post: Mobile Security Testing.  This is a topic I know very little about.  So I'll be doing some research, and you can expect Mobile Testing Part IV from me next week!


32 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. I am so happy to found your blog post because it's really very informative. Please keep writing this kind of blogs and I regularly visit this blog. Have a look at my services.
    Cyber Security Training Course in Chennai | Certification | Cyber Security Online Training Course | Ethical Hacking Training Course in Chennai | Certification | Ethical Hacking Online Training Course | CCNA Training Course in Chennai | Certification | CCNA Online Training Course | RPA Robotic Process Automation Training Course in Chennai | Certification | RPA Training Course Chennai | SEO Training in Chennai | Certification | SEO Online Training Course

    ReplyDelete
  3. Nice and interesting post, I appreciate your hard work. keep it up…!!!Thanks for such useful information, It is true that now if you want to grow your business you will surely need the mobile app testing services for your business. But for that purpose everyone needs best mobile app testing companies.

    ReplyDelete
  4. If the company has a winning track record and has been in business with more than just Salesforce they have the ability to supply more than just one area of solution. lwc interview questions

    ReplyDelete
  5. Nice work, truly valuable to me.
    I hope you keep it up.
    if you are looking for best offshore development services then,
    Visit here:
    Iyrix Technologies
    Remote Software Developers
    Software Development Services
    Backend Developers

    ReplyDelete
  6. Thank you for sharing this amazing post. Looking forward to reading more.
    Visit us: dot net training
    Visit us: Dot Net Online Training Hyderabad

    ReplyDelete
  7. If you are worried and facing financial problems in your life then don’t worry. Contact the Famous Astrologer in the USA, Pandit RamDial Ji by book an appointment and you can also call him directly. Chat or Whatsapp options are also available for you.

    best indian astrologer in new york

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete
  9. This is a very useful post by you. Thank you for sharing it .Please visit here Unmatched Top Mobile Performing Testing Services By 5Data Inc

    ReplyDelete
  10. Nice article on Mobile App Testing. For Mobile Apps Pls Visit Customized Software Development Services

    ReplyDelete
  11. The tips cover aspects like scripting reusable test cases, parallel execution, and incorporating real device testing. These strategies enhance test coverage and expedite the testing lifecycle. The highlighted tools, chosen for their versatility and effectiveness, empower testers with options for robust automation.https://www.mobilezmarket.com/

    ReplyDelete
  12. Thank you for precious information, Contact us today to learn about Selenium automation software testing and how it can benefit your team today! some helpful tips as to how users can prevent.

    ReplyDelete
  13. Thank You so much for wonderful information …great work…well done…keep doing…Looking for the software testing company in India contact today now.

    ReplyDelete
  14. CMOLDS is a leading website development company dubai offering web and app design and development services in this domain.

    ReplyDelete

New Blog Location!

I've moved!  I've really enjoyed using Blogger for my blog, but it didn't integrate with my website in the way I wanted.  So I&#...