Email Subscription Form

Saturday, January 5, 2019

The Automation Test Wheel in Practice

Last week's blog post, "Rethinking the Pyramid: The Automation Test Wheel", sparked many interesting discussions on LinkedIn, Twitter, and in the comments section of this blog!  The general consensus was that the Test Pyramid is still useful because it reminds us that tests closest to the code are the fastest and most reliable to run, and that the Automation Test Wheel reminds us to make sure to include categories such as security, accessibility, and performance testing.  Also, a reader pointed us to Abstracta's Software Testing Wheel, which looks at the definition of quality from a number of different perspectives.

This week I'm talking about how to put the Automation Test Wheel into practice.  Let's imagine that I have a simple web app called Contact List.  It allows a user to log in, view a list of their contacts, and add new contacts.  I want to design a complete automation strategy for this application that will enable my team to deploy all the way up to production confidently.  In order to feel confident about the quality of my application, I'll want to be sure to include tests from every segment of the Automation Test Wheel.

Unit Tests: I will make sure that every function of my code has at least one unit test.  I'll run these tests using mock objects.  For example, I will create a list of mock contacts and a mock new contact, add the new contact, and verify that the new contact has been added to the list of mock contacts.  I'll update a contact with new data and verify that the contact has been updated in the list.  I'll create a mock contact with invalid data and verify that attempting to add the contact results in an appropriate error.  These are just some examples; for each function in my app, I'll want to have several tests which exercise all possible code paths.

Component Tests:  My application is very simple and relies on just one database.  The database is used for both authentication and for retrieving the contact data.  I will include one test for each function; I'll send an authentication request for a valid user and verify that the user is authenticated, and I'll make one request to the database to retrieve a known contact, and verify that the contact is retrieved.

Services Tests: My application has an API which allows me to do CRUD operations (Create, Read, Update, Delete) on my contacts.  I have a GET endpoint which allows me to retrieve the list of contacts, and a GET endpoint which allows me retrieve one specific contact.  I have a POST endpoint which allows me to add a contact to the contact list.  I have a PUT endpoint which allows me to update the data for an existing contact, and I have a DELETE endpoint which allows me to delete an existing contact.  For each one of these endpoints, I will have a series of tests.  The tests will include both happy paths and error paths.  I'll verify that in each request, the response code is correct and the response body is correct.  For example, with the GET endpoint where I retrieve one contact, I'll verify that a GET on an existing contact returns a 200 response and the correct data for the contact.  I'll also verify that a GET on a contact that doesn't exist returns a 404 Not Found response.

User Interface (UI) Tests: This is where I will be testing in the browser, doing activities that a real user would do. A real user will want to fetch their list of contacts, add a new contact, update an existing contact, and delete a contact.  I will have one test for each of these activities, and each test will have a series of assertions.  To take one example, when I add a new contact, I will navigate to the new contact page, fill in all the form fields, and click the Save button.  Then I will navigate to the list page and verify that my new contact appears on the page.

Visual Tests: This is where I will verify that elements are actually appearing on the page the way I want them to.  I will navigate to the list page and verify that all of the columns are appearing on the page.  I will navigate to the add contact page and verify that all of the form fields and their labels are appearing appropriately on the page.  I will trigger all possible error messages (such as the one I would receive if I entered an invalid zip code), and verify that the error appears correctly on the screen.  And I will verify that all of the buttons needed to use the application are rendering correctly.

Security Tests: I will run security tests at both the Services layer and the UI layer.  I will test the API operations relating to authenticating a user, verifying that only a user with the correct credentials will be authenticated.  I will test every request endpoint to make sure that only those requests with a valid token are executing; requests without a valid token should return a 401.  For the UI layer, I will conduct a series of login tests that validate that only a user with correct credentials is logged in, and I will verify that I cannot navigate to the list page or the add contact page without being logged in.

Performance Tests: I will set benchmarks for both the server response time and the web page load time.  To measure the server response, I will add assertions to my existing Services tests that will verify that the response was returned within that benchmark.  To measure the web page load time, I will run a UI test that will load each page and assert that the page was loaded within the benchmark time.

Accessibility Tests:  I want to make sure that my application can be used by those with visual difficulties.  So I will run a set of UI and Visual tests on each page where I validate that I can zoom in and out on the text and that scroll bars appear and disappear depending on whether they are needed.  For example, if I zoom in on the contact list I will now need a vertical scrollbar, because some of the contacts will now be off the page.

With this series of automated tests, I will feel confident that I'll be able to deploy changes to my application and discover any problems quickly.

I've received a few questions over the last week about what percentage of total tests each of spokes in the Automation Test Wheel should have.  The answer will always be "It depends".  It will depend on these and many other considerations:

  • How many other services does your application depend on?  If it depends on many external services, you'll need more Component tests.
  • How complicated is your UI?  If it has just a page or two, you'll need fewer UI and Visual tests.  If it has several pages with many images, you'll need more UI and Visual tests.
  • How complicated is your data structure?  If you are dealing with large data objects, you'll need more Services tests to validate that CRUD operations are being handled correctly.
  • How secure does your application need to be?  An application that handles personal banking will need many more Security tests than an application that saves pictures of kittens.
  • How performant does your application need to be?  A solitaire game doesn't need to be as reliable as a heart monitor.

The beauty of the Automation Test Wheel is that it can be tailored to all types of software applications!  By considering each spoke in the wheel, we'll be sure that we are creating great automated test coverage.


  1. Simple example, good test ideas. For me, your explanation is great because it is about how to think during testing. That's the most important.

  2. Hello Kristin,

    I had experience with more types of automation:
    - Integration / Data driven (a few units get exercised together to check predefined inputs and outputs)
    - "Monkey" automation, generating random inputs ( ). Can reveal crashes, hangs or unexpected issues.

    I think the "wheel" model may have troubles with including more types of automation.

    1. Hi Podolyan! Data-driven and monkey tests are definitely methods of software testing. I would consider them methods rather than types, though. Integration/Data-driven testing could fit in the Component or Service test type, and Monkey automation could fit in the UI test type. The Automation Wheel focuses more on the "what" than the "how".

  3. This comment has been removed by a blog administrator.

  4. Great to know about Automation Test Wheel Everyone must wisely choose the Automation Testing Services for your project to achieve higher customer satisfaction and loyalty.


  5. Very Good Blog. Highly valuable information have been shared.Great information has been shared. We expect many more blogs from the author. Special thanks for sharing..
    software testing services
    software testing companies
    Regression testing services
    Performance testing Services
    Test automation services

  6. This comment has been removed by a blog administrator.

  7. Test automation helps in reducing regression testing time and cutting down the time to market with significant cost savings on a long-term basis. However, a clear automation strategy and roadmap are key to ensuring the right return on investment on your automation initiatives.

  8. I must thank you for the efforts you have put in penning this site. I am hoping to check out the same high-grade content by you later on as well. Keep up the good work
    software testing companies
    software testing tester
    software testing and quality assurance services
    Automation testing services

  9. Thank you so much for this nice information. Hope so many people will get aware of this and useful as well. And please keep update like this.

    Game QA Solutions

    PC Game Testing Services

  10. Great article on different type of testing. Are you looking for an IT solution Company? Then, contact Helenzys is the best. Here, at a cost-effective price, you will get excellent service that will help you to get happy consumers. The team leverages their complete effort to ensure that your business earns a greater revenue. We provide IT solutions to different industries- Healthcare, Automobile, Agriculture, Telecommunication, E-business & Retail, Education and technology.

  11. Very useful post and I think it is rather easy to see from the other comments as well that this post is well written and useful. I bookmarked this blog a while ago because of the useful content and I am never being disappointed. Keep up the good work..
    software testing company
    QA Outsourcing Sevices

  12. I like visiting your site since I always come across interesting articles like this one. Keep sharing! Regards. Read more about
    Selenium Training in Chennai | Certification | Online Courses

    selenium training in chennai

    selenium training in chennai

    selenium online training in chennai

    selenium training in bangalore

    selenium training in hyderabad

    selenium training in coimbatore

    selenium online training

  13. Great!! Thank you for sharing this post on software testing. Software testing outsourcing services are in great demand these days. Good to see such nice articulated post.

  14. Thanks for sharing such informative blog. It really helped me a lot to learn new things about software testing. Keep on sharing informative and useful stuffs. Great blog!
    Software Testing Services
    Software Testing Company
    Software Testing Companies in USA
    QA Testing Companies
    Software Testing Services in USA

  15. Great Article… I love to read your QA outsourcing services articles because your writing style is too good, its is very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.

  16. Thank you so much for this nice information, Nice Information About The Automation Test Wheel in Practice.

    We Providing :

    Software Automation Testing Training

  17. We're offering additional discount on early enrollment on full module Automation Training. Get Job Support with Unlimited Placement Opportunities.
    Call us at +91-9953489987, 9711287737
    For more details Visit

  18. Very interesting blog. Many blogs I see these days do not really provide anything that attracts others, but believe me the way you interact is literally awesome.You can also check my articles as well.

    Security Guard License
    Ontario Security License
    Security License Ontario
    Security License

    Thank you..

  19. I like your post there is a lot of information about software testing companies , which i would like to learn, thank you for the great guide.

  20. Thanks for this amazing blog, visit Ogen Infosystem for creative web design and development services at an affordable price.
    Website Designing Company in Delhi

  21. It is really amazing blog, Thanks for sharing.
    For any type of certification include BIS Certifications visit our website.

    Bis Consultant in delhi

  22. This comment has been removed by the author.

  23. Thanks for sharing this informative article related software testing, really helpful.
    Software Testing Training in Pune

  24. Thanks for sharing this meaningful article, I appreciated your work and your research. I am a trainer offer professional training of quality analyst/software testing and business analysts training with all essential concepts.

  25. Water bodies are the main source of transportation for international freight forwarding. Due to this, sea freight company in Delhi,
    Freight Forwarder in Vietnam
    Shipping Company In India

  26. Thank you so much for this nice information. Hope so many people will get aware of this and useful as well. And please keep update like this.
    software testing course in chennai

  27. The Best Article I ever read about The Automation Test Wheel in Practice
    Inspired By Your Content Keep Sharing Amazing Testing Content With Us
    We Provided Best SEO Services And Web Design Go chicago seo consultant


New Blog Location!

I've moved!  I've really enjoyed using Blogger for my blog, but it didn't integrate with my website in the way I wanted.  So I&#...