The Hierarchy of Quality

About a year ago, I wrote a post suggesting that we could think about automation in terms of a test wheel, where each section of the wheel represented a different type of automation.  A reader who works at Abstracta told me that my wheel reminded her of the wheel they use to think about all of the different facets of quality.  I thought their wheel was so great that I knew I would eventually want to write a post about it.

I've been thinking about the different types of quality mentioned in Abstracta's Software Testing Wheel, and wondering what I would do if I was brought on to a project that had never had any testing and I needed to start from scratch.  Where would I begin my testing?  I thought about what the most important needs are for quality, and I was reminded of Maslow's Hierarchy of Needs.

For those who are unfamiliar with this psychological concept, this is a theory that all human beings need to have certain basic needs met before they can grow as people.  The needs are as follows:

1. Physiological needs- food, water, shelter
2. Safety needs- security, property, employment
3. Love and belonging- friendship, family
4. Esteem- respect, self-esteem
5. Self-actualization- becoming the best person one can be

Looking at this list, it's clear that physiological needs are the most important.  After all, it doesn't matter if you have high self-esteem if you have no water to drink.  Each successive need builds on the more important one before it.

With this in mind, I realized that there is a Hierarchy of Quality- certain conditions of quality that need to be met before a team can move on to the next area of quality.  Here is my perception of where the different areas of the Abstracta test wheel fall in the hierarchy:

1.  Functionality and Reliability

These two areas share the most important spot.  Functionality means that the software does what it's supposed to do.  This is critical, because without this, the application might as well not exist.  Imagine a clock app that didn't tell time, or a calculator that didn't add numbers.

Reliability means the software is available when it's needed.  It doesn't really matter if the app works if a user can't get to it when they need it.

Once these quality needs have been met, we can move on to the next level:

2. Security and Performance

Security is important because users need to feel that their data is being protected.  Even applications that don't have login information or don't save sensitive data still need to be protected from things like cross-site scripting, which might allow a malicious user to gain control of someone else's device.

Performance is also important, because no one wants to wait for sixty seconds for a web page to load.  If an application isn't responsive enough, the users will go elsewhere.

Now that the application is secure and performant, we can go to the third level:

3. Usability and Compatibility

This is the level where we make sure that as many users as possible have a good experience with the application.  Usability means that the workflows of an application are intuitive so users don't get confused.  It also means that the application is internationalized, so users all around the world can use it, and that it is accessible, so users with visual, auditory, or physical differences can use it as well.

Compatibility means that users with different operating systems, browsers, or devices can use the application.  Have you ever filled out a form in a browser and had it not save correctly when you clicked the button?  This has happened to me more than once, and I've needed to fill out the form again in a different browser to have it save correctly.  It's important that our users have a positive experience no matter where they are using the software.

Now that we've made our application accessible to as many users as possible, it's time to go on to the next level:

4. Portability

Portability covers how easy it is to move an application from one place to another.  One example of portability would be the way I can access my Google Drive files on my laptop, my tablet, and my phone.  Portability also refers to how easy an application can be installed or updated.  Also, we want our application to keep working when a device has an operating system upgrade.

Finally, we have thought about all of our users' needs.  Now it's time for one more level:

5. Maintainability

This is a level of quality that benefits the software team.  Maintainability refers to how easily an application can be updated.  Is it possible to add new APIs or update existing ones?  How easy is it to test the system?  Is it easy to deploy new code?  Is it easy for other teams to use the code?  Is the code clear and easy to understand?

When software is accessible and easy to use for all end users, AND is easy to work with and maintain for the development team, then truly high quality has been achieved.

I hope that this Hierarchy of Quality will help you make decisions about what areas of an application should be focused on first when there are a number of different quality areas competing for your team's attention.

What do you think of this order?  Do you agree or disagree with where I placed items in the hierarchy?  Are there any missing quality areas?  Let me know in the comments below!

...but TEST like a QA Engineer!

In last week's post, I wrote about how it is important for software testers to code like a developer.  But there is a second half of the sentence "Code like a developer...", and that is that software testers should be TESTING. 

I'm not a stickler for using the right word for testing-related concepts, which is why I use the term "test automation".  But automated testing is really automated checking.  Automated tests serve a very valuable purpose in that they can run regression checks at any hour of any day, without human intervention.  But they do not actually test the software.

A sad casualty of the very important move towards test automation is the QA Engineer.  Many large software companies don't employ QA Engineers any more, feeling sure that Software Developers in Test are all that's needed to validate the quality of their software.  And many Software Developers in Test focus solely on the automation, working from acceptance criteria in development stories and looking at the code rather than manually interacting with the software.  How is that trend working out for end users?  

Just this week, I experienced the following:  I received a (legitimate) email that I had some money to accept from PayPal.  The email contained a button to click that said "Accept the Money".  When I clicked it, I got a message that said "The previous page is sending you to an invalid URL."  

Last week when I was using a mobile app, a screen that I needed stayed permanently blank.  And in a post I wrote two weeks ago, I mentioned that while I was writing, Blogger had a page load error when I tried to add an image.  

Three weeks, three major companies, three bugs.  This is what comes from not employing people who think and act like testers.  

It's true that the whole software development team owns quality, and that quality is everyone's responsibility.  And there are also non-QA people who care deeply about certain areas of an application:

• Developers write unit tests to check the quality of their code
• Product Owners care about whether the feature does what it's supposed to
• UX Designers care about whether the user journey is intuitive
• Security testers check the software for vulnerabilities
• Performance engineers care about the response time the application 

But only QA Engineers care so much about the quality of the application that they'll do things like:

• Type ~!@#$%^&*()-=_+{}|[]\:";'<>?,./ into every text field to test for invalid character handling
• Try to purchase -1, 99999999999, 1.3415, and foo of something
• Enter a birth year of 3019 to see what happens
• Click every button twice to check for multiple submissions
• Click the forward and back button on every single page of a website
• Test 48 different permutations of feature sets to be as thorough as possible 
• Create dozens of test users with many varieties of security settings, to have scenarios ready for testing at a moment's notice
• Become an expert on a particular feature and provide documentation and assistance to other testers
• Test the same thing in the QA environment, the Staging environment, the Demo environment, and the Production environment to make absolutely sure that the feature is working everywhere 
• Test every feature on every supported browser and every supported mobile device

This is why we need software testers who TEST.  We need people who will continually ask themselves "How could we break this?", "What haven't we tested yet?", and "What features will be used with this?".  We need software testers who don't rush into writing automation without first interacting with a feature.  We need software testers who remember that the goal of all their efforts is to have a user who has a positive, bug-free experience.  

Code Like a Developer...

I'll be honest: I don't love coding.  Don't get me wrong, I love test automation!  I love the feeling of solving a technical challenge and coming up with a great way to automatically assert that software is doing what it's supposed to be doing.  I love maintaining and updating my automated test suites.  But the actual writing of the code is not my favorite thing.  Whenever I find myself having to write another nested "for" loop, I sigh inwardly.

However, with all the coding I've done over the years, I've come to really appreciate the work that software developers do!  Software is complex stuff, and developers have come up with great ways to set standards, share repositories, and review each other's work.

The test automation code we write is important; just as important as the code the software developers are writing.  Therefore, we should write our code with the same standards the developers use.  Here are a few suggestions for coding practices you should adopt:

Your code should live in the same repository as the developers' code.

This is for a few reasons: first, the developers' unit tests reside with the code, so it makes sense to have your integration and UI tests in the same place.  Secondly, it's easier to maintain one repository instead of two; and finally, having your code in the same place serves to remind the whole team that test automation is everyone's responsibility.  

Write clean code.

When I first got started with test automation, I had absolutely no idea what I was doing.  All I had was my manual testing experience and a couple of courses in Java and C++.  I did a lot of Googling and a lot of guessing as I put together my first Selenium tests.  After much work, they ran and (mostly) passed, but boy, were they lousy!  I didn't know anything about how to write clean code.  Fortunately I had great developers around to teach me how to make my code better.

Here are some of the principles of writing clean code:

• Keep it simple.  Always look over your code and ask yourself if there's a simpler way of doing what it is that you are trying to do.  Sometimes the obvious solution to a testing problem only becomes clear after you have solved it in a complicated way; now it's time to go back and solve it more elegantly. 
• Don't repeat yourself.  If there's something you're doing in more than one test- for example, logging in to the application- write a method that you can call instead of putting those steps into every test.  Similarly, create a file where you save all of your variables and element locators, and have all of your tests refer to that file.  That way if a variable or a locator changes, you can make the change in one place rather than several. 
• Be consistent.  Consistent code is easier to read.  Be consistent with your casing: if you have a variable for the user's first name called "firstName", don't make the variable for the user's last name "LastName".  Follow the conventions that your developers are using: if they indent with two spaces, you should too.  If they put their opening curly braces on a separate line, you should as well.
• Comment your code.  It's not always obvious what test automation code is doing at first glance, and while you might be quite used to the syntax you are using for your tests, your developers might not be familiar with it.  Simple comments like "Polling the queue for the delete request" can be really helpful in explaining your intent.  Moreover, what might seem really obvious to you now might not be obvious in three months when you need to update the test!  Your future self will thank you for the comments you write today.  

Solicit feedback.  

Like me, you may not have had a thorough grounding in good coding principles.  Some of the best software testers I've had the pleasure of working with did not major in Software Engineering.  If you did not go through rigorous training in software development, it's important to get feedback from the developers you work with.  On my team, the software testers often review and approve each other's code, but I also like to have my code checked by developers to make sure I'm not doing anything unusual or creating steps that could possibly result in a race condition.  

Test automation helps the whole team by speeding up the feedback process and freeing testers up to do more exploratory testing.  We owe it to our whole team to write quality code that is readable, runs quickly and consistently, and provides valuable feedback!

You may be wondering why the title of this blog post ends with "...".  Be sure to check out next week's blog to read the other half of the story!  

SQL Query Secrets

Have you ever been querying a SQL table, and one of your queries seems to take forever?  And then the next query you run takes milliseconds?  This would frequently happen to me, and I thought it meant that the server that hosted the database was unreliable in some way.  But this week I learned about indexes, and that the way we structure our queries has a huge impact on how long they will take to execute!  In this post, I'll describe what indexes are and talk about the ways we can use them to optimize our queries.

An index is a database structure that is designed to speed up queries in a table.  An easy way to understand this is to think about the index at the back of a book.  Let's say you have a book on car repair, and you want to find information about your car's brakes.  You could look up "brakes" in the index, or you could search through every single page of the book for the word "brakes".  It's pretty obvious which would take less time!

Unlike books, databases can have more than one index.  There are two different kinds of indexes: clustered and unclustered.  A clustered index is used to store a table in sorted order.  There can only be one clustered index, because the table is stored in only one order.  Unclustered indexes are stored in the original table order, but they save the location of certain fields in the table.

Let's take a look at an example.  If we had a table like this, called the Users table:

UserId	State	LastName	FirstName	Email	Mobile Phone
1	MA	Prunewhip	Prunella	pprunewhip@fake.com	800-867-5309
2	RI	Schmoe	Joe	jschmoe@notreal.com	401-555-8765
3	NH	Smith	Amy	amysmith@foo.com	603-555-3635
4	RI	Jones	Bob	bob@bar.com	401-555-2344
5	MA	Jones	Amy	aj@me.com	617-555-2310

and we had a clustered index defined to have UserId as the key, a search on UserId would be very fast, and the data returned would be in order by UserId.

The table could also use unclustered indexes, such as the following:

State- the records in the table are indexed by state

LastNameFirstName- the records in the table are indexed by LastName and FirstName

When you query a database, the query will first look to see if an index can be used to speed up the search.  For example, if I made the request 

select LastName, FirstName from Users where UserId = 5 

the query would use the UserId index and the LastNameFirstName index to find the record.

Similarly, if I made the request

select LastName, FirstName from Users where State = 'MA'

the query would use the LastNameFirstName index and the State index to find the record.

Of course, with a table of only five records, optimizing in this way won't make much of a difference.  But imagine that this table had five million records, and you can see how using an index would be very helpful.

Querying a table on a non-indexed field is called a table scan.  The query needs to search through the entire table for the values, just as a person who wasn't using a book index would have to search through every single page of the book.  

How can you know what indexes a table has?  You can find out with one simple query:

EXEC sp_helpindex "Users" 

where you would replace "Users" with whatever the name of the table is.  This will return a result of all of the clustered and unclustered indexes applied to the table, and the result will include the name of the index, a description of the index, and all the keys used in the index.

If you want to optimize your SQL queries, only ask for the data that you really need, rather than asking for select *.  Because not every field in the table is indexed, looking for every field will take longer.  

Let's say that you want to query the Users table to find the email addresses of all of the users who live in Massachusetts (MA).  But you also would like to have some more information about those users.  You could ask for 

select FirstName, LastName, Email from Users where State = 'MA'.

To find the records, the query will use the FirstNameLastName index and the State index.  Only the Email will be a non-indexed field.

But if you asked for

select * from Users where State = 'MA'

now the query needs to look for two different non-indexed fields: Email and Mobile Phone.

Another helpful tip is to specify all the keys in an index when you want to use that index to make a query.  For example, if you wanted to find the Email for Prunella Prunewhip, you should ask for 

select Email from Users where LastName = 'Prunewhip' and FirstName = 'Prunella'

rather than asking for

select Email from Users where LastName = 'Prunewhip'.

In the second example, the LastNameFirstName index won't be used.

And when you want to use an index, the query will run faster if you specify the keys in the order they appear, so it's better to say
where LastName = 'Prunewhip' and FirstName = 'Prunella'
than it is to say
where FirstName = 'Prunella' and LastName = 'Prunewhip'

Here's one more tip: when you want to use an index, be sure not to manipulate one of the index keys in your query, because this will mean that the index won't be used.  For example, if you had a table like this, called Grades:

StudentId	LastName	FirstName	Grade
1	Miller	Kara	89
2	Smith	Carol	56
3	Jones	Bob	99
4	Davis	Frank	78
5	Green	Doug	65

and you had an unclustered index called LastNameGrade, and you executed a query like:
select LastName from Grades where (Grade + 100) = 178
the LastNameGrade index wouldn't be used, because the Grade value was being manipulated.  It's necessary for the query to go through the entire table and add 100 to each Grade field in order to search for the correct value.

Armed with this knowledge, you should be able to create queries that will run as fast as possible, getting you the data you need.  I'd like to extend my thanks to my colleague Cindy Gall, whose informative workshop inspired this post!

Six Ways Chrome DevTools Can Help With Testing

Did you know that there is a wealth of testing tools right in your browser?  Web browsers like Chrome and Firefox have developer tools that are available for free, for everyone.  And these tools are not just for developers!  In this post, I'll be sharing six ways that Chrome DevTools can help you with your testing.

To access Chrome DevTools, simply click on the three-dot menu in the upper right corner of your browser, choose "More Tools", and then choose "Developer Tools".  DevTools will open up alongside your browser window.  You can customize where you would like the tools to display by clicking on the three-dot menu in the DevTools nav bar and selecting an option for "Dock Side".  You can choose to have the DevTools display on the left, on the right, on the bottom, or in a separate window.

Here are some of the things that Dev Tools can do:

1. Inspect an HTML Element
Have you ever been writing UI automation and you just can't figure out how to access an element?  With DevTools, you can right-click on the element and choose "Inspect", and the Elements pane of DevTools will show you the element in the HTML.  You can then use this information to figure out the best way to access the element.

2. Edit HTML Elements
Not only can you find an element in the HTML, you can also edit it!  This is great for security testing.  Imagine that there is a page with a button that is hidden for users who are not admins.  A malicious user could find that element using DevTools, remove the "hide" tag, and use the button.  So it's helpful to try this while testing to verify that there's an additional check for user permissions when the button is used.

To edit an element, right-click on it in the HTML displayed in the Elements pane, and choose "Edit as HTML".  Make whatever edits to the element you want, then click out of the edit box.  You should see the element on the page change as a result of your edits.

3. View HTTP requests
If you click on the Network tab of DevTools, you can see all of the requests made to the server while using a web page.  This includes API calls, which you can then copy and use in a tool like Postman.  This feature is helpful for determining if your page is making the API calls that you are expecting, and it's also great for security testing.  For example, just because the front-end of a web page doesn't allow a user to submit a field with more than 50 characters doesn't mean that it can't be done.  If a malicious user copies the API call and submits it through Postman, through a curl command, or through some other tool, they may be able to send more than 50 characters directly to the server.  This is why it's important to have both front-end and back-end validation on a website.

4. Simulate device frames
When you are testing a webpage, it's important to make sure that the page appears correctly on both laptops and mobile devices.  But even the most well-equipped tester doesn't have access to every single device in use today.  So DevTools comes with a simulator that shows roughly what your webpage will look like on various devices.  To access this feature, click on the device logo

in the toolbar.  This will open the simulator in the webpage side of the browser.  Then you can use the dropdown to select specific devices (which seem to be a bit obsolete), or you can choose the "Responsive" setting and then manually expand or contract the window to get the size you want.  The exact size is displayed in the navbar at the top.

5. Simulate performance on slower networks
Testing a webpage while in your office usually means you are using a great high-speed network.  But what about your users who have slower connections?  You can use DevTools to simulate slower connections and throttled CPU, which could help uncover race conditions in your application.  To use this feature, go to the Performance tab in the navbar.  In the Network dropdown, you can choose "Fast 3G", "Slow 3G", or "Offline", and in the CPU dropdown, you can choose "No throttling", "4x slowdown" or "6x slowdown".  Don't forget to reverse your changes when you are done testing!

6. Investigate page load errors
As I was creating this post, I was reminded of one more way that DevTools are helpful.  I was trying to upload the Chrome logo to my post, and the popup that I usually use to add an image was completely blank.  I went to the Console tab of DevTools and saw that there was a 404 "File not found" error when I clicked on the Add Images button in Blogger.  When you are testing your team's application and you've found a bug on a page, checking for errors in the console can help you give more information to your developers so they can get to the root of the problem more quickly.

Sometimes the most useful testing tools are right there in front of you!  I hope this post has inspired you to take a look at DevTools to see how it can help you in your testing.