Ask Your Way to Success

Ten years ago, I didn’t know how to use a Windows computer.  I didn’t know how the file system worked.  I didn’t know what right-clicking on a mouse did.  Today I am a QA Engineer doing both manual and automated testing at a great company.  How did I get here from there?

I asked a lot of stupid questions.

Most people are reluctant to ask questions, because they are afraid to look ignorant.  But I maintain that the best way to learn anything quickly is to ask questions when you don’t understand what’s going on.
Here are six ways that asking questions improves your knowledge and the health of your company:

1. Questions give others an opportunity to help you, which helps them get to know you better and establishes a rapport.  At my first official QA job, I was working with hotshot developers, all of whom were at least a decade and a half younger than me.  It was embarrassing having to admit that I didn’t know how to reset a frozen iPhone or find the shared drive in File Explorer, but I asked those questions anyway, I remembered the answers, and I showed my co-workers that I was a fast learner.

2. Questions help developers discover things they may have missed.  On countless occasions where a developer has been demonstrating a feature to me I’ll ask a question like “But what if there are no records for that user?”, or “What if GPS isn’t on?”, and they will suddenly realize that there is a use case they haven’t handled.

3. Questions keep everyone honest.  I have worked with other QA engineers who bandy about terms like “back-end call” or “a different code path” without actually knowing what they are talking about.  Asking what they mean by what they are saying makes sure that they do the work to find out what they are actually testing.  And when they get their answers, I get my answers as well.

4. Questions give you an opportunity to clear things up in your head.  You may have heard the expression “Rubber duck debugging”; I think this method works well when you’re asking questions.  I have found that sometimes just formulating the question out loud while I’m asking it clears things up for me.

5. Questions clarify expectations.  Yes, sometimes I have felt dumb saying things like “You want me to test this on the latest build, right?”, but every now and then I discover that there’s been a miscommunication, and I’d much rather find out about it before I start testing rather than after I’ve been testing the wrong thing for an hour.

6. Questions clarify priorities.  There have been many times where I’ve asked “Why are we adding this feature?”  There is almost always a good reason, but the discussion helps the team understand what the business use case is, which helps the developers decide how to design their solution.

A caveat: Don’t ask questions that you can find the answers to by using a search engine (example: “How do I find the UDID of a device using iTunes?”) or by going back and reading your email (example: “What day did we decide on for code freeze?”).  Asking these types of questions results in wasted time for everyone!

In summary, asking might make you feel silly in the short run, but it will make you and your team much smarter in the long run.  And hopefully it will create an atmosphere in which others feel comfortable asking questions as well, improving teamwork for everyone!

Fix All the Things

It’s very easy when you are rushing to complete features to let some bugs slide.  This article will show why in most cases it’s better to fix all the bugs now rather than later.  The following scenario is hypothetical, but is based on my experience as a tester.  

NewTech Inc. is very excited about offering a new email editor to their customers.  Customers will be able to compose emails to their clients and schedule when they should be sent from within the NewTech app.  NewTech’s service reps will also have the ability to add or change a company logo for the customers.  

Because the feature is on a deadline, developers are rushing to complete the work.  The QA engineer finds an issue: when changing the company logo, the logo doesn’t appear to have changed unless the user logs out and back in again.  The team discusses this issue and decides that because customers won’t see the issue (since it’s functionality that only NewTech employees can use), it’s safe to let this issue go on the backlog to be fixed at another time.

The feature is released, and customers begin using it.  The customers would all like to add their company logo to their emails, so they begin calling the NewTech service reps asking for this service.  The service reps add the logo and save, but they don’t see the logo appear on the email config page.  The dev team has forgotten to let them know that there’s a bug here, and that the workaround is to log out and back in again.  

Let’s assume that each time a service provider encounters the issue and emails someone on the dev team about it, five minutes is wasted.  If there are ten service reps on the team, that’s fifty wasted minutes.

Total time wasted to date: Fifty minutes

But now everyone knows about the issue, so it won’t be a problem anymore, right?  Wrong!  Because NewTech has hired two new QA engineers.  Neither one of them knows about the issue.  They encounter it in their testing, and ask the original QA engineer about it.  “Oh, that’s a known issue,” he replies.  “It’s on the backlog.”  Time wasted: ten minutes for each new QA engineer to investigate the problem, and ten minutes for each new engineer to ask the first QA engineer about it.

Total time wasted to date:  One hour and ten minutes

Next, a couple of new service reps are hired.  At some point, they each get a request to from a company to change the company’s logo.  When they go to make the change, the logo doesn’t update.  They don’t know what’s going on, so they ask their fellow service reps.  “Oh yeah, that’s a bug,” say the senior service reps.  “You just need to log out and back in again.”  Time wasted: ten minutes for each new service provider to be confused about what’s going on, and ten more minutes in conversation with the senior service reps.

Total time wasted to date: One hour and thirty minutes

It’s time to add new features to the application.  NewTech decides to give their end users the option of adding a profile picture to their account.  A new dev is tasked with adding this functionality.  He sees that there’s an existing method to add an image to the application, so he chooses to call that method to add the profile picture.  He doesn’t know about the login/logout bug.  When one of the QA engineers tests the new feature, she finds that profile picture images don’t refresh unless she logs out and back in again.  She logs a new bug for the issue.  Investigating the problem and reporting it takes twenty minutes.

Total time wasted to date:  One hour and fifty minutes

The dev team meets and decides that because customers will see the issue, it’s worth fixing.  The dev who is assigned to fix the issue is a different dev from the one who wrote the image-adding method (who has since moved on to another company), so it takes her a while to familiarize herself with the code.  Time spent fixing the issue: two hours.  

Total time wasted to date: Three hours and fifty minutes

The dev who fixed the issue didn’t realize that the bug existed for the company logo as well, so didn’t mention it to the QA engineer assigned to test her bug fix.  The QA engineer tests the bug fix and finds that it works correctly, so she closes the issue.  Time spent testing the fix: thirty minutes.

Total time wasted to date: Four hours and twenty minutes

When it’s time for the new feature to be released, the QA team does regression testing.  They discover that there is now a new issue on the email page: because of the fix for the profile images, now the email page refreshes when customers make edits, and the company logo disappears from the page.  One of the QA engineers logs a separate bug for this issue.  Time spent investigating the problem and logging the issue: thirty minutes.

Total time wasted to date: Four hours and fifty minutes

The dev team realizes that this is an issue that will have significant impact on customers, so the developer quickly starts working on a fix.  Now she realizes that the code she is working on affects both the profile page and the email page, so she spends time checking her fix on both pages.  She advises the QA team to be sure test both pages as well.  Time spent fixing the problem: one hour.  Time spent testing the fixes: one hour.

Total time wasted to date: Six hours and fifty minutes

How much time would it have taken for the original developer to fix the original issue?  Let’s say thirty minutes, because he was already working with the code.  How much time would it have taken to test the fix?  Probably thirty minutes, because the QA engineer was already testing that page, and the code was not used elsewhere.  

So, by fixing the original issue when it was found, NewTech would have saved nearly six hours in work that could have been spent on other things.  This doesn’t seem like a lot, but when considering the number of features in an application, it really adds up.  And this scenario doesn’t account for lost productivity from interruptions.  If a developer is fielding questions from the service reps all day about known issues that weren’t fixed because they weren’t customer-facing bugs, it’s hard for him to stay focused on the coding he’s doing.

The moral of the story is: unless you think that no user, internal or external, will ever encounter the issue, fix things when you find them!

How to Train Your Dev

Training your Dev is really about training yourself.  A more accurate (but much less catchy) title for this blog post would be “How to work and communicate effectively in order to facilitate a productive relationship with developers”.  

There are two steps to having a good working relationship with your dev: 1) developing good work habits, and 2) communicating clearly.  We’ll take a look at these two steps in detail.

Good Work Habits:

• Make sure you have completely read a feature’s Acceptance Criteria and all available notes and documents.  This can help prevent unnecessary and time-consuming misunderstandings.
• Ask questions if there is anything in the feature that you don’t understand.  Don’t make potentially incorrect assumptions.
• Document your work.  This is especially helpful when you have found an issue and the developer needs to know what browser you were using or what server you are pointing to.
• Check twice to make sure that you really have a bug.  Perhaps what you are seeing is a configuration problem, a connection problem, or simply user error.

Communicating Clearly:

• Learn the preferred communication style of your dev and use it.  For example, some developers like to hear about issues immediately, and testing and bug fixing become a collaboration.  Other developers prefer to hear right away only if the issues are big ones, and would rather have you document the smaller issues for a later conversation.
• Ask your dev to walk you through any confusing features.  He or she will be happy to explain things to you, because they know that any information they give you at the outset of testing will save misunderstandings later.
• Be kind when reporting issues.  Your dev has worked hard on the feature he or she has delivered, and we all know it’s no fun to have our work criticized.
• Give feedback in the form of a question.  This can soften the blow of finding a bug.  For example, “I noticed that when I clicked the Save button, I wasn’t taken to the next page.  Is this as designed?”
• Let your dev know what he or she can do to help you do your job more efficiently.  A good example of this is asking them to not assign an issue to you until it is actually in the test environment, so you won’t inadvertently start testing it before the code is there.

A good working relationship with your dev is all about trust!  You trust that your dev has completed the work they’ve assigned to you, that they’ve done some of their own testing before the handoff, that the work is in the QA environment and ready for testing, and that your dev has let you know about any potential areas of regression to test.  

In turn, your developer trusts that you have tested everything in the Acceptance Criteria, that you’ve done regression testing, that you’ve tested with various security levels and on various browsers, that the issues you’ve found are legitimate, and that you will clearly communicate what you tested and what issues you found.  

Train yourself to work effectively and communicate clearly, and you will find this level of trust in your relationship with all the developers you work with!

The Trouble With Toggles

Some of you may have seen the classic Star Trek episode, "The Trouble with Tribbles".  In the episode, Lt. Uhura gets a gift of a cute little ball of fur from a passing trader.  It's a huge hit with the crew, and when the tribble has babies, everyone who wants one can enjoy these little fuzzy creatures.  The trouble begins when they discover that the tribbles are born pregnant and quickly give birth, increasing the tribble population exponentially until they are taking over the entire ship. 

Today I would like to talk about the trouble with toggles, and what they have in common with tribbles.  Toggles can be helpful to developers in many ways.  They make it easier for development of a new feature to continue in the main branch, without disturbing the work of others.  They enable us to do beta testing, where a handful of customers try out a feature or layout and provide their feedback while most customers continue using the software without the changes.  And they enable us to quickly remove a feature if it's discovered that it has a critical error.  But toggles can be problematic for two very important reasons, which I will explain using our tribble friends.

First, toggles make more work for QA engineers.  Just as the tribbles multiplied exponentially, the number of test passes we need to do when using toggles also multiplies exponentially.  Consider the following scenario: Team One adds a new feature with a toggle.  In order to make sure that the feature hasn't broken any existing functionality, they need to do two passes of smoke tests: one with the toggle on, and one with the toggle off.  Next they add a second feature with a toggle.  Now there are four different combinations of scenarios that their software can have: both toggles on, toggle A on and toggle B off, toggle B on and toggle A off, and both toggles off.  So they'll need to do four different passes of smoke tests to make sure that all their features are working correctly.  Now let's add in Team Two.  They have two toggles as well, so they'll need to do four passes of smoke tests for all the permutations of toggle A and toggle B. 

As release day approaches, Team One and Team Two are merging their code into the main branch.   Now there are EIGHT different possible combinations for toggle states. Even if the teams decide that it's a low risk to skip smoke testing in all eight scenarios, there will still be a lot of investigative work involved for every bug found.  Let's say someone finds a bug in an instance where Toggle A and C are on and Toggle B and D are off.  They'll need to answer questions like "What if just Toggle C is on?  Do we still see the bug? What about if Toggle D is turned on as well?  Does the bug go away?"  Research like this can be really time-consuming.  This is time that could be better spent doing exploratory testing.  If all the features were enabled in all scenarios, testers would only need to do one pass of smoke tests and spend the rest of their time digging deeper into each new feature.

Secondly, toggles can give software teams a false sense of security, which can lead to buggy software and tech debt.  In "The Trouble with Tribbles", the tribbles' purring sound had a soothing effect on human's nervous systems, which made the crew pay less attention to the fact that in a couple of days the tribble population would fill the Enterprise from deck to ceiling.  Similarly, engineers can be soothed by the knowledge that they can always turn the feature off, which means that they can test the feature less carefully. If bugs are found, they can be dismissed with the statement, "Well, it's Beta, and the Beta users will understand.  We'll log it and take a look at it later." Bugs like these can compound, especially when mixed with all the other bugs in all the other Beta features.  And the bugs can take a lot longer to fix six months later when the original developers don't remember what they did or may not even be with the company any more. Alternatively, if the teams tested every feature with the knowledge that all users would be seeing it on first release, they would test more carefully and fix bugs right away, ensuring much greater code quality from the outset.

Like tribbles, toggles can be fun, in that they enable us to add more pizazz more quickly to an application.  But as the crew of the Enterprise found, too much of a good thing has its consequences!