How to Reproduce a Bug

Have you ever seen something wrong in your application, but you haven't been able to reproduce it?  Has a customer ever reported a bug with a scenario that you just couldn't recreate?  It is tempting to just forget about these bugs, but chances are if one person has seen the issue, other people will see it as well.  In this post I'll discuss some helpful hints for reproducing bugs and getting to the root cause of issues.

Gather Information

The first thing to do when you have a bug to reproduce is to gather as much information as you can about the circumstances of the issue.  If it's a bug that you just noticed, think about the steps that you took before the bug appeared.  If it's a bug that someone else has reported, find out what they remember about the steps they took, and ask for details such as their operating system, browser, and browser version.

One Step at a Time

Next, see if you can follow the steps that you or the reporter of the bug took.  If you can reproduce the bug right away, you're in luck!  If not, try making one change at a time to your steps, and see if the bug will appear.  Don't just thrash around trying to reproduce the issue quickly; if you're making lots of disorganized changes, you might reproduce the bug and not know how you did it.  Keep track of the changes you made so you know what you've tried and what you haven't tried yet.

Logs and Developer Tools

Application logs and browser developer tools can be helpful in providing clues to what is going on behind the scenes in an application.  A browser's developer tools can generally be accessed in the menu found in the top right corner of the browser; in Chrome, for example, you click on the three-dot menu, then choose "More Tools", then "Developer Tools".  This will open a window at the bottom of the page where you can find information such as errors logged in the console or what network requests were made.


Factors to Consider When Trying to Reproduce a Bug

• User- what user was utilized when the bug was seen?  Did the user have a specific permission level?  You may be dealing with a bug that is only seen by administrators or by a certain type of customer.
• Authentication- was the user logged in?  There may be a bug that only appears when the user is not authenticated, or that only appears when the user is authenticated.
• State of the data- what kind of data does the user have?  Can you try reproducing with exactly the same data?  The bug might only appear when a user has a very long last name, or a particular type of image file.
• Configuration issues- There may be something in the application that isn't set up properly.  For example, a user who isn't getting an email notification might not be getting it because email functionality is turned off for their account.  Check all of the configurable settings in the application and try to reproduce the issue with exactly the same configuration.
• Actions taken before the issue- Sometimes bugs are caused not by the current state where they are seen, but by some event that happened before the current state.  For example, if a user started an action that used a lot of memory, such as downloading a very large file, and then continued on to other activities while the file was downloading, an error caused by running out of memory might affect their current activity.  
• Back button- the Back button can be the culprit in all kinds of mysterious bugs.  If a user navigates to a page through the Back button, the state of the data on the page might be different from what it would be through standard forward navigation.
• Caching- caching can result in unexpected behavior as well. For example, it might appear as if data is unchanged when it in fact has been changed.  If a cache never expires or takes too long to expire, the state of the data can be very different from what is displayed.
• Race conditions- these issues are very difficult to pin down.  Stepping through the code probably won't help, because when the program is run one step at a time the problem doesn't occur.  The best way to determine if there is a race condition is to run your tests a several times and document the inconsistent behavior.  You can also try clicking on buttons or links before a page has loaded in order to speed up input, or throttling your internet connection in order to slow down input.  
• Browser/Browser Version- Browsers are definitely more consistent in their behavior than they used to be, and most browsers are now updated to the latest version automatically, but it's still possible to find bugs that only appear in certain browsers or versions.  If your end user is using IE 8 on an old Windows desktop, for example, it's extremely likely that your application will behave differently for them.
• Browser Size- if a customer is saying that they don't see a Save button or a scrollbar in their browser window, ask them to run a browser size tool in another tab on their browser.  Set your browser to be the same size as theirs, and see if you now have the same problem.
• Machine or Device- Mobile devices are highly variable, so it's possible that a user is seeing an issue on an Android device that you are not seeing on an iOS device, or that a user is seeing a problem on a Motorola device when you are not seeing it on a Samsung.  Laptops and desktop computers are less variable, but it is still possible that there is an issue that a Mac owner is experiencing that you don't see in Windows.  Tools like Browserstack, Perfecto, and Sauce Labs can be helpful for diagnosing an issue on a machine or device that you don't own.  

Once You've Reproduced the Bug

Once you've finally reproduced a tricky bug, you might want to just show it to your developer and be done with it.  But your work is not done!  To make sure that the bug is fixed correctly, you'll want to narrow down the steps to reproduce it as much as you can, to be as precise as possible.  The more precise you can be, the easier it will be for the developer to locate the true cause in the code and fix it.

For example, if you were able to reproduce a bug by using an admin user who navigated to the page with the Back button on a Firefox browser, are you sure that you need all three of those conditions to see the bug?  Do you see the bug when you use an admin user and the Back button in Chrome?  If you do, then you can eliminate the browser from your bug report.  What about if you use a non-admin user?  If you see the bug when you are using a non-admin user, you can take that out of the bug report as well.  Now you have narrowed down the issue to just the Back button, giving the developer a clear indication of where to start with a fix.  

End Users Are Depending on Us

As a software tester, I am often annoyed by bugs and poor user experiences I encounter in my day-to-day living.  I frequently say to myself "Who is testing this website?  What were they thinking?  Does this company even have testers?"  We are the last line of defense to keep our customers from frustration, which is why it is important to chase down those elusive bugs.