Email Subscription Form

Saturday, June 20, 2020

Managing Test Data

It's never fun to start your work day and discover that some or all of your nightly automated tests failed. It's especially frustrating when you discover that the reason why your tests failed was because someone changed your test data. 
Test data issues are a very common source of aggravation for software testers. Whether you are testing manually or running automation, if you think your data is set the way you want it, and it has been changed, you will waste time trying to figure out why your test results aren't right. 

Here are some of the common problems with test data:
Users overwrite each other's data
I was on a team that had an API I'll call API 1. I wrote several automated tests for this API using a test user. API 1 was moved to another team, and my team started working on API 2. I wrote several automated tests for API 2 as well. Unfortunately, I used the same test user for API 2, and this test user needed to have a different email address for API 2 than it did for API 1. This meant that whenever automated tests were run on API 1, it changed the address of the test user, and then my API 2 tests would fail.
Configuration is changed by another team
When teams need to share a test environment, changes to the environment configuration made by one team can impact another team. This is especially common when using feature toggles. One team might have test automation set up with the assumption that a feature toggle will be on, but another team might have automation set up with the expectation that the feature toggle is off.
Data is deleted or changed by a database refresh
Companies that use sensitive data often need to periodically scramble or overwrite that data to make sure that no one is testing with real customer information. When this happens, test users that have been set up for automation or manual testing can be renamed, changed, or deleted, causing tests to fail.
Data becomes stale
Sometimes data that is valid at one point in time becomes invalid as time passes. A great example of this is a calendar date. If an automated test needs a date in the future, the test writer might choose a date a year or two from now. Unfortunately, in a year or two, that future date will become a past date, and then the test will fail.

What can we do to better manage test data? Here are some suggestions:
Use Docker
Using a virtual environment like Docker means that you have complete control over your test environment, including your application configuration and your database. To run your tests, you spin up a virtual machine, run the tests, and destroy the machine when the tests have completed.
Create a fresh database for testing
It's possible to create a brand-new database for the sole purpose of running your test automation. With Windows, this can be accomplished by creating a SQL DACPAC. You can set your database schema, add in only the data that you need for testing, create the database, point your tests to that database, and destroy the database when you are finished.
Give each team their own test space
Even if teams have to share the same test environment, they might be able to divide their testing up by account. For example, if your application has several test companies, each team can get a different test company to use for testing. This is especially helpful when dealing with toggles; one team's test company can have a feature toggled on while another team's test company has that feature toggled off.
Give each team their own users
If you have a situation where all teams have to use the same test environment and the same test account, you can still assign each team a different set of test users. This way teams won't accidentally overwrite one another's data. You can give your users names specific to your team, such as "Sue GreenTeamUser". 
Create new data each time you test
One great way to manage test data is to create the data you need at the beginning of the test. For example, if you need a customer for your test, you create the new customer at the beginning of your test suite, use that customer for your tests, and then delete the customer at the end of your tests. This ensures that your test data is always exactly the way you want it, and it doesn't add bloat to the existing database.
Use "today+1" for dates in the future
Rather than choosing an arbitrary date in the future, you can get today's date, and then use an operation like DateAdd to add some interval, like a day, month, or year, to today's date. This way your test date will always be in the future. 
Working with test data can be very frustrating. But with some planning and strategy, you can ensure that your data will be correct whenever you run a test. 

8 comments:

  1. Great advice. I've already faced some of the problems you described when I started to automate some test cases at the company I work for. Test data is something we should always keep an eye on when designing our tests.
    I'm almost finishing your course on LinkedIn by the way. It is great!
    Have a nice week!

    ReplyDelete
  2. É um tema que dentro das empresas sempre estamos debatendo e é recorrente! Adorei o texto porque não fica só no âmbito filosófico, traz sugestão práticas e fáceis de aplicar. Obrigada!

    ReplyDelete
  3. Hey Guys !

    USA Fresh & Verified SSN Leads with DL Number AVAILABLE with 99.9% connectivity
    All Leads have genuine & valid information

    **HEADERS IN LEADS**
    First Name | Last Name | SSN | Dob | DL Number | Address | City | State | Zip | Phone Number | Account Number | Bank Name | Employee Details | IP Address

    *Price for SSN lead $2
    *You can ask for sample before any deal
    *If anyone buy in bulk, we can negotiate
    *Sampling is just for serious buyers

    ==>ACTIVE, FRESH CC & CVV FULLZ AVAILABLE<==
    ->$5 PER EACH

    ->Hope for the long term deal
    ->Interested buyers will be welcome

    **Contact 24/7**
    Whatsapp > +923172721122
    Email > leads.sellers1212@gmail.com
    Telegram > @leadsupplier
    ICQ > 752822040

    ReplyDelete
  4. Hey Guys !

    USA Fresh & Verified SSN Leads with DL Number AVAILABLE with 99.9% connectivity
    All Leads have genuine & valid information

    **HEADERS IN LEADS**
    First Name | Last Name | SSN | Dob | DL Number | Address | City | State | Zip | Phone Number | Account Number | Bank Name | Employee Details | IP Address

    *Price for SSN lead $2
    *You can ask for sample before any deal
    *If anyone buy in bulk, we can negotiate
    *Sampling is just for serious buyers

    ==>ACTIVE, FRESH CC & CVV FULLZ AVAILABLE<==
    ->$5 PER EACH

    ->Hope for the long term deal
    ->Interested buyers will be welcome

    **Contact 24/7**
    Whatsapp > +923172721122
    Email > leads.sellers1212@gmail.com
    Telegram > @leadsupplier
    ICQ > 752822040

    ReplyDelete
  5. Great post, thank you! By the way if you need to upgrade CRM or another data strage, to find programmers become easier now.

    ReplyDelete
  6. This is such a great resource that you are providing and you give it away for free. This is really a nice and informative, containing all information and also has a great impact on the new technology.
    Herbie is the foremost Chatbot Companies in UAE. Our Chatbot Development Companies. We are specialized in AI Conversational Chatbots and Virtual Assistant Chatbot. Herbie is most intelligent Virtual Assistant ever developed by AI Chatbot Companies

    ReplyDelete
  7. Selling USA FRESH SSN Leads/Fullz, along with Driving License/ID Number with good connectivity.

    **PRICE FOR ONE LEAD/FULLZ 2$**

    All SSN's are Tested & Verified. Fresh spammed data.

    **DETAILS IN LEADS/FULLZ**

    ->FULL NAME
    ->SSN
    ->DATE OF BIRTH
    ->DRIVING LICENSE NUMBER
    ->ADDRESS WITH ZIP
    ->PHONE NUMBER, EMAIL
    ->EMPLOYEE DETAILS

    ->Bulk order negotiable
    ->Minimum buy 25 to 30 leads/fullz
    ->Hope for the long term business
    ->You can asked for specific states too

    **Contact 24/7**

    Whatsapp > +923172721122

    Email > leads.sellers1212@gmail.com

    Telegram > @leadsupplier

    ICQ > 752822040

    ReplyDelete
  8. Thanks for posting such a Useful information .You have done a great job.
    software testing course in chennai

    ReplyDelete

New Blog Location!

I've moved!  I've really enjoyed using Blogger for my blog, but it didn't integrate with my website in the way I wanted.  So I&#...