Email Subscription Form

Saturday, March 2, 2019

Easy Free Automation Part VII: Load Tests

Load testing is a key part of checking the health of your application.  Just because you get a timely response when you make an HTTP request in your test environment doesn't mean that the application will respond appropriately when 100,000 users are making the same request in your production environment.  With load testing, you can simulate different scenarios as you make HTTP calls to determine how your application will behave under real-world conditions.

There are a wide variety of load testing tools available, but many of them require a subscription.  Both paid and free tools can often be confusing to use or difficult to set up.  For load testing that is free, easy to install, and fairly easy to set up, I recommend K6.


As with every installment of this "Easy Free Automation" series, I've created an automation script that you can download here.  In order to run the load test script, you'll need to install K6, which is easy to do with these instructions.

Once you have installed K6 and downloaded your test script, open a command window and navigate to the location where you downloaded the script.  Then type k6 run loadTestScript.js.  The test should run and display a number of metrics as the result.

Let's take a look at what this script is doing.  I'm making four different requests to the Swagger Pet Store.  (For more information about the Swagger Pet Store, take a look at this blog post and the posts that follow it.)  I've kept my requests very simple to make it easier to read the test script: I'm adding a pet with just a pet name, I'm retrieving the pet, I'm updating the pet by changing the name, and I'm deleting the pet.

import http from "k6/http";
import { check, sleep } from "k6";

In the first two lines of the script, I'm importing the modules needed for the script: the http module that allows me to make http requests, and the check and sleep modules that allow me to do assertions and put in a wait in between requests. 

export let options = {
  vus: 1,
  duration: "5s"
};

In this section, I'm setting the options for running my load tests.  The word "vus" stands for "virtual users", and "duration" describes how long in seconds the test should run.

var id = Math.floor(Math.random() * 10000);
console.log("ID used: "+id);

Here I'm coming up with a random id to use for the pet, which will get passed through one complete test iteration. 

var url = "http://petstore.swagger.io/v2/pet";
var payload = JSON.stringify({ id: id, name: "Grumpy Cat" });
var params =  { headers: { "Content-Type": "application/json" } }
let postRes = http.post(url, payload, params);

This is how the POST request is set up.  First I set the URL, then the payload, then the headers; then I do the POST request. 

check(postRes, {
    "post status is 200": (r) => r.status == 200,
    "post transaction time is OK": (r) => r.timings.duration < 200
  });
sleep(1);

Once the POST has executed, and the result has been assigned to the postRes variable, I check to make sure that the status returned was 200, and that the transaction time was less than 200 milliseconds.  Finally, I have the script sleep for one second. 

Now let's take a look at the load test output:

INFO[0005] ID used: 1067

This is the id created for the test, which I set up to be logged in line 12 of the script.  At the beginning of each iteration of the script, a new id will be created and logged. 

✓ put status is 200
✓ put transaction time is OK
✓ delete status is 200
✓ delete transaction time is OK
✓ post status is 200
✓ post transaction time is OK
✓ get status is 200
✓ get transaction time is OK

Here are the results of my assertions.  All the POSTs, GETs, PUTs, and DELETEs were successful.

http_req_duration..........: avg=27.56ms min=23.16ms med=26.68ms max=34.69ms p(90)=31.66ms p(95)=33.18ms  

This section shows metrics about the duration of each request.  The average request duration was 27.56 milliseconds, and the maximum request time was 34.69 milliseconds.

iterations................. : 1       0.199987/s
vus........................ : 1         min=1 max=1
vus_max.................... : 1    min=1 max=1

This section shows how many complete iterations were run during the test, and what the frequency was; how many virtual users there were; and how many virtual users there were in maximum.

Obviously, this wasn't much of a load test, because we only used one user and it only ran for five seconds!  Let's make a change to the script and see how our results change.  First we'll leave the number of virtual users at 1, but we'll set the test to run for a full minute.  Change line 6 of the script to duration: "1m", and run the test again with the k6 run loadTestScript.js command. 

http_req_duration..........: avg=26.13ms min=22.3ms  med=25.86ms max=37.45ms p(90)=27.57ms  p(95)=33.56ms

The results look very similar to our first test, which isn't surprising, since we are still using just one virtual user. 

iterations.................: 14      0.233333/s

Because we had a longer test time, we went through several more iterations, at the rate of .23 per second.

Now let's see what happens when we use 10 virtual users.  Change line 5 of the test to: vus: 10, and run the test again.

✓ delete transaction time is OK
✓ post status is 200
✓ post transaction time is OK
✗ get status is 200
     ↳  83% — ✓ 117 / ✗ 23
✓ get transaction time is OK
✓ put status is 200
✓ put transaction time is OK
✗ delete status is 200
     ↳  77% — ✓ 108 / ✗ 31

We are now seeing the impact of adding load to the test; some of our GET requests and DELETE requests failed.

http_req_duration..........: avg=27.8ms  min=21.17ms med=26.67ms max=63.74ms p(90)=33.08ms p(95)=34.98ms

Note also that our maximum duration was much longer than our duration in the previous two test runs.

This is by no means a complete load test; it's just an introduction to what can be done with the K6 tool.  It's possible to set up the test to have realistic ramp-up and ramp-down times, where there's less load at the beginning and end of the test and more load in the middle.  You can also create your own custom metrics to make it easier to analyze the results of each request type.  If you ever find yourself needing to some quick free load testing, K6 may be the tool for you.

Next week, I'll close out the "Easy Free Automation" series with a look at accessibility tests!


38 comments:

  1. Very useful post and I think it is rather easy to see from the other comments as well that this post is well written and useful. I bookmarked this blog a while ago because of the useful content and I am never being disappointed. Keep up the good work..
    software testing company
    QA Outsourcing Sevices

    ReplyDelete
  2. Freelance maintenance technicians also benefit from establishing relationships with multiple clients. Working as a freelancer, you’ll gain employment experience with a variety of businesses, boosting your overall skill set and providing you with more work opportunities in the future.

    maintenance technician

    ReplyDelete
  3. Entertainment can be more fun with Hulu as here you get access to a world full of your favorite TV shows. Here will get the opportunity to enjoy the latest content that you can stream in any of your devices supporting the app. Get your free trial and access to your device in the


    Hulu Login

    Hulu Plus Login

    Hulu Sign In
    Hulu Account

    Hulu Customer Service

    ReplyDelete


  4. Garmin Express is an extremely versatile software application available for both Windows and Mac devices. However, you are required to check the specified requirements and ensure your system conforms to it. This is extremely important for flawless setup and unmatched performance.


    Garmin.com/Express

    ReplyDelete



  5. The software gives you desktop notifications and step-by-step instructions so that you can update your devices with ease. Even if your application is not open or the device is not connected to your system, it will give you alerts on a regular basis. So for garmin.com/express setup, you need to follow all the full-proof steps given below.

    Garmin.com/express

    ReplyDelete


  6. Garmin.com/express is a computer application for easily setting up, registering and managing your Garmin device. The software gives you alerts whenever there are new updates available so that you can install them easily. You can also save multiple maps and preferred places with Garmin Express.







    Garmin.com/express

    ReplyDelete



  7. Among all this, Product Activation Key is the most important requirement for Webroot Download. This 20-digit reference code combines a unique set of alphanumeric characters that identifies your Subscription Terms. You can get this code only after purchasing the preferred version of this software. This implies that you must purchase a Subscription.






    Webroot Download

    ReplyDelete



  8. Your Microsoft Office retail card or the receipt you purchased comes with an alphanumeric product key. If the activation key does not come with your product package, you should contact your retailer or Microsoft support in the correct manner.Nevertheless,





    Office.com/setup

    ReplyDelete

  9. An extensively famous and adored web-based email service, AOL Mail is now a renowned name. The mail service has gained its major fan following because of the fastest interface and features that are user-friendly. You need to choose this platform because here, you get convenient access to your emails and also enjoy several features that are brilliant.





    AOL Mail

    ReplyDelete

  10. AOL Desktop Gold is an advanced and easy-to-use version of AOL software. It includes the features of AOL mail with the personalized chat window and the AOL app that can be installed on both Android or iOS versions.







    AOL Desktop Gold Download

    ReplyDelete

  11. Magellan Roadmate Update provides the best possible guidance for GPS navigation. The Magellan Roadmate delivers a best in class GPS navigation experience with spectacular features. Also, it is developed with features to reduce driving stress, save time and money.




    Magellan Roadmate Update

    ReplyDelete


  12. Norton is an antivirus or anti-malware software product that provides users security with a variety of different virus detection methods. It offers real-time protection against viruses, malware, spyware, and other cyber-threats.





    Norton Login

    ReplyDelete
  13. Hi my name is Jimyyandrew. I am from technical team i have 2 year of expriance.If you face any problem for setting up hp printer you can visit on this link 123.hp.com/setup here you will find right steps for setting up hp printer.

    ReplyDelete
  14. Check out the option of assignment help in New Zealand to buy the assistance of academic writing. Online Assignment Help connects you with professional writers so you can get requisite help for your submission.

    ReplyDelete
  15. Check out the option of assignment help in New Zealand to buy the assistance of academic writing. Online Assignment Help connects you with professional writers so you can get requisite help for your submission.

    ReplyDelete
  16. Thanks for sharing this great article. Hi, I am John Smith, I am working as a technical manager at email support. I have 3 years of experience in this field. If you have any problems related to the mail.com email login issue, then please contact for instant help related to email problems.

    ReplyDelete
  17. Is your Canon printer offline? Here’s how you can get it back online in a few simple ways. First of all, you should try to reconnect your printer to the wifi and then see if it is back online or not.
    If not, then hold the printer’s wifi switch till the lights start flickering. Once if constantly blinks blue, press the router’s WPS button. If this doesn’t help then you can restart the spooler services.
    You should also make sure that the canon printer has been set as default otherwise it might continue to show offline status even though it will be on and perfectly connected.

    ReplyDelete
  18. There could be several possible reasons for hp Printer not printing color. Check-out the list noted below:
    • The color cartridge is out of ink
    • Printer software may get outdated
    • Printhead problem
    • Color Settings are not adjusted correctly
    • Color Cartridge might be empty or low
    • Inappropriate placement of cartridge
    To figure-out the actual cause for not getting printouts in color is quite difficult. So, it is recommended to take fruitful advice from well-qualified technicians. They will help you and give right direction so the printer not printing problem will instantly be fixed from the root


    ReplyDelete
  19. Very wonderful article. I liked reading your article. Very wonderful share. Thanks ! .
    software testing course in chennai

    ReplyDelete
  20. If you have HP Printer, you must expect quality printouts. But being a technical device, some error you may experience. One common problem mostly users have come across is their hp printer not printing black. However, this can be caused by low black ink cartridge install or the printer may not be appropriately connected to the device. So, the productive solution to this problematic error is to check the ink cartridge and it has low or empty ink, refill the black ink correctly and then print a test page. If your printer gives quality printouts or printing black, it means the problem has been sorted out

    ReplyDelete
  21. Want to do a comcast email login setup? So, get ready for it, because her in this blog you will find all the details about it. You can also use Xfinity Connect Page and app as they come with inbuilt setup ready and you don’t need to do setup Comcast email. However, if you choose using an email program instead then these settings will help you in setting up your Comcast mail.

    ReplyDelete
  22. Accounting assignment help allows you to connect with experts and get effective assistance for your project submission. Don’t forget to check out all the features before buying the services of online assignment help for your accounting papers.

    ReplyDelete
  23. QuickBooks company file is one of the most secured files that contains crucial information about your QB application. Only an administrator or someone with administrator rights can open this file on the server computer. This can not be accessed by any other user. If you want to access the company file but QuickBooks won't open the company file this clearly means you are not allowed to open it. You can do one of the following things to open the file-
    1. Run QuickBooks as Administrator.
    2. Verify the network date and time settings.
    3. Check do you have permission to access the QB installation folder?
    4. Check the internet explorer settings.
    5. Check your firewall and antivirus settings.

    ReplyDelete
  24. Mobile real money online casinos because they are the Vegus168 แทงบอลออนไลน์ betting agent of various games that many casinos accept and are used as the main factor in playing online casino games
    Vegus168.

    ReplyDelete
  25. Actually, Facebook Marketplace allows its user who is 18 years or up to post images of their garage sales items and collections and browse items which are offered by nearby sellers as well. Here, the sellers set the price of the item, but buyers are also welcome here for the best price with a discount.

    ReplyDelete
  26. Thanks for sharing post.

    Hi, I am Sofi Hayat, I am working as a tech expert at email support. I have 3 years of experience in this field. If you have any problems related to 1&1 email settings, etc, then please contact for instant help related to email problems.

    ReplyDelete
  27. Actually, Facebook Marketplace allows its user who is 18 years or up to post images of their garage sales items and collections and browse items which are offered by nearby sellers as well. Here, the sellers set the price of the item, but buyers are also welcome here for the best price with a discount.

    ReplyDelete
  28. Go to the official Netflix website. There’s no require to be anxious about finding the Australian website as the Netflix website will automatically direct you there based on your location. Now that you’re on the Netflix website you should make out the screen below. Contact on Netflix Support Number Australia.

    ReplyDelete

  29. Thanks for sharing post.

    Hi, I am Sofi Hayat, I am working as a tech expert at email support. I have 3 years of experience in this field. If you have any problems related to Earthlink email not working, etc, then please contact for instant help related to email problems.

    ReplyDelete
  30. The blog written is extremely impressive, with a great topic. However, a bit more research could have strengthened it even further. You can explore the services as offered by livewebtutors.com, a premium academic writing services platform offering the best of Dissertation Editing Service teamed with knowledge and experience.

    ReplyDelete
  31. Keep one thing in your mind, if you permanently delete Cash app account, then you are not allowed to access it again. And all account’s details erase from there as well as your contact, funds, transaction history, etc. So, if you want to remove someone from your account, then need to do it before deleting the Cash app account. https://www.monktech.us/blog/how-to-delete-cash-app-account/

    ReplyDelete
  32. There are many Epson printer customers who boost a voice of help for faced Epson Printer Troubleshooting to their troubles as well. Being a top-notch help provider, it's miles our duty to address higher care of the customers’ queries. As all of us understand the Epson printer can occasionally forestall functioning because of unknown errors.

    ReplyDelete
  33. The information you are providing that is really good. Thank for making and spending your precious time for this useful information. Thanks again and keep it up.
    Top CA firm in India

    ReplyDelete
  34. This comment has been removed by the author.

    ReplyDelete
  35. Many aspirants face different types of issues to deal with their essays and assignments. For them, writing assignment help services are the right option. When you hire teachers, you are safe because the mentors are skilled in this field. They know how to deal with critical essays and complete it within the timeframe. If you are looking for the best Assignment Help or you need the top assignment experts for your assignment, you can u can get all the support here. We have a team of dedicated and professional staff who can give you a fruitful solution and help you get good grades.

    ReplyDelete
  36. This article is enlightening lastly I got what I needed. Thanks much for this sort of data and keep it up. Audit firm in Delhi

    ReplyDelete

New Blog Location!

I've moved!  I've really enjoyed using Blogger for my blog, but it didn't integrate with my website in the way I wanted.  So I&#...