Easy Free Automation Part VII: Load Tests

Load testing is a key part of checking the health of your application.  Just because you get a timely response when you make an HTTP request in your test environment doesn't mean that the application will respond appropriately when 100,000 users are making the same request in your production environment.  With load testing, you can simulate different scenarios as you make HTTP calls to determine how your application will behave under real-world conditions.

There are a wide variety of load testing tools available, but many of them require a subscription.  Both paid and free tools can often be confusing to use or difficult to set up.  For load testing that is free, easy to install, and fairly easy to set up, I recommend K6.

As with every installment of this "Easy Free Automation" series, I've created an automation script that you can download here.  In order to run the load test script, you'll need to install K6, which is easy to do with these instructions.

Once you have installed K6 and downloaded your test script, open a command window and navigate to the location where you downloaded the script.  Then type k6 run loadTestScript.js.  The test should run and display a number of metrics as the result.

Let's take a look at what this script is doing.  I'm making four different requests to the Swagger Pet Store.  (For more information about the Swagger Pet Store, take a look at this blog post and the posts that follow it.)  I've kept my requests very simple to make it easier to read the test script: I'm adding a pet with just a pet name, I'm retrieving the pet, I'm updating the pet by changing the name, and I'm deleting the pet.

import http from "k6/http";
import { check, sleep } from "k6";

In the first two lines of the script, I'm importing the modules needed for the script: the http module that allows me to make http requests, and the check and sleep modules that allow me to do assertions and put in a wait in between requests. 

export let options = {
  vus: 1,
  duration: "5s"
};

In this section, I'm setting the options for running my load tests.  The word "vus" stands for "virtual users", and "duration" describes how long in seconds the test should run.

var id = Math.floor(Math.random() * 10000);
console.log("ID used: "+id);

Here I'm coming up with a random id to use for the pet, which will get passed through one complete test iteration. 

var url = "http://petstore.swagger.io/v2/pet";
var payload = JSON.stringify({ id: id, name: "Grumpy Cat" });
var params =  { headers: { "Content-Type": "application/json" } }
let postRes = http.post(url, payload, params);

This is how the POST request is set up.  First I set the URL, then the payload, then the headers; then I do the POST request. 

check(postRes, {
    "post status is 200": (r) => r.status == 200,
    "post transaction time is OK": (r) => r.timings.duration < 200
  });
sleep(1);

Once the POST has executed, and the result has been assigned to the postRes variable, I check to make sure that the status returned was 200, and that the transaction time was less than 200 milliseconds.  Finally, I have the script sleep for one second. 

Now let's take a look at the load test output:

INFO[0005] ID used: 1067

This is the id created for the test, which I set up to be logged in line 12 of the script.  At the beginning of each iteration of the script, a new id will be created and logged. 

✓ put status is 200
✓ put transaction time is OK
✓ delete status is 200
✓ delete transaction time is OK
✓ post status is 200
✓ post transaction time is OK
✓ get status is 200
✓ get transaction time is OK

Here are the results of my assertions.  All the POSTs, GETs, PUTs, and DELETEs were successful.

http_req_duration..........: avg=27.56ms min=23.16ms med=26.68ms max=34.69ms p(90)=31.66ms p(95)=33.18ms  

This section shows metrics about the duration of each request.  The average request duration was 27.56 milliseconds, and the maximum request time was 34.69 milliseconds.

iterations................. : 1       0.199987/s
vus........................ : 1         min=1 max=1
vus_max.................... : 1    min=1 max=1

This section shows how many complete iterations were run during the test, and what the frequency was; how many virtual users there were; and how many virtual users there were in maximum.

Obviously, this wasn't much of a load test, because we only used one user and it only ran for five seconds!  Let's make a change to the script and see how our results change.  First we'll leave the number of virtual users at 1, but we'll set the test to run for a full minute.  Change line 6 of the script to duration: "1m", and run the test again with the k6 run loadTestScript.js command. 

http_req_duration..........: avg=26.13ms min=22.3ms  med=25.86ms max=37.45ms p(90)=27.57ms  p(95)=33.56ms

The results look very similar to our first test, which isn't surprising, since we are still using just one virtual user. 

iterations.................: 14      0.233333/s

Because we had a longer test time, we went through several more iterations, at the rate of .23 per second.

Now let's see what happens when we use 10 virtual users.  Change line 5 of the test to: vus: 10, and run the test again.

✓ delete transaction time is OK
✓ post status is 200
✓ post transaction time is OK
✗ get status is 200
     ↳  83% — ✓ 117 / ✗ 23
✓ get transaction time is OK
✓ put status is 200
✓ put transaction time is OK
✗ delete status is 200
     ↳  77% — ✓ 108 / ✗ 31

We are now seeing the impact of adding load to the test; some of our GET requests and DELETE requests failed.

http_req_duration..........: avg=27.8ms  min=21.17ms med=26.67ms max=63.74ms p(90)=33.08ms p(95)=34.98ms

Note also that our maximum duration was much longer than our duration in the previous two test runs.

This is by no means a complete load test; it's just an introduction to what can be done with the K6 tool.  It's possible to set up the test to have realistic ramp-up and ramp-down times, where there's less load at the beginning and end of the test and more load in the middle.  You can also create your own custom metrics to make it easier to analyze the results of each request type.  If you ever find yourself needing to some quick free load testing, K6 may be the tool for you.

Next week, I'll close out the "Easy Free Automation" series with a look at accessibility tests!