Easy Free Automation Part VI: Security Tests

Often when people think of security testing, they think of complicated software scans, request intercepts, and IP address spoofing.  But some of the most crucial application security testing can be done simply through making API requests.  In this week's post, I'm taking a look at examples of authentication testing, authorization testing, and field validation testing.

As I have in every post in this Easy Free Automation series, I've created an example that you can download here.  This is a simple json file that can be run with Newman.  As you recall from Easy Free Automation Part III: Services Tests, Newman is the command-line runner for Postman, my favorite API testing tool.  If you need to install Postman or Newman, take a look at that post.

For my test application, I'm using the awesome Restful-Booker API.  It's worth noting that this API does come with some intentional bugs, two of which I'll mention below.

The json file I've made available on Github is for the test collection.  I didn't include an environment file this week, because I didn't need to store any variables.  Once you have downloaded the json file, open a command window, change directories to get to the directory where the file is stored, and type newman run easyFreeSecurityTests.json. You should see sixteen tests run and pass.

Let's take a look at the kinds of tests we're running.  The tests will be easier to interpret if you upload them into Postman; take a look at this post if you need help doing that.

The first six tests in the collection are authentication tests.  I am verifying that I can't log in with invalid credentials.  But I'm verifying six different invalid username and password combinations:

• Create token with empty username
• Create token with invalid username
• Create token with empty password
• Create token with invalid password
• Create token with empty username and password
• Create token with invalid username and password

This may seem like overkill, but I have actually encountered bugs where a user can log in if the password field is blank, and where a user can log in if both the username and password are incorrect. 

The assertion I am using for each of the six authentication tests is the following:
pm.test("Bad credential message returned", function () {
    pm.expect(pm.response.text()).to.include("Bad credentials");
});

Ordinarily I would assert that the response code I was getting was a 401, but since this request is (incorrectly) returning a 200, I'm instead verifying the text of the response: "Bad credentials".  (For more information on HTTP error codes, see this post.)

The next six tests in my collection are authorization tests.  There are three actions in the Restful-Booker that require a valid token: Put, Patch, and Delete.  So for each of these requests, I'm testing that I cannot run the request with a missing token, and I cannot run the request with an invalid token:

• Update booking with no token
• Update booking with invalid token
• Partial update booking with no token
• Partial update booking with invalid token
• Delete booking with no token
• Delete booking with invalid token

For each of these requests, I am asserting that I receive a 403 status code as a response:

pm.test("Status code is 403", function () {

     pm.response.to.have.status(403);

});

If a developer makes a change to the code and accidentally removes the token requirement for one of these operations, automated tests like these will discover the error right away, because the response code will change to a 200 or a 201.

Finally, I have four field validation tests.  I would like to have more tests here, but because some of the fields in this API aren't validated, I'm sticking to the date fields.  In each of these tests, I am sending in an invalid date:

• Create booking invalid checkin month
• Create booking invalid checkin day
• Create booking invalid checkout month
• Create booking invalid checkout day

In each of these tests, I am validating that I receive an invalid date message:

pm.test("Invalid date response", function () {

     pm.response.to.have.body("Invalid date");

});

Field validation might not seem like a security concern, but it's one of the easiest ways to hack an application, through entering in a script for XSS or a SQL command for SQL injection.  By verifying that the application's input fields are only allowing in certain data types or formats and only allowing in a certain number of characters, we are going a long way towards protecting ourselves from these attacks.

Astute readers will have noticed that we could also have date field validation on the PUT requests that update a booking, and the PATCH requests that partially update a booking.  And of course, if the other fields such as First Name and Last Name had validation (as they should), we would want to test that validation as well.

Running simple repetitive tests like this is not particularly glamorous and will never make headlines.  But it's simple tests like these that can catch a security problem way before it becomes an issue.  Just recently I was alerted that some tests I had set up to run nightly in a QA environment were failing.  Upon investigation, I discovered that my test user was now able to access information belonging to another user.  If I hadn't had authorization tests in place, that security hole might have been missed and might have made it to production.  

Next week, we'll move on to Easy Free Load Tests!