Automating Your API Tests

Running a test collection in Postman is a great way to test your APIs quickly.  But an even faster way to run your tests is to run them automatically!  In order to automate your Postman tests, we first need to learn how to run them from the command line.

Newman is the command-line running tool for Postman.  It is a NodeJS module, and it is easy to install with npm (the Node Package Manager).  If you do not already have Node installed, you can easily install it by going to https://nodejs.org.  Node contains npm, so once Node is installed, it is extremely easy to install Newman.  Simply open a command line window and type:

npm install -g newman

The "-g" in this command is telling npm to install newman globally on your computer.  This means that it will be easy to run Newman from the command line no matter what folder you are in.

Now that Newman is installed, let's try running a collection!  We will use the Pet Store collection that we created in this blog post and updated with assertions in this post.  In order to run Newman, you will need to export your collection and your environment as .json files.

Find your collection in the left panel of Postman, and click on the three dots next to it.  Choose "Export", then choose Collection v.2.1 as your export option, then click on the "Export" button.  You'll be prompted to choose a location for your collection.  Choose any location you like, but make a note of it, because you'll need to use that location in your Newman command.  I have chosen to export my collection to my Desktop.

Next, click on the gear button in the top right of the screen to open the Environments window.  Next to the Pet Store environment, click on the down arrow.  Save the exported environment to the same location where you saved your collection.

Now we are ready to run the Newman command.  In your command window, type:

newman run <pathToYourFile> -e <pathToYourEnvironment>

Obviously, you will want to replace the <pathToYourFile> and the <pathToYourEnvironment> with the correct paths.  Since I exported my .json files to my Desktop, this is what my command looks like:

newman run "Desktop/Pet Store.postman_collection.json" -e "Desktop/Pet Store.postman_environment.json"

The -e in the command specifies what environment should be used with the collection.

An alternative way of pointing to the .json files is to cd directly into the location of the files.  If I ran cd Desktop, then my command would look like:

newman run "Pet Store.postman_collection.json" -e "PetStore.postman_environment.json"

Also note that the quote marks are not necessary if your file name doesn't have any spaces.  If I were to rename my files so there was no space between Pet and Store, I could run my collection like this:

newman run PetStore.postman_collection.json -e PetStore.postman_environment.json

And, once you have exported your .json files, you can really name them whatever you want.  As long as you call them correctly in the Newman command, they will run.

When you make the call to Newman, you will see your tests run in the command line, and you will wind up with a little chart that shows the test results like this:

If you are running your tests against your company's test environment, you may find that running Newman returns a security error.  This may be because your test environment has a self-signed certificate.  Newman is set by default to have strict SSL, which means that it is looking for a valid SSL certificate.  You can relax this setting by sending your Newman command with a -k option, like this:

newman run "Desktop/Pet Store.postman_collection.json" -e "Desktop/Pet Store.postman_environment.json" -k

Another handy option is the -n option.  This specifies the number of times you want the Newman collection to run.  If you'd like to put some load on your application, you could run:

newman run "Desktop/Pet Store.postman_collection.json" -e "Desktop/Pet Store.postman_environment.json" -n 1000

This will run your collection 1000 times.  In the results chart, you can see what the average response time was for your requests.  While this is not exactly a load test, since the requests are running one at a time, it will still give you a general idea of a typical response time.

Once you have your Newman command working, you can set it to run in a cron job, or a Powershell script, or put the command into a continuous integration tool such as Jenkins, or Travis CI, or Docker. There is also an npm module for reporting Newman results with TeamCity.

Whatever tool you choose, you will have a way to run your Postman API tests automatically!  Next week, we'll talk about which API tests to automate, and when to run them.