Azure API Management: Streamline Testing with Postman & Azure DevOps

Have you ever needed to test an API that you have exposed via Azure APIM?

It can be a tedious and repetitive process… right?

Not only do you have to redo testing after every minor change, but you also have to repeat it for each environment you deploy to.

What if there was a way to automate the process with minimal coding?

First, what exactly is Azure API Management (APIM), if you have not used it before? It's a fully managed service provided by Microsoft Azure that enables you to publish, secure, and manage APIs for your organization. By acting as a gateway between your backend services and the clients that consume them, APIM provides a centralized point of control for your API ecosystem. And with features such as rate limiting, authentication, caching, and analytics, APIM is an essential tool for modern application development and integration, especially in large-scale and distributed systems.

So, what are you waiting for? This article describes how we can leverage the capabilities of Postman to automate testing of APIs published in APIM using Azure DevOps

Before we dive into detail, we need to do a bit of prework.

I have set up an APIM instance and created an API following the steps below:

1. Deployed demo conference API as described in this exercise - Learn: Publish your first API.
2. Added a dummy validate-jwt policy on get sessions endpoint. This is to demonstrate valid jwt token generation using pre-request scripts from Postman. You can learn more about jwt policy here APIM Policy: Validate JWT.

Now that we have our demo API deployed with a dummy policy, let’s export it to Postman.

Export API to Postman from APIM

From the Azure Portal, select your API Management instance. From the left navigation of your APIM instance, select APIs and then select the Demo Conference API. Now click on export.

Once you’ve clicked on export, a popup like the one below will appear prompting for export targets.

Select Postman here and you will get a pop-up to choose Postman for Web or Windows. Select as per your preference and this will prompt a dialogue in Postman to select a workspace.

Once you have selected the workspace, import the API into Postman. There is also an option to create a collection at the same time and a few advanced options which you can see in the screen-shot below.

This completes the initial setup of the Postman collection, but it needs further configuration before exporting to Azure DevOps pipeline.  Follow the next steps to complete configuration.

Configure Postman Collection

In this example, we need to create required variables, set up pre-request steps & define tests.

1. Create Variables – Here we are defining important variables to store and reuse values throughout this collection. There are several variables configured in this example as shown below:

2. Configure pre-request – Pre-request scripts in Postman execute java scripts before an actual API request runs. Since our get sessions end point has JWT validate policy defined, it expects a valid token passed in the request header. A pre-request step is defined to get a valid token. The script used is defined in this file DemoConfApi.json under GetSessions.

3. Define Tests - The test configured in this example is checking if the API response status is 200. Additional tests can be easily added here , e.g. – tests to check response content, response time etc.

Once these configurations have been completed, it is a good idea to check everything is working as expected by running the tests in Postman. After making sure everything is working, export the collection in v2.1 format and save the json file.

Now we’re ready to create our DevOps pipeline to run the Postman Collection.

Create DevOps Pipeline to run Postman Collection

The exported json file from the previous step - Configure Postman Collection - contains the variables we have configured originally (baseUrl, apiKey etc) and their values. For security reasons, it is not a good idea to have these sensitive values hardcoded as plain text in a file we are going to upload in DevOps Repo. Also, this prevents reusability of the same collection across different environments since the values will change based on the environment.

To ensure security and reusability of the collection, follow the steps below:

• Create an Azure Key Vault and store all sensitive data as secrets in the key vault.
• Create a variable group in ADO and link the key vault to the variable group.
• Replace hard coded values in the json file with placeholders. For example, I have used the format ##....## (eg. ##apiKey##, ##baseUrl##, etc)

The placeholders in the json file will be later replaced with values from ADO variable group you created in the step above during pipeline execution. We will make use of the command-line tool newman  to execute the exported collection from ADO pipeline.

Now we have “prepared” the json file from Postman, let’s look at the steps required in the yaml pipeline definition to run this Postman collection.

1. Install newman using the command install newman -g
2. Replace the token values in Postman collection using qetza.replacetokens.replacetokens-task.replacetokens task and specifying the token prefix and suffix (##).
3. Execute newman command and specify the exported json collection as parameter. e.g., newman run DemoConfApi.json -x -r junit --reporter-junit-export JUnitReport.xml. Please note that the -x flag will ignore the exit code and continue executing pipeline steps even after the test failures. If you want to stop the pipeline execution on tests failure, remove this flag.
4. Publish the test results using PublishTestResults task and passing the report from the previous step as test result file.

You can take a look at sample yml pipeline definition here. This sample can be used to create a new ADO pipeline to test demo conference API. In a real scenario, it’s good practise to create reusable yml templates and refer the templates from your pipeline definition.

On a side note…

Whilst this blog is targeted on Azure DevOps and APIM, you can use the same methodology to test any of your APIs if you set up a Postman collection. Additionally, you can automate the same testing process in GitHub actions. You can have a look at an example here. I have used GitHub environment secrets to store the sensitive data.

Conclusion

As you can see, it’s very easy to automate API testing using Postman collection. In addition, ADO provides a visual dashboard of your test results, which is easy to analyse. Overall this approach delivers a rapid, traceable, reproducible testing methodology and it facilitates testing as part of your CI/CD process!!