How to Run OWASP ZAP Security Tests in Azure DevOps Pipeline

Security Testing is an essential part of testing, every organization wants to do at least basic security testing before releasing the code to production. Security Testing is an ocean it might be difficult to perform complete security testing without the help of trained professionals. Some of the open-source tools provide automated basic scanning of the website. Once we add it to pipelines like any other test such as Smoke, Regression the security tests also can run as part of deployment and report issues.

What is OWASP ZAP?

ZAP is a popular security testing tool and open source. ZAP tool helps to find the vulnerabilities in the applications or API endpoints. Vulnerabilities include cross-site scripting, SQL injection, Broken Authentication, Sensitive data exposure, Broken Access control, Security misconfiguration, Insecure Deserialization, etc. 

The beauty of this tool is that it provides both UI and Command Line interfaces to run the tests. Since it provides a command-line interface we can integrate it as part of our pipeline. The pipeline can be triggered when we release code into production, this helps to find the potential security issues.

In this detailed tutorial we are going to learn:

1. How to configure and set up  OWASP ZAP  Security test into Azure Release Pipeline?

2. How to Run OWASP ZAP Security Tests on Websites in Azure DevOps Pipeline using Docker?

3. How to Perform API Security Testing using OWASP ZAP Security Testing tool in Azure DevOps Pipelines with Docker Images?

4. How to Publish OWASP ZAP Security Testing Results in Azure DevOps Pipeline?

5. How to Publish OWASP ZAP HTML Test Results into Azure Artifacts by Creating Feed and Packages?

6. How to Download artifacts containing OWASP ZAP HTML Test results using Azure CLI Tool?

Pre-Requisite:

1. Create a Repository inside your organization (Preferred), download the file OWASPToNUnit3.xslt, and keep it inside the repository. This file is needed to convert OWASP ZAP Security Test result XML file to publish results in Azure DevOps.
2. Create a Feed Azure DevOps artifacts: This feed is helpful to publish OWASP ZAP HTML Result

         Steps to Create a Feed in Azure DevOps

1. Navigate to Azure DevOps > Click on Artifacts > Click on Create Feed
2.  In the Create new Feed form Enter correct text, and Click on Create
   Note: We will be using feed name while configuring tasks, You need to choose the same from drop-down so note down the Feed Name 

   3. Create a Sample package inside Feed using Command Line

        Steps to Create Sample Package:

1. Install Azure CLI
2. After Installation Run the below command to create sample package 

az artifacts universal publish - -organization https://dev.azure.com/[Your_Org_Name] --feed SecurityTesting --name security_testing --version 1.0.0 --description "Your description" --path .

After completion of Pre-Requisite 3, Navigate to Azure DevOps > Artifact > Select Feed as SecutityTesting > Then you should see the newly created package 

Refer to Microsoft Documentation for more details.

We have completed all initial setup and pre-requisites, we are good to start with pipelines now.

How to Configure OWASP ZAP Security Tests in Azure DevOps Pipeline?

Let's discuss in detail step by step by setting up OWASP ZAP Security Tests Pipeline using Docker Image.

Step 1: Create a new Release Pipeline

i. Navigate to Azure DevOps > Pipeline > Click on Releases

                  ii. Click on New and Choose New Release Pipeline

                 iii.  Choose Empty job when Template window prompts

               iv. Name the stage as Security Teting (or any other name you wish)

Step 2: Add artifact to Release Pipeline

           i. Click on add artifact

          ii. In the Popup window Choose Azure Repository

          iii. Choose your project 

          iv. Choose Source Repository ( This is the place where you created XSLT file in Pre-Requisite)

          v. Choose the default branch as master 

          vi. Click Add 

Step 3: Add Tasks to Pipeline

We need to add tasks to the pipeline, In our case, we have created only one Stage That is Security Testing

Step 4: Configure Agent Job Details

i. Display Name: Agent Job or anything you wish
ii. Agent pool: Choose Azure Pipelines
iii. Agent Specification: Choose any ubuntu Agent from the dropdown

Step 5: Add Docker Installer Task

From Search box, Search for Docker CLI and Add the task and Configure Docker CLI Task

Step 6: Add Bash Script Task

Step 7: Configure Bash Script Task

Enter Display name: Security Test Run

Type: click on Inline Radio Button

Script: Copy paste the below code, (Don't forget to replace your url )

chmod -R 777  ./
docker run --rm -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-full-scan.py -t [your company url] -g gen.conf -x OWASP-ZAP-Report.xml -r scan-report.html
true

Example:

chmod -R 777  ./

docker run --rm -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-full-scan.py -t https://dzone.com -g gen.conf -x OWASP-ZAP-Report.xml -r scan-report.html

How to run OWASP ZAP Security Test for API?

The above-mentioned script works well with websites, webpages but if your requirement is API then you need to add different inline scripts rest of the things remain the same.

Script for OWASP ZAP API Security Scan

chmod -R 777 ./
docker run — rm -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-weekly zap-api-scan.py -t [your-api-url] -f openapi -g api-scan.conf -x OWASP-ZAP-Report.xml -r api-scan-report.html
true

Example:

chmod -R 777 ./

docker run — rm -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-weekly zap-api-scan.py -t https://dzone.com/swagger/v1/swagger.json -f openapi -g api-scan.conf -x OWASP-ZAP-Report.xml -r api-scan-report.html

true

So, Thanks to sudhinsureshr for this.

Step 8: Add Powershell Task to convert ZAP XML Report to Azure DevOps Nunit Report Format to Publish Results

Add PowerShell Task using Add Azure DevOps Add Tasks Window

Configure Powershell Task, Convert ZAP XML to NUnit XML

i. Display Name: Anything you wish
ii. Type: Inline
iii. Script: Inline

Sample Inline Script

Note: This script contains a Relative path to the repository and folder, the content of the script may change based on the name you specified in your project.

$XslPath = "$($Env:SYSTEM_DEFAULTWORKINGDIRECTORY)/_Quality/SecurityTesting/OWASPToNUnit3.xslt"
$XmlInputPath = "$($Env:SYSTEM_DEFAULTWORKINGDIRECTORY)/OWASP-ZAP-Report.xml"
$XmlOutputPath = "$($Env:SYSTEM_DEFAULTWORKINGDIRECTORY)/Converted-OWASP-ZAP-Report.xml"
$XslTransform = New-Object System.Xml.Xsl.XslCompiledTransform
$XslTransform.Load($XslPath)
$XslTransform.Transform($XmlInputPath, $XmlOutputPath)

Step 9: [Optional] Publish OWASP ZAP Security Testing HTML Results to Azure Artifact

Add Universal Package Task

Configure Universal Package Task

Step 10: Publish OWASP ZAP Results into Azure DevOps Pipeline

Add Publish Results Task

Configure Publish Results Task

Display Name: Any name

Test Result format: Nunit

Test Result Files: Output file name in Step 8, In our Case it's Converted-OWASP-ZAP-Report.xml

Search Folder: $(System.DefaultWorkingDirectory) 

After completion of Step 10, Trigger Azure OWASP ZAP Release. The release starts running shows the progress in the command line.

Step 11: Viewing OWASP / ZAP Security Testing Results

Once Release is completed, Navigate to completed tasks, Click on Publish Test Results Task

The window with the link to result opens

Once you click the link you can see the results.

Additional Resouces for Leaning:

ZAP API Scan

ZAP Full Scan

ZAP API Scan

If you are looking for any help, support, guidance contact me on LinkedIn|https://www.linkedin.com/in/ganeshsirsi