AWS Codepipeline Docker Vulnerability Scan

CloudFormation template for deploying CoreOS Clair and setting up AWS CodePipeline for automated vulnerability scanning of Docker Image pushed to Amazon Elastic Container Registry (ECR).

Reference Architecture

Prerequisites

• Docker
• Git
• AWS CLI installed.
• AWS CLI is configured with IAM Access Key, Secret Access Key and Default region as US-EAST-1.

Clair Deployment

CoreOS Clair uses PostgreSQL as the database. We will use Aurora-PostgreSQL RDS Cluster to host the CoreOS Clair database. We will deploy CoreOS Clair as ECS Fargate Service behind an Application Load Balancer (ALB). The CoreOS Clair container will be deployed in a Private Subnet behind an ALB that is hosted in the Public Subnets. The Private Subnets must have a route to the internet via the NAT Gateway as CoreOS Clair will fetch the latest vulnerability information from multiple sources on the internet.

1. Build the CoreOS Clair Docker Image and push it to Elastic Container Registry (ECR).

# Create ECR Repository
aws ecr create-repository --repository-name coreos-clair \
--region us-east-1

# Build the Docker Image
cd aws-codepipeline-docker-vulnerability-scan/coreos-clair
docker build -t <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/coreos-clair:latest ./

# Push the Docker Image to ECR
aws ecr get-login --no-include-email | bash
docker push <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/coreos-clair:latest

2. Deploy CoreOS Clair using CloudFormation Template.

# Create CloudFormation Stack

aws cloudformation create-stack \
--stack-name coreos-clair-stack \
--template-body file://clair-template.yaml \
--capabilities CAPABILITY_IAM \
--parameters \
ParameterKey="VpcId",ParameterValue="<VpcId>" \
ParameterKey="PublicSubnet01",ParameterValue="<PublicSubnet01-ID>" \
ParameterKey="PublicSubnet02",ParameterValue="<PublicSubnet02-ID>" \
ParameterKey="PrivateSubnet01",ParameterValue="<PrivateSubnet01-ID>" \
ParameterKey="PrivateSubnet02",ParameterValue="<PrivateSubnet02-ID>" \
ParameterKey="ECRRepositoryUri",ParameterValue="<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/coreos-clair"

3. Note the output parameters of the above CloudFormation stack. These parameters are required for the subsequent commands.

Deploying a sample website container

In this section we will deploy a simple static website running on Nginx as a container on ECS Fargate. A CloudFormation Template is included in the sample code you cloned from GitHub. We will then setup a CodePipeline for this sample nginx website docker image. Whenever there is a change in source code, the CodePipeline will build the docker image and scan it vulnerabilities and then update the ECS service only if the vulnerabilities are below the defined threshold.

1. Build the Nginx website Docker Image and push it to Elastic Container Registry (ECR).

# Create ECR Repository
# Note the URI and ARN of the ECR Repostiory
aws ecr create-repository --repository-name nginx-website \
--region us-east-1

# Build the Docker Image
cd ../nginx-website
docker build -t <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/nginx-website:latest ./

# Push the Docker Image to ECR
aws ecr get-login --no-include-email | bash
docker push <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/nginx-website:latest

2. We will now deploy the Nginx website as an ECS Service using the Fargate deployment mode. A CloudFormation template is provided to achieve this step.

# Create CloudFormation Stack
# <ECRRepositoryUri> - Nginx-Website ECR Repository URI without Image tag
# <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/nginx-website

aws cloudformation create-stack \
--stack-name nginx-website-stack \
--template-body file://nginx-website-template.yaml \
--capabilities CAPABILITY_IAM \
--parameters \
ParameterKey="VpcId",ParameterValue="<VpcId>" \
ParameterKey="PublicSubnet01",ParameterValue="<PublicSubnet01-ID>" \
ParameterKey="PublicSubnet02",ParameterValue="<PublicSubnet02-ID>" \
ParameterKey="PrivateSubnet01",ParameterValue="<PrivateSubnet01-ID>" \
ParameterKey="PrivateSubnet02",ParameterValue="<PrivateSubnet02-ID>" \
ParameterKey="ECRRepositoryUri",ParameterValue="<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/nginx-website"

3. Note the output parameters of the above CloudFormation stack. These parameters are required for the subsequent commands.

AWS CodePipeline

In this section we will build a CodePipeline to automate the vulnerability scanning of future Nginx-Website docker image builds. The nginx-website folder includes a buildspec.yml file that provides build instructions to CodeBuild.

We will be using Klar, a simple tool to analyse images stored in a private or public Docker registry for security vulnerabilities using CoreOS Clair. Klar serves as a client which coordinates the image checks between the ECR and Clair.

The buildspec.yml we have set the CLAIR_OUTPUT=Critical. CLAIR_OUTPUT variable defines the severity level threshold, vulnerabilities with severity level higher than or equal to this threshold will be outputted. Supported levels are Unknown, Negligible, Low, Medium, High, Critical, Defcon1. You can configure Klar to your requirements by setting the variables as defined in the Klar documentation.

1. Deploy the CodePipeline using CloudFormation Template.

# Deploy CodePipeline 
# Replace the Variables below
# ClairAlbDnsName - Output variable from coreos-clair-stack
# WebsiteECSServiceName - Output variable from nginx-website-stack

aws cloudformation create-stack \
--stack-name nginx-website-codepipeline-stack \
--template-body file://clair-codepipeline-template.yaml \
--capabilities CAPABILITY_IAM \
--disable-rollback \
--parameters \
ParameterKey="CodeBuildProjectName",ParameterValue="nginx-codeBuild-project" \
ParameterKey="CodeCommitRepoName",ParameterValue="nginx-website-repo" \
ParameterKey="EcrRepositoryArn",ParameterValue="<nginx-website-ECR-Repo-ARN>" \
ParameterKey="EcrRepositoryUri",ParameterValue="<nginx-website-ECR-Repo-URI>" \
ParameterKey="ClairAlbUrl",ParameterValue="<ClairAlbDnsName>" \
ParameterKey="EcsServiceName",ParameterValue="<WebsiteECSServiceName>"

2. Trigger the above CodePipeline by pushing a change to the CodeCommit Repository.

# Clone the CodeCommit Repo to the Workstation
git clone <CodeCommitRepoSshUrl>

# Copy the aws-codepipeline-docker-vulnerability-scan/nginx-website to the cloned folder
cp -Rv nginx-website/ nginx-website-repo/

# Commit the copied files to the CodeCommit Repository
cd nginx-website-repo
git add *
git commit -m "Initial Commit"
git push

Once the source code has been pushed to the CodeCommit Repository, the CodePipeline is triggered. You can login to your AWS console to monitor the status of the CodePipeline. The Vulnerability scan information is available in the CodeBuild CloudWatch Logs.

You can also modify the CLAIR_OUTPUT from Critical to High in the buildspec.yml in the cores-clair-ecs-cicd/nginx-website-repo folder and check the status of the build.

The CoreOS Clair we deployed can used as a centralized Docker Image vulnerability scanner and used by other CodeBuild projects.

License Summary

This sample code is made available under a modified MIT license. See the LICENSE file.