This project deploys an architecture providing a CI/CD pipeline that will package an application into a container when code is pushed to a Git repository. It will then trigger a job on AWS Batch using the newly produced container.
This pipeline can be used to develop and test your application in the same environment you would be using in production.
This architecture has been built with the AWS CDK and make use of the following services:
- AWS CodeCommit as the repository manager.
- AWS CodeBuild to build the container.
- AWS CodePipeline to orchestrate the build pipeline.
- Amazon ECR as the container registry manager.
- AWS EventBridge to trigger job submission when containers are stored.
- AWS Batch to run the jobs.
- Amazon Virtual Private Cloud to deploy instances that will be running the jobs.
The AWS CDK stack deploys a source code repository, the pipeline with the build workflow, an Amazon ECR repository, a rule to trigger a job when a new container is stored on the repository, an AWS Batch Compute Environment as well as a Job Queue and a Job Definition and, a VPC in which instances will be residing.
To deploy this architecture, you will need to follow these steps:
- Create a virtual env and activate it
git clone https://github.com/aws-samples/aws-batch-cicd-test-pipeline.git
cd aws-batch-cicd-test-pipeline
python3 -m venv .env
source .env/bin/activate
- Install the required dependencies
pip install -r requirements.txt
- Provisioning resources for the AWS CDK before you can deploy AWS CDK apps into an AWS environment
cdk bootstrap
- Synthesize the CloudFormation template
cdk synth
- Deploy the stack
cdk deploy '*' --require-approval never
Your stack will take a few minutes to deploy. The code in the directory app-package will be uploaded and added to your AWS CodeCommit repository.
cdk destroy '*' -f
You can extend the stack to modify the following components:
- Code repository: use Github or Bitbucket.
- Build Pipeline: add other stages, for example to include static code analysis or run tests as part of the integration workflow.
- Job submission: run Array Jobs instead of a single job.
- Job Execution: replace AWS Batch by another service such as Amazon ECS, Amazon EKS or AWS ParallelCluster.
You could also integrate an Amazon Lambda function as a target of the Job Submission Rule if you want to integrate additional logic on the job submission process.
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.