From 209c4628b0cec5eee70eb1a5b5a75a13dd2143b0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vector=20Nguy=E1=BB=85n?= <80930272+vectornguyen76@users.noreply.github.com> Date: Mon, 2 Dec 2024 12:05:02 +0700 Subject: [PATCH] Update README file and smoke test --- .github/workflows/README.md | 162 +++++++++++++++++++++++++----------- .github/workflows/cd.yml | 2 +- 2 files changed, 115 insertions(+), 49 deletions(-) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 4de8e3a..47440c2 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -1,20 +1,36 @@ # CI/CD Pipeline Documentation -This document describes the Continuous Integration and Continuous Deployment (CI/CD) pipeline setup for the project. +This document describes the Continuous Integration and Continuous Deployment (CI/CD) pipeline setup for the project, implementing a blue-green deployment strategy. -## Overview +## Pipeline Overview -The project uses GitHub Actions for automated testing, building, and deployment with a blue-green deployment strategy. There are three main environments: +The project uses GitHub Actions with three deployment environments: -- Development (develop branch) -- Staging (staging branch) -- Production (master branch) +- **Development** (`develop` branch) +- **Staging** (`staging` branch) +- **Production** (`master` branch) -## Required Secrets and Variables +### Pipeline Architecture + +```mermaid +graph TD + A[Push Code] --> B[CI Pipeline] + B --> C[Unit Tests] + B --> D[Code Quality] + B --> E[Build Images] + C & D & E --> F[CD Pipeline] + F --> G[Create Infrastructure] + G --> H[Deploy Applications] + H --> I[Smoke Tests] + I --> J[Switch Traffic] + J --> K[Cleanup Old Infrastructure] +``` + +## Required Configuration ### GitHub Secrets -1. **AWS Credentials** +1. **AWS Configuration** - `AWS_ACCESS_KEY_ID`: AWS access key - `AWS_SECRET_ACCESS_KEY`: AWS secret key @@ -25,27 +41,43 @@ The project uses GitHub Actions for automated testing, building, and deployment - `DOCKERHUB_USERNAME`: Docker Hub username - `DOCKERHUB_PASSWORD`: Docker Hub password -3. **Frontend Authentication** - - `GOOGLE_CLIENT_ID`: Google OAuth client ID - - `GOOGLE_CLIENT_SECRET`: Google OAuth client secret - - `NEXTAUTH_SECRET`: NextAuth secret key +3. **Application Secrets** + - `OPENAI_API_KEY`: OpenAI API key for analysis service ### GitHub Variables 1. **AWS Resource Tags** - - `TAGS`: JSON array of AWS resource tags - ```json - [ - { "Key": "ApplicationName", "Value": "Search Engine" }, - { "Key": "Purpose", "Value": "Learning" }, - { "Key": "Project", "Value": "Search Engine" }, - { "Key": "Creator", "Value": "VectorNguyen" } - ] - ``` -## Pipeline Workflows +```json +{ + "TAGS": [ + { "Key": "ApplicationName", "Value": "Resume Ranking" }, + { "Key": "Purpose", "Value": "MVP" }, + { "Key": "Project", "Value": "Resume Ranking" }, + { "Key": "Creator", "Value": "VectorNguyen" } + ] +} +``` + +## Pipeline Components + +### 1. Continuous Integration (`ci.yml`) + +- Code quality checks using ruff +- Unit tests +- Frontend build verification + +### 2. Continuous Deployment (`cd.yml`) -### 1. Development Pipeline +- Infrastructure provisioning with CloudFormation +- Blue-green deployment implementation +- Application deployment using Ansible +- Health checks and traffic switching +- Cleanup of old infrastructure + +### 3. Environment-Specific Pipelines + +#### Development Pipeline **File:** [development_pipeline.yml](development_pipeline.yml) @@ -55,7 +87,7 @@ The project uses GitHub Actions for automated testing, building, and deployment - Run unit tests - Build Docker images -### 2. Staging Pipeline +#### Staging Pipeline **File:** [staging_pipeline.yml](staging_pipeline.yml) @@ -65,7 +97,7 @@ The project uses GitHub Actions for automated testing, building, and deployment - Deploy to staging environment - Automatic rollback on failure -### 3. Production Pipeline +#### Production Pipeline **File:** [production_pipeline.yml](production_pipeline.yml) @@ -75,39 +107,38 @@ The project uses GitHub Actions for automated testing, building, and deployment - Deploy to production environment - Automatic rollback on failure -## Deployment Process (CD Pipeline) +### 4. Rollback Process -The CD pipeline implements blue-green deployment using AWS infrastructure: +The rollback workflow ([rollback.yml](rollback.yml)) is triggered automatically if deployment fails: -1. **Infrastructure Creation** +- Identifies failed deployment stack +- Removes newly created infrastructure +- Traffic remains routed to previous stable environment - - Creates VPC, subnets, security groups - - Launches EC2 instance - - Sets up Application Load Balancer - - Configures SSL certificate +## Deployment Process -2. **Application Deployment** +### Infrastructure Creation - - Builds and pushes Docker images - - Configures EC2 instance using Ansible - - Deploys applications using Docker Compose +- Creates VPC, subnets, security groups +- Launches EC2 instance +- Sets up Application Load Balancer +- Configures SSL certificate -3. **Traffic Switch** +### Application Deployment - - Performs health checks - - Updates Route53 DNS records - - Switches traffic to new environment +- Builds and pushes Docker images +- Configures EC2 instance using Ansible +- Deploys applications using Docker Compose -4. **Cleanup** - - Removes old infrastructure after successful deployment +### Traffic Management -## Rollback Process +- Performs health checks +- Updates Route53 DNS records +- Switches traffic to new environment -The rollback workflow ([rollback.yml](rollback.yml)) is triggered automatically if deployment fails: +### Cleanup -1. Identifies failed deployment stack -2. Removes newly created infrastructure -3. Traffic remains routed to previous stable environment +- Removes old infrastructure after successful deployment ## Infrastructure as Code @@ -122,8 +153,43 @@ The infrastructure is defined using AWS CloudFormation: - SSL certificate - DNS configuration +## Usage + +### Development Workflow + +1. Create feature branch from `develop` +2. Push changes to trigger CI pipeline +3. Merge to `develop` for development deployment + +### Staging Deployment + +1. Merge `develop` to `staging` +2. Automated deployment to staging environment +3. Verify changes in staging + +### Production Deployment + +1. Create pull request to `master` +2. CI/CD pipeline runs automatically +3. Review and merge for production deployment + +## Monitoring and Maintenance + +### Health Checks + +- Application endpoint monitoring +- Infrastructure health verification +- Automatic rollback on failure + +### Cleanup + +- Automatic removal of old infrastructure +- Resource tag-based management +- Cost optimization + ## References - [GitHub Actions Documentation](https://docs.github.com/en/actions) - [AWS CloudFormation Documentation](https://docs.aws.amazon.com/cloudformation/) -- [Blue-Green Deployment](https://martinfowler.com/bliki/BlueGreenDeployment.html) +- [Blue-Green Deployment Pattern](https://martinfowler.com/bliki/BlueGreenDeployment.html) +- [Ansible Documentation](https://docs.ansible.com/) diff --git a/.github/workflows/cd.yml b/.github/workflows/cd.yml index 7683229..0dfa935 100644 --- a/.github/workflows/cd.yml +++ b/.github/workflows/cd.yml @@ -286,7 +286,7 @@ jobs: for i in $(seq 1 $MAX_RETRIES); do # Using curl with -w to get HTTP status code and -o /dev/null to discard the response body - status_code=$(curl -s -w "%{http_code}" -o /dev/null "http://${{ env.DNS_SERVER }}") + status_code=$(curl -s -w "%{http_code}" -o /dev/null "http://${{ env.DNS_SERVER }}/dashboard/jobs") if [ "$status_code" = "200" ]; then echo "Health check passed - Status code: 200"