Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FS-4528: Build and deploy to AWS environment #16

Merged
merged 22 commits into from
Aug 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/workflows/copilot_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
with:
version_to_build: sha-${{ github.sha }}
owner: ${{ github.repository_owner }}
application: funding-service-design-self-serve
application: funding-service-design-fund-application-builder
assets_required: false

dev_deploy:
Expand All @@ -74,7 +74,7 @@ jobs:
AWS_ACCOUNT: ${{ secrets.AWS_ACCOUNT }}
with:
environment: dev
app_name: self-serve
app_name: fund-application-builder
version: sha-${{ github.sha }}

post_dev_deploy_tests:
Expand All @@ -90,7 +90,7 @@ jobs:
run_performance_tests: ${{ inputs.run_performance_tests || true }}
run_e2e_tests_assessment: ${{ inputs.run_e2e_tests_assessment || false }}
run_e2e_tests_application: ${{ inputs.run_e2e_tests_application || false }}
app_name: self-serve
app_name: fund-application-builder
environment: dev

test_deploy:
Expand All @@ -101,7 +101,7 @@ jobs:
AWS_ACCOUNT: ${{ secrets.AWS_ACCOUNT }}
with:
environment: test
app_name: self-serve
app_name: fund-application-builder
version: sha-${{ github.sha }}

post_test_deploy_tests:
Expand All @@ -118,7 +118,7 @@ jobs:
run_performance_tests: ${{ inputs.run_performance_tests || false }}
run_e2e_tests_assessment: ${{ inputs.run_e2e_tests_assessment || false }}
run_e2e_tests_application: ${{ inputs.run_e2e_tests_application || true }}
app_name: self-serve
app_name: fund-application-builder
environment: test

uat_deploy:
Expand All @@ -129,7 +129,7 @@ jobs:
AWS_ACCOUNT: ${{ secrets.AWS_ACCOUNT }}
with:
environment: uat
app_name: self-serve
app_name: fund-application-builder
version: sha-${{ github.sha }}

post_uat_deploy_tests:
Expand All @@ -146,7 +146,7 @@ jobs:
run_performance_tests: ${{ inputs.run_performance_tests || false }}
run_e2e_tests_assessment: ${{ inputs.run_e2e_tests_assessment || false }}
run_e2e_tests_application: ${{ inputs.run_e2e_tests_application || true }}
app_name: self-serve
app_name: fund-application-builder
environment: uat

prod_deploy:
Expand All @@ -157,5 +157,5 @@ jobs:
AWS_ACCOUNT: ${{ secrets.AWS_ACCOUNT }}
with:
environment: prod
app_name: self-serve
app_name: fund-application-builder
version: sha-${{ github.sha }}
19 changes: 15 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,23 @@
###############################################################################

FROM python:3.10-bullseye as fab-dev

ARG USE_DEV_REQUIREMENTS

WORKDIR /app

COPY requirements.txt requirements.txt
COPY requirements-dev.txt requirements-dev.txt

RUN if "$USE_DEV_REQUIREMENTS"; then \
echo "Installing development dependencies..." && \
python3 -m pip install --upgrade pip && pip install -r requirements-dev.txt; \
else \
echo "Installing production dependencies..." && \
python3 -m pip install --upgrade pip && pip install -r requirements.txt; \
fi

COPY . .
RUN apt-get update && apt-get install -y postgresql-client

RUN python3 -m pip install --upgrade pip && pip install pip-tools && pip install -r requirements.txt
RUN python3 -m pip install -r requirements-dev.txt
EXPOSE 8080

CMD ["flask", "run", "--port", "8080", "--host", "0.0.0.0"]
1 change: 1 addition & 0 deletions Procfile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
web: gunicorn wsgi:app -c run/gunicorn/devtest.py
1 change: 1 addition & 0 deletions app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import app # noqa
1 change: 1 addition & 0 deletions app/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from app.create_app import app # noqa
2 changes: 1 addition & 1 deletion app/app.py → app/create_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,4 +59,4 @@ def create_app() -> Flask:
return flask_app


app = create_app()
app = create_app()
14 changes: 5 additions & 9 deletions config/__init__.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,15 @@
# flake8: noqa
from os import environ
from os import getenv

FLASK_ENV = environ.get("FLASK_ENV")
FLASK_ENV = getenv("FLASK_ENV")

match FLASK_ENV:
case "tasks" | "development":
case "development":
from config.envs.development import DevelopmentConfig as Config
# case "dev":
# from config.envs.dev import DevConfig as Config
# case "test":
# from config.envs.test import TestConfig as Config
case "dev":
from config.envs.dev import DevConfig as Config
case "unit_test":
from config.envs.unit_test import UnitTestConfig as Config
# case "uat" | "production":
# from config.envs.production import ProductionConfig as Config
case _:
from config.envs.default import DefaultConfig as Config

Expand Down
14 changes: 14 additions & 0 deletions config/envs/dev.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
"""Flask configuration."""
import logging
from os import environ

from config.envs.default import DefaultConfig
from fsd_utils import configclass


@configclass
class DevConfig(DefaultConfig):
FSD_LOGGING_LEVEL = logging.INFO
SQLALCHEMY_TRACK_MODIFICATIONS = False

SQLALCHEMY_DATABASE_URI = environ.get("DATABASE_URL").replace("postgres://", "postgresql://")
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@ Parameters:
Type: String
Description: The name of the service, job, or workflow being deployed.
# Customize your Aurora Serverless cluster by setting the default value of the following parameters.
fsdselfserveclusterDBName:
fsdfundapplicationbuilderclusterDBName:
Type: String
Description: The name of the initial database to be created in the Aurora Serverless v2 cluster.
Default: fsd_self_serve
Default: fsd_application_builder
# Cannot have special characters
# Naming constraints: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html#RDS_Limits.Constraints
Mappings:
fsdselfserveclusterEnvScalingConfigurationMap:
fsdfundapplicationbuilderclusterEnvScalingConfigurationMap:
All:
"DBMinCapacity": 0.5 # AllowedValues: from 0.5 through 128
"DBMaxCapacity": 8 # AllowedValues: from 0.5 through 128
Expand All @@ -31,27 +31,27 @@ Mappings:
"SecurityGroup": "sg-08cecea8f9b8a4ec9"

Resources:
fsdselfserveclusterDBSubnetGroup:
fsdfundapplicationbuilderclusterDBSubnetGroup:
Type: 'AWS::RDS::DBSubnetGroup'
Properties:
DBSubnetGroupDescription: Group of Copilot private subnets for Aurora Serverless v2 cluster.
SubnetIds:
!Split [',', { 'Fn::ImportValue': !Sub '${App}-${Env}-PrivateSubnets' }]
fsdselfserveclusterSecurityGroup:
fsdfundapplicationbuilderclusterSecurityGroup:
Metadata:
'aws:copilot:description': 'A security group for your workload to access the Aurora Serverless v2 cluster fsdselfservecluster'
'aws:copilot:description': 'A security group for your workload to access the Aurora Serverless v2 cluster fsdfundapplicationbuildercluster'
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: !Sub 'The Security Group for ${Name} to access Aurora Serverless v2 cluster fsdselfservecluster.'
GroupDescription: !Sub 'The Security Group for ${Name} to access Aurora Serverless v2 cluster fsdfundapplicationbuildercluster.'
VpcId:
Fn::ImportValue:
!Sub '${App}-${Env}-VpcId'
Tags:
- Key: Name
Value: !Sub 'copilot-${App}-${Env}-${Name}-Aurora'
fsdselfserveclusterDBClusterSecurityGroup:
fsdfundapplicationbuilderclusterDBClusterSecurityGroup:
Metadata:
'aws:copilot:description': 'A security group for your Aurora Serverless v2 cluster fsdselfservecluster'
'aws:copilot:description': 'A security group for your Aurora Serverless v2 cluster fsdfundapplicationbuildercluster'
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: The Security Group for the Aurora Serverless v2 cluster.
Expand All @@ -60,7 +60,7 @@ Resources:
FromPort: 5432
IpProtocol: tcp
Description: !Sub 'From the Aurora Security Group of the workload ${Name}.'
SourceSecurityGroupId: !Ref fsdselfserveclusterSecurityGroup
SourceSecurityGroupId: !Ref fsdfundapplicationbuilderclusterSecurityGroup
- ToPort: 5432
FromPort: 5432
IpProtocol: tcp
Expand All @@ -72,7 +72,7 @@ Resources:
Tags:
- Key: Name
Value: !Sub 'copilot-${App}-${Env}-${Name}-Aurora'
fsdselfserveclusterAuroraSecret:
fsdfundapplicationbuilderclusterAuroraSecret:
Metadata:
'aws:copilot:description': 'A Secrets Manager secret to store your DB credentials'
Type: AWS::SecretsManager::Secret
Expand All @@ -84,7 +84,7 @@ Resources:
ExcludePunctuation: true
IncludeSpace: false
PasswordLength: 16
fsdselfserveclusterDBClusterParameterGroup:
fsdfundapplicationbuilderclusterDBClusterParameterGroup:
Metadata:
'aws:copilot:description': 'A DB parameter group for engine configuration values'
Type: 'AWS::RDS::DBClusterParameterGroup'
Expand All @@ -93,35 +93,35 @@ Resources:
Family: 'aurora-postgresql14'
Parameters:
client_encoding: 'UTF8'
fsdselfserveclusterDBCluster:
fsdfundapplicationbuilderclusterDBCluster:
Metadata:
'aws:copilot:description': 'The fsdselfservecluster Aurora Serverless v2 database cluster'
'aws:copilot:description': 'The fsdfundapplicationbuildercluster Aurora Serverless v2 database cluster'
Type: 'AWS::RDS::DBCluster'
Properties:
MasterUsername:
!Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdselfserveclusterAuroraSecret, ":SecretString:username}}" ]] # pragma: allowlist secret
!Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdfundapplicationbuilderclusterAuroraSecret, ":SecretString:username}}" ]] # pragma: allowlist secret
MasterUserPassword:
!Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdselfserveclusterAuroraSecret, ":SecretString:password}}" ]] # pragma: allowlist secret
DatabaseName: !Ref fsdselfserveclusterDBName
!Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdfundapplicationbuilderclusterAuroraSecret, ":SecretString:password}}" ]] # pragma: allowlist secret
DatabaseName: !Ref fsdfundapplicationbuilderclusterDBName
Engine: 'aurora-postgresql'
EngineVersion: '14.4'
DBClusterParameterGroupName: !Ref fsdselfserveclusterDBClusterParameterGroup
DBSubnetGroupName: !Ref fsdselfserveclusterDBSubnetGroup
DBClusterParameterGroupName: !Ref fsdfundapplicationbuilderclusterDBClusterParameterGroup
DBSubnetGroupName: !Ref fsdfundapplicationbuilderclusterDBSubnetGroup
Port: 5432
StorageEncrypted: true
BackupRetentionPeriod: 8
VpcSecurityGroupIds:
- !Ref fsdselfserveclusterDBClusterSecurityGroup
- !Ref fsdfundapplicationbuilderclusterDBClusterSecurityGroup
ServerlessV2ScalingConfiguration:
# Replace "All" below with "!Ref Env" to set different autoscaling limits per environment.
MinCapacity: !FindInMap [fsdselfserveclusterEnvScalingConfigurationMap, All, DBMinCapacity]
MaxCapacity: !FindInMap [fsdselfserveclusterEnvScalingConfigurationMap, All, DBMaxCapacity]
fsdselfserveclusterDBWriterInstance:
MinCapacity: !FindInMap [fsdfundapplicationbuilderclusterEnvScalingConfigurationMap, All, DBMinCapacity]
MaxCapacity: !FindInMap [fsdfundapplicationbuilderclusterEnvScalingConfigurationMap, All, DBMaxCapacity]
fsdfundapplicationbuilderclusterDBWriterInstance:
Metadata:
'aws:copilot:description': 'The fsdselfservecluster Aurora Serverless v2 writer instance'
'aws:copilot:description': 'The fsdfundapplicationbuildercluster Aurora Serverless v2 writer instance'
Type: 'AWS::RDS::DBInstance'
Properties:
DBClusterIdentifier: !Ref fsdselfserveclusterDBCluster
DBClusterIdentifier: !Ref fsdfundapplicationbuilderclusterDBCluster
DBInstanceClass: db.serverless
Engine: 'aurora-postgresql'
PromotionTier: 1
Expand All @@ -130,11 +130,11 @@ Resources:
- !GetAZs
Ref: AWS::Region

fsdselfserveclusterSecretAuroraClusterAttachment:
fsdfundapplicationbuilderclusterSecretAuroraClusterAttachment:
Type: AWS::SecretsManager::SecretTargetAttachment
Properties:
SecretId: !Ref fsdselfserveclusterAuroraSecret
TargetId: !Ref fsdselfserveclusterDBCluster
SecretId: !Ref fsdfundapplicationbuilderclusterAuroraSecret
TargetId: !Ref fsdfundapplicationbuilderclusterDBCluster
TargetType: AWS::RDS::DBCluster

FormRunnerFormUploadsBucketAccessPolicy:
Expand Down Expand Up @@ -166,18 +166,18 @@ Outputs:
Value:
!Sub
- "postgres://${USERNAME}:${PASSWORD}@${HOSTNAME}:${PORT}/${DBNAME}"
- USERNAME: !Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdselfserveclusterAuroraSecret, ":SecretString:username}}" ]] # pragma: allowlist secret
PASSWORD: !Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdselfserveclusterAuroraSecret, ":SecretString:password}}" ]] # pragma: allowlist secret
HOSTNAME: !Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdselfserveclusterAuroraSecret, ":SecretString:host}}" ]] # pragma: allowlist secret
PORT: !Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdselfserveclusterAuroraSecret, ":SecretString:port}}" ]] # pragma: allowlist secret
DBNAME: !Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdselfserveclusterAuroraSecret, ":SecretString:dbname}}" ]] # pragma: allowlist secret
- USERNAME: !Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdfundapplicationbuilderclusterAuroraSecret, ":SecretString:username}}" ]] # pragma: allowlist secret
PASSWORD: !Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdfundapplicationbuilderclusterAuroraSecret, ":SecretString:password}}" ]] # pragma: allowlist secret
HOSTNAME: !Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdfundapplicationbuilderclusterAuroraSecret, ":SecretString:host}}" ]] # pragma: allowlist secret
PORT: !Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdfundapplicationbuilderclusterAuroraSecret, ":SecretString:port}}" ]] # pragma: allowlist secret
DBNAME: !Join [ "", [ '{{resolve:secretsmanager:', !Ref fsdfundapplicationbuilderclusterAuroraSecret, ":SecretString:dbname}}" ]] # pragma: allowlist secret

fsdselfserveclusterSecret: # injected as FSDselfserveCLUSTER_SECRET environment variable by Copilot.
fsdfundapplicationbuilderclusterSecret: # injected as FSDfundapplicationbuilderCLUSTER_SECRET environment variable by Copilot.
Description: "The JSON secret that holds the database username and password. Fields are 'host', 'port', 'dbname', 'username', 'password', 'dbClusterIdentifier' and 'engine'"
Value: !Ref fsdselfserveclusterAuroraSecret
fsdselfserveclusterSecurityGroup:
Value: !Ref fsdfundapplicationbuilderclusterAuroraSecret
fsdfundapplicationbuilderclusterSecurityGroup:
Description: "The security group to attach to the workload."
Value: !Ref fsdselfserveclusterSecurityGroup
Value: !Ref fsdfundapplicationbuilderclusterSecurityGroup
FormRunnerFormUploadsBucketAccessPolicyArn:
Description: "The ARN of the ManagedPolicy to attach to the task role."
Value: !Ref FormRunnerFormUploadsBucketAccessPolicy
Loading