Skip to content

[Tools] Add scripts to facilitate 1-click deployment of PCUI in personal environment #291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 23, 2023
Merged

[Tools] Add scripts to facilitate 1-click deployment of PCUI in personal environment #291

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion scripts/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,24 @@ bash ./rollforward_awslambda_image.sh

**NOTE:** The rollforward script should be used only if the rollback script is launched by mistake, the normal procedure to publish a new release after a rollback is by launching the `build_and_release_image.sh --tag YYYY.MM.REVISION` after fixing the code

# How to deploy PCUI image in personal account
# How to deploy PCUI in personal account
Create configurations files for your personal environment (una tantum):

```
bash scripts/setup-env.sh [ENVIRONMENT_NAME]
```

where *ENVIRONMENT_NAME* could be your login.

Adapt the configurations files `[ENVIRONMENT_NAME]-*` created in `infrastructure/environments` with your data.

Deploy the local PCUI to your environment:

```
bash scripts/deploy.sh [ENVIRONMENT_NAME]
```

# How to deploy the PCUI image to a private ECR repository
Create an ECR *private* repository in your personal account, where PCUI Docker images will be stored for testing:

```
Expand Down
13 changes: 13 additions & 0 deletions scripts/common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/bash

function info() {
echo "[INFO] $1"
}

function warn() {
echo "[WARN] $1"
}

function fail() {
echo "[ERROR] $1" && exit 1
}
125 changes: 125 additions & 0 deletions scripts/deploy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
#!/bin/bash
set -e
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
# with the License. A copy of the License is located at
#
# http://aws.amazon.com/apache2.0/
#
# or in the "LICENSE.txt" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
# OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and
# limitations under the License.

# This script is used to update the infrastructure of a given PCM environment.
# An environment is composed of a list of variables with the entrypoints of the environment
# and a CloudFormation request file where the stack update can be customized,
# for example by changing the parameters provided to the previous version of the stack
#
# Usage: ./scripts/deploy.sh [ENVIRONMENT]
# Example: ./scripts/deploy.sh demo

CURRENT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" &> /dev/null && pwd)"

INFRASTRUCTURE_DIR="$CURRENT_DIR/../infrastructure"

source "$CURRENT_DIR/common.sh"

ENVIRONMENT=$1

[[ -z $ENVIRONMENT ]] && fail "Missing required argument: ENVIRONMENT"

info "Selected environment: $ENVIRONMENT"

source "$INFRASTRUCTURE_DIR/environments/$ENVIRONMENT-variables.sh"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: it's useful to use {} when variables are used to compose a string

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK it's useful only when you're putting variables isde by side, e.g. "$VAR_1$VAR_2" => "${VAR_1}${VAR_2}". In this case what's the advantage?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In your case there are no differences because after $ENVIRONMENT you have a -, but in general it's worth using curly braces. Let's think to this simple example:

$ test=one
$ echo "$test1"

$ echo "${test}1"
one1

The result of the first echo is an empty string, because $test1 would expand the variable identified by test1 rather than test. This kind of issues are hard to discover, so as a best practice is better to use curly braces everywhere.


# STEP: Upload templates
info "Retrieving infrastructure bucket"
if [[ -n $INFRA_BUCKET_NAME ]]; then
BUCKET=$INFRA_BUCKET_NAME
else
BUCKET=$(aws cloudformation describe-stack-resources \
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

useful to add region parameter in every aws call.

--region "$REGION" \
--stack-name "$INFRA_BUCKET_STACK_NAME" \
--logical-resource-id InfrastructureBucket \
--output text \
--query 'StackResources[0].PhysicalResourceId')
fi
info "Using infrastructure bucket $BUCKET"

info "Uploading templates to infrastructure bucket $BUCKET"
FILES=(parallelcluster-ui-cognito.yaml parallelcluster-ui.yaml)
for FILE in "${FILES[@]}"; do
aws s3 cp --region "$REGION" "$INFRASTRUCTURE_DIR/$FILE" "s3://$BUCKET/$FILE"
done
# -----------------------------

# STEP: Setup private ECR repository
ECR_REPOSITORY_NAME="parallelcluster-ui"
info "Settings up private ECR repository $ECR_REPOSITORY_NAME"
if [[ ! $(aws ecr describe-repositories --repository-name $ECR_REPOSITORY_NAME --region "$REGION" 2>/dev/null) ]]; then
info "The private ECR repository $ECR_REPOSITORY_NAME does not exist, creating ..."
aws ecr create-repository \
--region "$REGION" \
--repository-name $ECR_REPOSITORY_NAME \
--image-tag-mutability MUTABLE

else
info "The private ECR repository $ECR_REPOSITORY_NAME already exists"
fi
REPOSITORY_INFO=$(aws ecr describe-repositories \
--region "$REGION" \
--repository-name $ECR_REPOSITORY_NAME \
--query "repositories[0].{arn:repositoryArn,uri:repositoryUri}" \
--output json)
REPOSITORY_ARN=$(echo "$REPOSITORY_INFO" | jq -cr '.arn')
REPOSITORY_ENDPOINT=$(echo "$REPOSITORY_INFO" | jq -cr '.uri' | cut -d '/' -f 1)
info "Using private ECR repository $REPOSITORY_ARN at ECR endpoint $REPOSITORY_ENDPOINT"
# -----------------------------

# STEP: Build image locally and publish to private ECR repository
info "Building image locally and publishing it to private ECR repository"
ECR_IMAGE_TAG=$(date +%Y.%m.%d)
bash "$CURRENT_DIR/build_and_release_image.sh" --ecr-region "$REGION" --ecr-endpoint "$REPOSITORY_ENDPOINT" --tag "$ECR_IMAGE_TAG"
info "Image built with tag '$ECR_IMAGE_TAG' and 'latest'"
# -----------------------------

# STEP: Deploy stack
info "Deploying stack $STACK_NAME"
BUCKET_URL="https://$BUCKET.s3.$REGION.amazonaws.com"

if [[ ! $(aws cloudformation describe-stacks --stack-name "$STACK_NAME" --region "$REGION" 2>/dev/null) ]]; then
info "The stack $STACK_NAME does not exist, creating ..."
CFN_DEPLOY_COMMAND="create-stack"
CFN_WAIT_COMMAND="stack-create-complete"
CFN_CLI_INPUT_YAML_FILE="$INFRASTRUCTURE_DIR/environments/$ENVIRONMENT-cfn-create-args.yaml"

else
info "The stack $STACK_NAME exists, updating ..."
CFN_DEPLOY_COMMAND="update-stack"
CFN_WAIT_COMMAND="stack-update-complete"
CFN_CLI_INPUT_YAML_FILE="$INFRASTRUCTURE_DIR/environments/$ENVIRONMENT-cfn-update-args.yaml"
fi

CLI_INPUT_YAML=$(sed "s#BUCKET_URL_PLACEHOLDER#$BUCKET_URL#g" "$CFN_CLI_INPUT_YAML_FILE")

AWS_PAGER="cat" aws cloudformation $CFN_DEPLOY_COMMAND \
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to pass cat or can we use AWS_PAGER=""?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to keep cat at least for the time being because I took that from a script used to update the demo environment from github.
The idea is to have a unique 1click deployment script that works for both personal env and demo env, so I suggest to keep it as is for the time being.

--cli-input-yaml "$CLI_INPUT_YAML" \
--stack-name "$STACK_NAME" \
--region "$REGION"

aws cloudformation wait $CFN_WAIT_COMMAND \
--stack-name "$STACK_NAME" \
--region "$REGION"

info "Deployment completed!"
# -----------------------------

# STEP: Retrieve stack details
PCUI_ENDPOINT=$(aws cloudformation describe-stacks \
--stack-name "$STACK_NAME" \
--region "$REGION" \
--query "Stacks[0].Outputs[?OutputKey == 'ParallelClusterUIUrl'].OutputValue" \
--output text)

info "PCUI endpoint is $PCUI_ENDPOINT"
45 changes: 45 additions & 0 deletions scripts/setup-env.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#!/bin/bash
set -e
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
# with the License. A copy of the License is located at
#
# http://aws.amazon.com/apache2.0/
#
# or in the "LICENSE.txt" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
# OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and
# limitations under the License.

# This script is used to update the infrastructure of a given PCM environment.
# An environment is composed of a list of variables with the entrypoints of the environment
# and a CloudFormation request file where the stack update can be customized,
# for example by changing the parameters provided to the previous version of the stack
#
# Usage: ./scripts/setup-env.sh [ENVIRONMENT]
# Example: ./scripts/setup-env.sh demo

CURRENT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"

ENVIRONMENTS_DIR=$(realpath "$CURRENT_DIR/../infrastructure/environments")

source "$CURRENT_DIR/common.sh"

ENVIRONMENT=$1

[[ -z $ENVIRONMENT ]] && fail "Missing required argument: ENVIRONMENT"

info "Selected environment: $ENVIRONMENT"

if [[ $(ls $ENVIRONMENTS_DIR/$ENVIRONMENT-*) ]]; then
warn "Configuration files for the environment $ENVIRONMENT already exist. Nothing to do."
exit 0
fi

info "Creating environment files"
cp "$ENVIRONMENTS_DIR/demo-variables.sh" "$ENVIRONMENTS_DIR/$ENVIRONMENT-variables.sh"
chmod +x "$ENVIRONMENTS_DIR/$ENVIRONMENT-variables.sh"
cp "$ENVIRONMENTS_DIR/demo-cfn-update-args.yaml" "$ENVIRONMENTS_DIR/$ENVIRONMENT-cfn-create-args.sh"
cp "$ENVIRONMENTS_DIR/demo-cfn-update-args.yaml" "$ENVIRONMENTS_DIR/$ENVIRONMENT-cfn-update-args.sh"

info "Environment files created! Check out $ENVIRONMENTS_DIR"