From 94cae1dc8d957a34d8c18908e99fec99b1de12f1 Mon Sep 17 00:00:00 2001 From: Joni Harker Date: Mon, 18 Nov 2024 15:00:34 -0800 Subject: [PATCH 1/3] [IT-3523] Document current process for testing AMIs Document the current manual process used for testing changes to AMIs used by service catalog products. --- sceptre/scipool/README.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/sceptre/scipool/README.md b/sceptre/scipool/README.md index 0f807950..5980fe07 100644 --- a/sceptre/scipool/README.md +++ b/sceptre/scipool/README.md @@ -31,6 +31,8 @@ Requirements: * Run `pre-commit install` to install the git hook. ## Testing + +### Pre-Commit As a pre-deployment step we syntatically validate our sceptre and cloudformation yaml files with [pre-commit](https://pre-commit.com). @@ -40,6 +42,30 @@ execute the validations by running `pre-commit run --all-files`. Please install pre-commit, once installed the file validations will automatically run on every commit. +### Functional Testing +The process to test the functionality of an AMI and it's integration with our +Service Catalog products is to first manually create a new AMI, then manually +upload a new product template to S3, and finally create a test product in +Service Catalog through a pull request to `organizations-infra`: + +1. Make local changes to the packer repo to update or modify the AMI. +1. Run packer manually with admin credentials for the `itsandbox` account + to create a test AMI. +1. Create an EC2 instance in the `itsandbox` account from the test AMI for any + initial system validation. +1. Make local changes to `service-catalog-library` to update the desired template + to reference the test AMI. +1. Manually upload the test template to S3 in the `itsandbox` account. +1. Create a pull request for `organizations-infra` to add a new Service Catalog + product to `scipool-dev` with 'test' in the name. +1. Provision the test product from http://sc-dev.sageit.org to validate AMI + integration with the product template +1. Create a pull request for the packer repo to modify the AMI. +1. Create a pull request for `service-catalog-library` to reference the new AMI. +1. Create a pull request for `organizations-infra` to remove the test product and + update the target product with the new template version. +1. Delete the manually-created resources from `itsandbox`. + ## Issues * https://sagebionetworks.jira.com/projects/IT From b1c005ba66ade34e2e8a3251ae9ee2b52478c2bc Mon Sep 17 00:00:00 2001 From: Joni Harker Date: Thu, 5 Dec 2024 13:41:35 -0800 Subject: [PATCH 2/3] [IT-3523] Update AMI testing to describe automation Update the documentation on how to perform functional and integration testing of AMIs used by service catalog products now that test artifacts are automatically created. Depends on Sage-Bionetworks-IT/packer-ami-template Depends on Sage-Bionetworks-IT/service-catalog-library --- sceptre/scipool/README.md | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/sceptre/scipool/README.md b/sceptre/scipool/README.md index 5980fe07..c2f2c1ce 100644 --- a/sceptre/scipool/README.md +++ b/sceptre/scipool/README.md @@ -37,34 +37,37 @@ As a pre-deployment step we syntatically validate our sceptre and cloudformation yaml files with [pre-commit](https://pre-commit.com). Please install pre-commit, once installed the file validations will -automatically run on every commit. Alternatively you can manually +automatically run on every commit. Alternatively you can manually execute the validations by running `pre-commit run --all-files`. Please install pre-commit, once installed the file validations will automatically run on every commit. ### Functional Testing The process to test the functionality of an AMI and it's integration with our -Service Catalog products is to first manually create a new AMI, then manually -upload a new product template to S3, and finally create a test product in -Service Catalog through a pull request to `organizations-infra`: - -1. Make local changes to the packer repo to update or modify the AMI. -1. Run packer manually with admin credentials for the `itsandbox` account - to create a test AMI. -1. Create an EC2 instance in the `itsandbox` account from the test AMI for any - initial system validation. -1. Make local changes to `service-catalog-library` to update the desired template - to reference the test AMI. -1. Manually upload the test template to S3 in the `itsandbox` account. +Service Catalog products is to first create a test AMI, upload a modified +product template to S3, and create a new Service Catalog product in the +scipool dev account to verify manually from https://sc-dev.sageit.org/ + +The deploy pipelines for both our packer repos and our service catalog library +will create artifacts for branches that begin with `test/` in the sandbox +account, allowing anyone with write access to the packer repos to create test +AMIs, and anyone with write access to service-catalog-library to upload test +templates for service catalog to S3. + +1. Commit changes to the packer repo to update or modify the AMI on a branch + that starts with `test/`, and push directly to the origin repo. +1. Manually create an EC2 instance in the `itsandbox` account from the test AMI + for any initial system validation, then terminate it. +1. Commit changes to `service-catalog-library` on a branch that starts with + `test/` to update the desired template, and push directly to the origin repo. 1. Create a pull request for `organizations-infra` to add a new Service Catalog - product to `scipool-dev` with 'test' in the name. + product to `scipool-dev` with 'test' in the name for the test template. 1. Provision the test product from http://sc-dev.sageit.org to validate AMI - integration with the product template + integration with the product template 1. Create a pull request for the packer repo to modify the AMI. 1. Create a pull request for `service-catalog-library` to reference the new AMI. 1. Create a pull request for `organizations-infra` to remove the test product and - update the target product with the new template version. -1. Delete the manually-created resources from `itsandbox`. + update the target product with the new template version. ## Issues * https://sagebionetworks.jira.com/projects/IT From 088bbbbc9d7c4bb05b5c18127b5b67d208087a1e Mon Sep 17 00:00:00 2001 From: Joni Harker Date: Mon, 30 Dec 2024 16:14:06 -0800 Subject: [PATCH 3/3] review feedback Reword and expand both the reasoning and the process documented. --- sceptre/scipool/README.md | 73 ++++++++++++++++++++++++++------------- 1 file changed, 49 insertions(+), 24 deletions(-) diff --git a/sceptre/scipool/README.md b/sceptre/scipool/README.md index c2f2c1ce..45171ae7 100644 --- a/sceptre/scipool/README.md +++ b/sceptre/scipool/README.md @@ -16,7 +16,7 @@ to build out the service catalog. # sceptre launch-stack prod ``` -The above should setup resources for the AWS account. Once the infrastructure +The above should setup resources for the AWS account. Once the infrastructure for the account has been setup you can access and view the account using the [AWS console](https://AWS-account-ID-or-alias.signin.aws.amazon.com/console). @@ -42,32 +42,57 @@ execute the validations by running `pre-commit run --all-files`. Please install pre-commit, once installed the file validations will automatically run on every commit. -### Functional Testing -The process to test the functionality of an AMI and it's integration with our -Service Catalog products is to first create a test AMI, upload a modified -product template to S3, and create a new Service Catalog product in the -scipool dev account to verify manually from https://sc-dev.sageit.org/ - -The deploy pipelines for both our packer repos and our service catalog library -will create artifacts for branches that begin with `test/` in the sandbox -account, allowing anyone with write access to the packer repos to create test -AMIs, and anyone with write access to service-catalog-library to upload test -templates for service catalog to S3. - -1. Commit changes to the packer repo to update or modify the AMI on a branch - that starts with `test/`, and push directly to the origin repo. -1. Manually create an EC2 instance in the `itsandbox` account from the test AMI - for any initial system validation, then terminate it. +### Pre-Merge Integration Testing + +To validate the integration of AMI changes with Service Catalog products several +resources need to be created in AWS including a test AMI, a test CFN template +(uploaded to S3), and a test product in Service Catalog. Finally, the test +product can be provisioned from https://sc-dev.sageit.org/ for user-acceptance +testing of the changes to the AMI and/or CFN template. + +This process has been automated by using standard deploy pipelines on test +branches. Automating the process improves test consistency and accuracy and +reduces the impact of human error. + +Anyone with write access to the upstream repos can automate creating AMIs and +uploading CFN templates to S3 by pushing `test/*` branches directly to the +uptsream repo. For example, pushing a branch named `test/foo` directly to +`Sage-Bionetworks-IT/packer-rstudio` will create an AMI named +`packer-rstudio-test/foo` in `imagecentral`, and pushing a branch named +`test/bar` directly to `Sage-Bionetworks/service-catalog-library` will upload +the templates to `service-catalog-library/test/bar/` in the bootstrap bucket in +`admincentral`. + +The standard PR development process is used to create a test product in Service +Catalog in this `scipool` sceptre project, such as in [this PR](https://github.com/Sage-Bionetworks-IT/organizations-infra/pull/1109). + +The full process with automation is: + +1. Commit changes to the target packer repo to modify the AMI on a branch that + starts with `test/`, and push it directly to the upstream repo to create a + test AMI. + - Initial manual testing of the AMI can be performed by using it to boot an + EC2 instance. 1. Commit changes to `service-catalog-library` on a branch that starts with - `test/` to update the desired template, and push directly to the origin repo. -1. Create a pull request for `organizations-infra` to add a new Service Catalog - product to `scipool-dev` with 'test' in the name for the test template. -1. Provision the test product from http://sc-dev.sageit.org to validate AMI - integration with the product template + `test/` to update the relevant template with the new AMI, and push directly + to the origin repo to upload the templates to S3. +1. Create a pull request for `organizations-infra` to modify or add a template + in [/sceptre/scipool/config/develop/](./config/develop) to deploy a Service + Catalog test product in the `scipool-dev` account. +1. Provision the test product from http://sc-dev.sageit.org to manually validate + AMI integration with the product template; and manually terminate when done. 1. Create a pull request for the packer repo to modify the AMI. 1. Create a pull request for `service-catalog-library` to reference the new AMI. -1. Create a pull request for `organizations-infra` to remove the test product and - update the target product with the new template version. +1. Create a pull request for `organizations-infra` to update the product templates + in the production accounts [scipool-prod](./config/prod), [bmgf-ki](./config/dmgf-ki), + and [strides](./config/strides). + +The list of active packer repos that are used by Service Catalog products: + +1. `Sage-Bionetworks-IT/packer-amazonlinux-docker` +1. `Sage-Bionetworks-IT/packer-base-ubuntu-jammy` +1. `Sage-Bionetworks-IT/packer-docker-server` +1. `Sage-Bionetworks-IT/packer-winserver-2022` ## Issues * https://sagebionetworks.jira.com/projects/IT