Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow test, don't merge #53

Closed
wants to merge 67 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
13ac1fe
Initial pipeline project setup.
nateglims May 17, 2023
ae68c70
(WIP) Initial CDK Project for Demo Pipeline.
nateglims May 18, 2023
836de6a
(WIP) Create GitHub Action For CDK Project
nateglims May 18, 2023
7ee785f
Update actions/checkout to v3.
nateglims May 18, 2023
6fecf30
Fix lints
nateglims May 18, 2023
0c3a286
(WIP) Create pipeline for ubuntu build host image.
nateglims May 18, 2023
0742ea1
WIP: starting demos pipeline
nateglims Jun 1, 2023
20c2ba0
WIP: test using artifact transfer for TMP_DIR
nateglims Jun 5, 2023
550e28e
WIP: fix formatting after removing codestar arn
nateglims Jun 5, 2023
32f4dd4
WIP: add test stage and use efs instead of artifacts
nateglims Jun 7, 2023
3ddb1c6
WIP: try adding efs permissions
nateglims Jun 8, 2023
8c4d78a
WIP: fix efs and transfer TMP_DIR to test stage.
nateglims Jun 10, 2023
5a6c296
WIP: update readme, upload image artifacts out of test stage.
nateglims Jun 12, 2023
1975397
wip: cleanup and testing
nateglims Jun 28, 2023
e21f5a7
docs: add typedoc and documation directory
nateglims Jun 28, 2023
3b9de8e
npm: change versioning range of cdk and run audit fix
nateglims Jun 28, 2023
54e04d8
add basic test for build-image-data stack
nateglims Jun 28, 2023
deec5b1
partially parameterize user specific data, update docs
nateglims Jun 28, 2023
3a10589
fix formatting
nateglims Jun 28, 2023
f370443
add branch parameter
thomas-roos Jul 5, 2023
bbd41ec
add more info
thomas-roos Jul 5, 2023
c1474dc
Fix formatting
nateglims Jul 10, 2023
a60f350
Update test setup for cdk pipeline.
nateglims Jul 10, 2023
d565643
add kas and repo tool to Dockerfile
thomas-roos Jul 14, 2023
1d651d8
add some notes
thomas-roos Jul 14, 2023
eb472b7
add examples for repo, kas
thomas-roos Jul 14, 2023
335905e
add examples for repo, kas
thomas-roos Jul 14, 2023
0484be7
Update demos-pipeline.ts
thomas-roos Jul 17, 2023
b2c33bc
Update setup.md
thomas-roos Jul 17, 2023
3dfdc6d
fix yaml
thomas-roos Jul 17, 2023
3d63879
fix branch
thomas-roos Jul 17, 2023
eb6acc4
demo pipeline: remove test stage
nateglims Jul 18, 2023
6134deb
demo pipeline: run zip-data command before build
nateglims Jul 18, 2023
798cdf4
demo pipeline: pass account and user specific data
nateglims Jul 18, 2023
514b667
demo pipeline: run npm audit fix
nateglims Jul 18, 2023
8b4b497
demo pipeline: fix formatting
nateglims Jul 18, 2023
51d504c
fix kas setup, formatting
thomas-roos Jul 20, 2023
ab3a4d2
separate lib and examples
thomas-roos Jul 25, 2023
f3cc6d7
delete not necessary stuff
thomas-roos Jul 25, 2023
26346e1
change .gitignore
thomas-roos Jul 25, 2023
6ae2074
remove binary
thomas-roos Jul 25, 2023
b4c41d4
use local lib in package.json
thomas-roos Jul 25, 2023
31358b1
add agl-nxp-goldbox example
thomas-roos Jul 26, 2023
7c67929
move build-image into lib
thomas-roos Jul 27, 2023
b9d61e3
fix tests
thomas-roos Jul 27, 2023
ab5fed5
GitHub workflow: fix test
thomas-roos Jul 27, 2023
3a3d1d3
fix trailing whitespace
thomas-roos Jul 27, 2023
7fc7eb1
fix test-cdk.yml
thomas-roos Jul 27, 2023
b0ae278
add .eslintrc.js
thomas-roos Jul 27, 2023
25bf6dd
run npm install in lib prebuilding examples
thomas-roos Jul 27, 2023
005d633
fix build-image asset
thomas-roos Jul 27, 2023
c93c5a8
fix ws
thomas-roos Jul 27, 2023
89ebd2c
fix image and sdk upload
thomas-roos Jul 27, 2023
df7476e
fix ws
thomas-roos Jul 27, 2023
677f97d
add 3 additional examples
thomas-roos Aug 4, 2023
51dc86e
fix lint errors
thomas-roos Aug 4, 2023
8d1bbac
add eslintrc
thomas-roos Aug 4, 2023
ad91a73
demo-pipeline: build out of tree
nateglims Aug 7, 2023
ba60def
demo pipeline: convert poky example to use code commit
nateglims Aug 7, 2023
bf13293
demo pipeline: update github test action
nateglims Aug 7, 2023
2759877
demo pipeline: fix test
nateglims Aug 8, 2023
ff01153
demo pipeline: fix 3p package-lock
nateglims Aug 8, 2023
f6a000e
demo pipeline: consolidate examples
nateglims Aug 14, 2023
bfd2632
demo pipeline: add repo based poky and submodule(ish) based demo code…
nateglims Aug 15, 2023
135a698
fix ignore files in example pipeline
nateglims Aug 15, 2023
4e524a8
remove build-image resources on stack delete
nateglims Aug 15, 2023
924942f
Add dynamic fallback name for CodeCommit repo to prevent errors on mu…
alinadima Aug 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/check-yaml.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out repository code
uses: actions/checkout@v2
uses: actions/checkout@v3
- name: Install Pre-Commit
run: pip install pre-commit
- name: Run Pre-Commit
Expand Down
33 changes: 33 additions & 0 deletions .github/workflows/test-cdk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Test CDK Demos Pipeline
on:
push:
branches: ["**"]
paths: ["demos-pipeline/**", ".github/workflows/test-cdk.yml"]
pull_request:
branches: [master]
paths: ["demos-pipeline/**", ".github/workflows/test-cdk.yml"]
jobs:
Run-CDK-Tests:
runs-on: ubuntu-latest

strategy:
matrix:
node-version: [16.x, 18.x, 20.x]
dir: ['demos-pipeline/lib', 'demos-pipeline/example']

defaults:
run:
working-directory: ${{ matrix.dir }}

steps:
- name: Check out repository code
uses: actions/checkout@v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
- run: npm ci
- run: npm run check
- run: npm run build --if-present
- name: Test ${{ matrix.dir }}
run: npm test
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,5 @@ bucket-policy.json

# In case you root the workspace in this directory:
auto-upgrader/.vscode/
/meta-aws-ci.iml
/.idea/
3 changes: 3 additions & 0 deletions demos-pipeline/example/.eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
npm_modules/
dist/
cdk.out/
22 changes: 22 additions & 0 deletions demos-pipeline/example/.eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
module.exports = {
env: {
browser: true,
es2021: true,
node: true,
jest: true,
},
extends: [
"eslint:recommended",
"plugin:@typescript-eslint/recommended",
"plugin:prettier/recommended",
],
overrides: [],
parser: "@typescript-eslint/parser",
parserOptions: {
ecmaVersion: "latest",
sourceType: "module",
},
plugins: ["@typescript-eslint"],
rules: {},
root: true,
};
8 changes: 8 additions & 0 deletions demos-pipeline/example/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
node_modules
dist

# CDK asset staging directory
.cdk.staging
cdk.out

cdk.context.json
26 changes: 26 additions & 0 deletions demos-pipeline/example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Meta-AWS CDK Library

An AWS [Cloud Developer Toolkit](https://docs.aws.amazon.com/cdk/v2/guide/home.html) Library for building Yocto projects in AWS.

## Quickstart

Refer to [`docs/setup.md`](docs/setup.md)

## Useful commands

- `npm run build` compile typescript to js
- `npm run watch` watch for changes and compile
- `npm run test` perform the jest unit tests
- `cdk deploy` deploy this stack to your default AWS account/region
- `cdk diff` compare deployed stack with current state
- `cdk synth` emits the synthesized CloudFormation template

Project Specific:
- `npm run format` runs prettier and eslint on the repository
- `npm run zip-data` bundles the files for creating build host containers
- `npm run check` checks for lint and format issues
- `npm run docs` to generate documentation

## Contributing

TODO: Notes contribution process (pre-commit, running tests, formatting and test standards, etc)
90 changes: 90 additions & 0 deletions demos-pipeline/example/bin/poky-pipeline.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
#!/usr/bin/env node
import * as cdk from "aws-cdk-lib";
import {
DemoPipelineStack,
RepoKind,
BuildImageDataStack,
BuildImagePipelineStack,
BuildImageRepoStack,
PipelineNetworkStack,
ImageKind,
} from "aws4embeddedlinux-cdk-lib";

const app = new cdk.App();

/**
* User Data
*/
const env = {
account: process.env.CDK_DEFAULT_ACCOUNT,
region: process.env.CDK_DEFAULT_REGION,
};

/**
* Use these default props to enable termination protection and tag related AWS
* Resources for tracking purposes.
*/
const defaultProps: cdk.StackProps = {
tags: { PURPOSE: "META-AWS-BUILD" },
terminationProtection: false, // TODO: enable.
env,
};

/**
* Set up the Stacks that create our Build Host.
*/
const buildImageData = new BuildImageDataStack(app, "BuildImageData", {
...defaultProps,
bucketName: `build-image-data-${env.account}-${env.region}`,
});

const buildImageRepo = new BuildImageRepoStack(app, "BuildImageRepo", {
...defaultProps,
});

new BuildImagePipelineStack(app, "BuildImagePipeline", {
...defaultProps,
dataBucket: buildImageData.bucket,
repository: buildImageRepo.repository,
imageKind: ImageKind.Ubuntu22_04,
});

/**
* Set up networking to allow us to securely attach EFS to our CodeBuild instances.
*/
const vpc = new PipelineNetworkStack(app, "PipelineNetwork", {
...defaultProps,
});

/**
* Create a poky pipeline.
*/
new DemoPipelineStack(app, "PokyPipeline", {
...defaultProps,
imageRepo: buildImageRepo.repository,
imageTag: ImageKind.Ubuntu22_04,
vpc: vpc.vpc,
});

/**
* Create a meta-aws-demos pipeline for the Qemu example.
*/
new DemoPipelineStack(app, "QemuDemoPipeline", {
...defaultProps,
imageRepo: buildImageRepo.repository,
imageTag: ImageKind.Ubuntu22_04,
vpc: vpc.vpc,
layerKind: RepoKind.MetaAwsDemo,
layerRepoName: "qemu-demo-layer-repo",
});

/**
* Create a poky pipeline.
*/
// TODO(nateglims): implement
// new DemoPipelineStack(app, "three-p-Pipeline", {
// ...defaultProps,
// imageRepo: buildImageRepo.repository,
// imageTag: ImageKind.Ubuntu22_04,
// vpc: vpc.vpc,
// });
51 changes: 51 additions & 0 deletions demos-pipeline/example/cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
{
"app": "npx ts-node --prefer-ts-exts bin/poky-pipeline.ts",
"poky-pipeline": "npx ts-node --prefer-ts-exts bin/poky-pipeline.ts",
"watch": {
"include": ["**"],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"yarn.lock",
"node_modules",
"test"
]
},
"context": {
"@aws-cdk-lib/aws-lambda:recognizeLayerVersion": true,
"@aws-cdk-lib/core:checkSecretUsage": true,
"@aws-cdk-lib/core:target-partitions": ["aws", "aws-cn"],
"@aws-cdk-lib-containers/ecs-service-extensions:enableDefaultLogDriver": true,
"@aws-cdk-lib/aws-ec2:uniqueImdsv2TemplateName": true,
"@aws-cdk-lib/aws-ecs:arnFormatIncludesClusterName": true,
"@aws-cdk-lib/aws-iam:minimizePolicies": true,
"@aws-cdk-lib/core:validateSnapshotRemovalPolicy": true,
"@aws-cdk-lib/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
"@aws-cdk-lib/aws-s3:createDefaultLoggingPolicy": true,
"@aws-cdk-lib/aws-sns-subscriptions:restrictSqsDescryption": true,
"@aws-cdk-lib/aws-apigateway:disableCloudWatchRole": true,
"@aws-cdk-lib/core:enablePartitionLiterals": true,
"@aws-cdk-lib/aws-events:eventsTargetQueueSameAccount": true,
"@aws-cdk-lib/aws-iam:standardizedServicePrincipals": true,
"@aws-cdk-lib/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
"@aws-cdk-lib/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
"@aws-cdk-lib/aws-s3:serverAccessLogsUseBucketPolicy": true,
"@aws-cdk-lib/aws-route53-patters:useCertificate": true,
"@aws-cdk-lib/customresources:installLatestAwsSdkDefault": false,
"@aws-cdk-lib/aws-rds:databaseProxyUniqueResourceName": true,
"@aws-cdk-lib/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
"@aws-cdk-lib/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
"@aws-cdk-lib/aws-ec2:launchTemplateDefaultUserData": true,
"@aws-cdk-lib/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
"@aws-cdk-lib/aws-redshift:columnId": true,
"@aws-cdk-lib/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
"@aws-cdk-lib/aws-ec2:restrictDefaultSecurityGroup": true,
"@aws-cdk-lib/aws-apigateway:requestValidatorUniqueId": true,
"@aws-cdk-lib/aws-kms:aliasNameRef": true,
"@aws-cdk-lib/core:includePrefixInUniqueNameGeneration": true
}
}
16 changes: 16 additions & 0 deletions demos-pipeline/example/docs/setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Setting Up

1. Run the following:

```
npm install .
npm run build

cdk bootstrap

cdk deploy --all
```

Run the newly created pipeline `ubuntu_22_04BuildImagePipeline` from the CodePipeline console page to create the build host.

After that completes, the DemoPipeline in the CodePipeline console page is ready to run.
Loading