diff --git a/.gitignore b/.gitignore index 6841c51..b4bfe08 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,5 @@ lib/ *.bak +*.sh +sample*.js +!test_create_instances.sh diff --git a/README.md b/README.md index 21234b1..f0979a5 100644 --- a/README.md +++ b/README.md @@ -1,44 +1,56 @@ -# Autotag - -This is an open-source tagging solution for AWS. Deploy autotag to lambda and set up CloudTrail and have each of your resources tagged with the resource who created it. It was written by GorillaStack. +# Auto Tag +This is an open-source tagging solution for AWS. Deploy auto tag to lambda and set up CloudTrail and have each of your resources tagged with the resource who created it. It was written by [GorillaStack](http://www.gorillastack.com/). +[Read a blog post about the project](http://blog.gorillastack.com/gorillastack-presents-auto-tag). ## Setup -#### Baseline policies for your lambda IAM role +### 1. Turn on CloudTrail for your region -If you install your lambda function and don't plan on tagging resources, at very least you will need these permissions: +1. Turn on CloudTrail. +2. Create a new Amazon S3 bucket for storing your log files, or specify an existing bucket where you want the log files delivered. +3. (Optional and NOT REQUIRED for auto tag) Create a new Amazon SNS topic in order to receive notifications when new log files are delivered. -1. Permissions to save logs for your lambda execution. -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "logs:CreateLogGroup", - "logs:CreateLogStream", - "logs:PutLogEvents" - ], - "Resource": "arn:aws:logs:*:*:*" - } - ] -} -``` +More [documentation on creating a Trail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html) + +### 2. Create a lambda function + +1. Within lambda, press the 'Create a Lambda Function' button +2. Press the 'Skip' button to bypass the suggested blueprints +3. Enter the lambda function name (e.g. 'autotag') +4. Select 'Node.js' as the Runtime +5. Upload the [latest release's zip file](https://github.com/GorillaStack/auto-tag/releases) +6. Under 'Handler' add 'autotag.handler' +7. Under 'Advanced settings' set the Timeout to 30s + +More [documentation on Lambda](https://docs.aws.amazon.com/lambda/latest/dg/getting-started.html) -2. Permissions to retrieve zipped CloudTrail log items from S3. +### 3. Configure the access policy for your lambda role + +Your lambda function will run as an IAM role. This is where we configure the permissions required. + +#### Lambda function master policy ```json { "Version": "2012-10-17", "Statement": [ + { + "Effect": "Allow", + "Action": [ + "logs:CreateLogGroup", + "logs:CreateLogStream", + "logs:PutLogEvents" + ], + "Resource": "arn:aws:logs:*:*:*" + }, { "Sid": "Stmt1442379848000", "Effect": "Allow", "Action": [ "s3:GetObject", - "s3:ListBucket" + "s3:ListBucket", + "sts:*" ], "Resource": [ "*" @@ -48,43 +60,28 @@ If you install your lambda function and don't plan on tagging resources, at very } ``` +This contains permissions for: -#### Necessary policies for your lambda's IAM role -Actions to allow for all resources: +1. Saving logs for your lambda execution. +2. Retrieving zipped CloudTrail log items from S3. +3. Assuming the auto-tag role on targeted accounts. -* S3: `s3:GetBucketTagging` - `s3:PutBucketTagging` -* EC2: `ec2:CreateTags` -* ELB: `elasticloadbalancing:AddTags` -* AutoScaling: `autoscaling:CreateOrUpdateTags` -* EBS: `ec2:CreateTags` (Same as EC2) -* VPC: `ec2:CreateTags` (Same as EC2) -* Subnet: `ec2:CreateTags` (Same as EC2) -* InternetGateway: `ec2:CreateTags` (Same as EC2) -* RDS: `rds:AddTagsToResource` -* EMR: `elasticmapreduce:AddTags` -* DataPipeline: `datapipeline:AddTags` +### 4. Configure the access policy for your Auto-Tag role -## Whole master policy +When auto-tag finds an event that indicated the creation of a resource, it needs permissions to tag that resource. On each account, a role for cross account access will need to be created, with the following permissions. + +*NOTE;* The role must be named 'AutoTagRole'. +(This is a temporary hard requirement, given that after uploading the code via zip file, AWS doesn't allow the user to edit the main file inline. This introduces complexity in user defined configuration for the lambda function.) + +#### Auto-Tag role master policy ```json { "Version": "2012-10-17", "Statement": [ - { - "Effect": "Allow", - "Action": [ - "logs:CreateLogGroup", - "logs:CreateLogStream", - "logs:PutLogEvents" - ], - "Resource": "arn:aws:logs:*:*:*" - }, { "Sid": "Stmt1442379848000", "Effect": "Allow", "Action": [ - "s3:GetObject", - "s3:ListBucket", "s3:GetBucketTagging", "s3:PutBucketTagging", "ec2:CreateTags", @@ -102,6 +99,22 @@ Actions to allow for all resources: } ``` +Details on specific requirements for each service, in case you want a subset of this master policy: + +* S3: `s3:GetBucketTagging` + `s3:PutBucketTagging` +* EC2: `ec2:CreateTags` +* ELB: `elasticloadbalancing:AddTags` +* AutoScaling: `autoscaling:CreateOrUpdateTags` +* EBS: `ec2:CreateTags` (Same as EC2) +* VPC: `ec2:CreateTags` (Same as EC2) +* Subnet: `ec2:CreateTags` (Same as EC2) +* InternetGateway: `ec2:CreateTags` (Same as EC2) +* RDS: `rds:AddTagsToResource` +* EMR: `elasticmapreduce:AddTags` +* DataPipeline: `datapipeline:AddTags` + + ## Contributing If you have questions, feature requests or bugs to report, please do so on the github repository. @@ -110,7 +123,7 @@ If you are interested in contributing, please get started by forking our github ### Development guide -Autotag is implemented in Javascript (ECMAScript 2015 - a.k.a. es6). To make this compatible with lambda and other es5 environments, we use [babel](https://babeljs.io/) to transpile the es6 code to es5. For this reason, you will need to install babel globally to get started: +Auto tag is implemented in Javascript (ECMAScript 2015 - a.k.a. es6). To make this compatible with lambda and other es5 environments, we use [babel](https://babeljs.io/) to transpile the es6 code to es5. For this reason, you will need to install babel globally to get started: ```bash $ npm install -g babel @@ -127,3 +140,9 @@ To assist you in packaging and deploying your code to your lambda function, I ha ```bash $ bash deploy_lambda_code.sh ``` + +To assist with running during development without having to deploy to lambda, use the `main.js` and `sample_data.js` files we have provided. + +```bash +$ node main.js +``` diff --git a/src/autotag.js b/src/autotag.js index 7e228e8..e78cd32 100644 --- a/src/autotag.js +++ b/src/autotag.js @@ -1,7 +1,8 @@ const AwsCloudTrailListener = require('./aws_cloud_trail_listener'); const exports = {}; + exports.handler = function(cloudtrailEvent, context) { - let enabledListeners = [ + const enabledListeners = [ AwsCloudTrailListener.EC2.name, AwsCloudTrailListener.S3.name, AwsCloudTrailListener.AUTOSCALE_GROUPS.name, diff --git a/src/autotag_factory.js b/src/autotag_factory.js index c1f103a..d933e19 100644 --- a/src/autotag_factory.js +++ b/src/autotag_factory.js @@ -1,16 +1,16 @@ const _ = require('underscore'); -const AutotagDefaultWorker = require('./autotag_default_worker.js'); -const AutotagEC2Worker = require('./autotag_ec2_worker.js'); -const AutotagS3Worker = require('./autotag_s3_worker.js'); -const AutotagELBWorker = require('./autotag_elb_worker.js'); -const AutotagEBSWorker = require('./autotag_ebs_worker.js'); -const AutotagAutoscaleWorker = require('./autotag_autoscale_worker.js'); -const AutotagVPCWorker = require('./autotag_vpc_worker.js'); -const AutotagSubnetWorker = require('./autotag_subnet_worker.js'); -const AutotagInternetGatewayWorker = require('./autotag_internet_gateway_worker.js'); -const AutotagRDSWorker = require('./autotag_rds_worker.js'); -const AutotagEMRWorker = require('./autotag_emr_worker.js'); -const AutotagDataPipelineWorker = require('./autotag_data_pipeline_worker.js'); +const AutotagDefaultWorker = require('./workers/autotag_default_worker.js'); +const AutotagEC2Worker = require('./workers/autotag_ec2_worker.js'); +const AutotagS3Worker = require('./workers/autotag_s3_worker.js'); +const AutotagELBWorker = require('./workers/autotag_elb_worker.js'); +const AutotagEBSWorker = require('./workers/autotag_ebs_worker.js'); +const AutotagAutoscaleWorker = require('./workers/autotag_autoscale_worker.js'); +const AutotagVPCWorker = require('./workers/autotag_vpc_worker.js'); +const AutotagSubnetWorker = require('./workers/autotag_subnet_worker.js'); +const AutotagInternetGatewayWorker = require('./workers/autotag_internet_gateway_worker.js'); +const AutotagRDSWorker = require('./workers/autotag_rds_worker.js'); +const AutotagEMRWorker = require('./workers/autotag_emr_worker.js'); +const AutotagDataPipelineWorker = require('./workers/autotag_data_pipeline_worker.js'); const CONFIG = require('./cloud_trail_event_config'); let AutotagFactory = { diff --git a/src/autotag_autoscale_worker.js b/src/workers/autotag_autoscale_worker.js similarity index 75% rename from src/autotag_autoscale_worker.js rename to src/workers/autotag_autoscale_worker.js index f4c9008..95e38ae 100644 --- a/src/autotag_autoscale_worker.js +++ b/src/workers/autotag_autoscale_worker.js @@ -1,10 +1,10 @@ const AutotagDefaultWorker = require('./autotag_default_worker'); const AWS = require('aws-sdk'); +const co = require('co'); class AutotagAutoscaleWorker extends AutotagDefaultWorker { constructor(event) { super(event); - this.autoscaling = new AWS.AutoScaling({region: event.awsRegion}); } /* tagResource @@ -14,6 +14,18 @@ class AutotagAutoscaleWorker extends AutotagDefaultWorker { */ tagResource() { + let _this = this; + return co(function* () { + let credentials = yield _this.assumeRole(); + _this.autoscaling = new AWS.AutoScaling({ + region: _this.event.awsRegion, + credentials: credentials + }); + yield _this.tagAutoscalingGroup(); + }); + } + + tagAutoscalingGroup() { let _this = this; return new Promise(function(resolve, reject) { try { diff --git a/src/autotag_data_pipeline_worker.js b/src/workers/autotag_data_pipeline_worker.js similarity index 75% rename from src/autotag_data_pipeline_worker.js rename to src/workers/autotag_data_pipeline_worker.js index bb2531c..4012f83 100644 --- a/src/autotag_data_pipeline_worker.js +++ b/src/workers/autotag_data_pipeline_worker.js @@ -1,11 +1,11 @@ const AutotagDefaultWorker = require('./autotag_default_worker'); const AWS = require('aws-sdk'); +const co = require('co'); const _ = require('underscore'); class AutotagDataPipelineWorker extends AutotagDefaultWorker { constructor(event) { super(event); - this.dataPipeline = new AWS.DataPipeline({region: event.awsRegion}); } /* tagResource @@ -15,6 +15,18 @@ class AutotagDataPipelineWorker extends AutotagDefaultWorker { */ tagResource() { + let _this = this; + return co(function* () { + let credentials = yield _this.assumeRole(); + _this.dataPipeline = new AWS.DataPipeline({ + region: _this.event.awsRegion, + credentials: credentials + }); + yield _this.tagDataPipelineResource(); + }); + } + + tagDataPipelineResource() { let _this = this; return new Promise(function(resolve, reject) { try { diff --git a/src/autotag_default_worker.js b/src/workers/autotag_default_worker.js similarity index 50% rename from src/autotag_default_worker.js rename to src/workers/autotag_default_worker.js index 5ae7da2..9799f7c 100644 --- a/src/autotag_default_worker.js +++ b/src/workers/autotag_default_worker.js @@ -1,4 +1,7 @@ +const AWS = require('aws-sdk'); const AUTOTAG_TAG_NAME = 'AutoTag_Creator'; +const ROLE_PREFIX = 'arn:aws:iam::'; +const ROLE_SUFFIX = ':role/AutoTagRole'; class AutotagDefaultWorker { constructor(event) { @@ -22,6 +25,34 @@ class AutotagDefaultWorker { }); } + assumeRole() { + let _this = this; + return new Promise(function(resolve, reject) { + try { + AWS.config.region = 'us-east-1'; + let sts = new AWS.STS(); + sts.assumeRole({ + RoleArn: ROLE_PREFIX + _this.event.recipientAccountId + ROLE_SUFFIX, + RoleSessionName: 'AutoTag-' + (new Date()).getTime(), + DurationSeconds: 900 + }, function(err, data) { + if (err) { + reject(err); + } else { + let credentials = { + accessKeyId: data.Credentials.AccessKeyId, + secretAccessKey: data.Credentials.SecretAccessKey, + sessionToken: data.Credentials.SessionToken + }; + resolve(credentials); + } + }); + } catch (err) { + reject(err); + } + }); + } + dumpEventInfo() { console.log('Event Name: ' + this.event.eventName); console.log('Event Type: ' + this.event.eventType); diff --git a/src/autotag_ebs_worker.js b/src/workers/autotag_ebs_worker.js similarity index 55% rename from src/autotag_ebs_worker.js rename to src/workers/autotag_ebs_worker.js index 9655c48..abc0e6f 100644 --- a/src/autotag_ebs_worker.js +++ b/src/workers/autotag_ebs_worker.js @@ -1,5 +1,6 @@ const AutotagEC2Worker = require('./autotag_ec2_worker'); const AWS = require('aws-sdk'); +const co = require('co'); class AutotagEBSWorker extends AutotagEC2Worker { /* tagResource @@ -9,7 +10,15 @@ class AutotagEBSWorker extends AutotagEC2Worker { */ tagResource() { - return this.tagEC2Resources([this.getVolumeId()]); + let _this = this; + return co(function* () { + let credentials = yield _this.assumeRole(); + _this.ec2 = new AWS.EC2({ + region: _this.event.awsRegion, + credentials: credentials + }); + yield _this.tagEC2Resources([_this.getVolumeId()]); + }); } getVolumeId() { diff --git a/src/autotag_ec2_worker.js b/src/workers/autotag_ec2_worker.js similarity index 67% rename from src/autotag_ec2_worker.js rename to src/workers/autotag_ec2_worker.js index 32d057e..633df33 100644 --- a/src/autotag_ec2_worker.js +++ b/src/workers/autotag_ec2_worker.js @@ -1,20 +1,27 @@ const AutotagDefaultWorker = require('./autotag_default_worker'); const AWS = require('aws-sdk'); +const co = require('co'); class AutotagEC2Worker extends AutotagDefaultWorker { constructor(event) { super(event); - this.ec2 = new AWS.EC2({region: event.awsRegion}); - } + /* tagResource ** method: tagResource ** - ** Do nothing + ** Tag the ec2 instance */ - tagResource() { - return this.tagEC2Resources([this.getInstanceId()]); + let _this = this; + return co(function* () { + let credentials = yield _this.assumeRole(); + _this.ec2 = new AWS.EC2({ + region: _this.event.awsRegion, + credentials: credentials + }); + yield _this.tagEC2Resources([_this.getInstanceId()]); + }); } tagEC2Resources(resources) { @@ -27,10 +34,11 @@ class AutotagEC2Worker extends AutotagDefaultWorker { _this.getAutotagPair() ] }, function(err, res) { - if (err) + if (err) { reject(err); - else + } else { resolve(true); + } }); } catch(e) { reject(e); diff --git a/src/autotag_elb_worker.js b/src/workers/autotag_elb_worker.js similarity index 74% rename from src/autotag_elb_worker.js rename to src/workers/autotag_elb_worker.js index d2f0ec3..10c79e6 100644 --- a/src/autotag_elb_worker.js +++ b/src/workers/autotag_elb_worker.js @@ -1,10 +1,10 @@ const AutotagDefaultWorker = require('./autotag_default_worker'); const AWS = require('aws-sdk'); +const co = require('co'); class AutotagELBWorker extends AutotagDefaultWorker { constructor(event) { super(event); - this.elb = new AWS.ELB({region: event.awsRegion}); } /* tagResource @@ -12,8 +12,19 @@ class AutotagELBWorker extends AutotagDefaultWorker { ** ** Add tag to elastic load balancer */ - tagResource() { + let _this = this; + return co(function* () { + let credentials = yield _this.assumeRole(); + _this.elb = new AWS.ELB({ + region: _this.event.awsRegion, + credentials: credentials + }); + yield _this.tagELBResource(); + }); + } + + tagELBResource() { let _this = this; return new Promise(function(resolve, reject) { try { diff --git a/src/autotag_emr_worker.js b/src/workers/autotag_emr_worker.js similarity index 73% rename from src/autotag_emr_worker.js rename to src/workers/autotag_emr_worker.js index fe59159..1667ff6 100644 --- a/src/autotag_emr_worker.js +++ b/src/workers/autotag_emr_worker.js @@ -1,10 +1,10 @@ const AutotagDefaultWorker = require('./autotag_default_worker'); const AWS = require('aws-sdk'); +const co = require('co'); class AutotagEMRWorker extends AutotagDefaultWorker { constructor(event) { super(event); - this.emr = new AWS.EMR({region: event.awsRegion}); } /* tagResource @@ -14,6 +14,18 @@ class AutotagEMRWorker extends AutotagDefaultWorker { */ tagResource() { + let _this = this; + return co(function* () { + let credentials = yield _this.assumeRole(); + _this.emr = new AWS.EMR({ + region: _this.event.awsRegion, + credentials: credentials + }); + yield _this.tagEMRResource(); + }); + } + + tagEMRResource() { let _this = this; return new Promise(function(resolve, reject) { try { diff --git a/src/autotag_internet_gateway_worker.js b/src/workers/autotag_internet_gateway_worker.js similarity index 58% rename from src/autotag_internet_gateway_worker.js rename to src/workers/autotag_internet_gateway_worker.js index 66dd9a0..1aac071 100644 --- a/src/autotag_internet_gateway_worker.js +++ b/src/workers/autotag_internet_gateway_worker.js @@ -1,5 +1,6 @@ const AutotagEC2Worker = require('./autotag_ec2_worker'); const AWS = require('aws-sdk'); +const co = require('co'); class AutotagInternetGatewayWorker extends AutotagEC2Worker { /* tagResource @@ -9,7 +10,15 @@ class AutotagInternetGatewayWorker extends AutotagEC2Worker { */ tagResource() { - return this.tagEC2Resources([this.getInternetGatewayId()]); + let _this = this; + return co(function* () { + let credentials = yield _this.assumeRole(); + _this.ec2 = new AWS.EC2({ + region: _this.event.awsRegion, + credentials: credentials + }); + yield _this.tagEC2Resources([_this.getInternetGatewayId()]); + }); } getInternetGatewayId() { diff --git a/src/autotag_rds_worker.js b/src/workers/autotag_rds_worker.js similarity index 81% rename from src/autotag_rds_worker.js rename to src/workers/autotag_rds_worker.js index 0ec3c02..c181c50 100644 --- a/src/autotag_rds_worker.js +++ b/src/workers/autotag_rds_worker.js @@ -1,10 +1,10 @@ const AutotagDefaultWorker = require('./autotag_default_worker'); const AWS = require('aws-sdk'); +const co = require('co'); class AutotagRDSWorker extends AutotagDefaultWorker { constructor(event) { super(event); - this.rds = new AWS.RDS({region: event.awsRegion}); } /* tagResource @@ -14,6 +14,18 @@ class AutotagRDSWorker extends AutotagDefaultWorker { */ tagResource() { + let _this = this; + return co(function* () { + let credentials = yield _this.assumeRole(); + _this.rds = new AWS.RDS({ + region: _this.event.awsRegion, + credentials: credentials + }); + yield _this.tagRDSResource(); + }); + } + + tagRDSResource() { let _this = this; return new Promise(function(resolve, reject) { try { diff --git a/src/autotag_s3_worker.js b/src/workers/autotag_s3_worker.js similarity index 90% rename from src/autotag_s3_worker.js rename to src/workers/autotag_s3_worker.js index 985683b..16f9027 100644 --- a/src/autotag_s3_worker.js +++ b/src/workers/autotag_s3_worker.js @@ -3,10 +3,8 @@ const AWS = require('aws-sdk'); const co = require('co'); class AutotagS3Worker extends AutotagDefaultWorker { - constructor(event) { super(event); - this.s3 = new AWS.S3({region: event.awsRegion}); } /* tagResource ** method: tagResource @@ -17,6 +15,11 @@ class AutotagS3Worker extends AutotagDefaultWorker { tagResource() { let _this = this; return co(function* () { + let credentials = yield _this.assumeRole(); + _this.s3 = new AWS.S3({ + region: _this.event.awsRegion, + credentials: credentials + }); let tags = yield _this.getExistingTags(); tags.push(_this.getAutotagPair()); yield _this.setTags(tags); diff --git a/src/autotag_subnet_worker.js b/src/workers/autotag_subnet_worker.js similarity index 54% rename from src/autotag_subnet_worker.js rename to src/workers/autotag_subnet_worker.js index 6a01375..00c7b10 100644 --- a/src/autotag_subnet_worker.js +++ b/src/workers/autotag_subnet_worker.js @@ -1,5 +1,6 @@ const AutotagEC2Worker = require('./autotag_ec2_worker'); const AWS = require('aws-sdk'); +const co = require('co'); class AutotagSubnetWorker extends AutotagEC2Worker { /* tagResource @@ -9,7 +10,15 @@ class AutotagSubnetWorker extends AutotagEC2Worker { */ tagResource() { - return this.tagEC2Resources([this.getSubnetId()]); + let _this = this; + return co(function* () { + let credentials = yield _this.assumeRole(); + _this.ec2 = new AWS.EC2({ + region: _this.event.awsRegion, + credentials: credentials + }); + yield _this.tagEC2Resources([_this.getSubnetId()]); + }); } getSubnetId() { diff --git a/src/autotag_vpc_worker.js b/src/workers/autotag_vpc_worker.js similarity index 53% rename from src/autotag_vpc_worker.js rename to src/workers/autotag_vpc_worker.js index a5f77a6..cbd43e9 100644 --- a/src/autotag_vpc_worker.js +++ b/src/workers/autotag_vpc_worker.js @@ -1,5 +1,6 @@ const AutotagEC2Worker = require('./autotag_ec2_worker'); const AWS = require('aws-sdk'); +const co = require('co'); class AutotagVPCWorker extends AutotagEC2Worker { /* tagResource @@ -9,7 +10,15 @@ class AutotagVPCWorker extends AutotagEC2Worker { */ tagResource() { - return this.tagEC2Resources([this.getVPCId()]); + let _this = this; + return co(function* () { + let credentials = yield _this.assumeRole(); + _this.ec2 = new AWS.EC2({ + region: _this.event.awsRegion, + credentials: credentials + }); + yield _this.tagEC2Resources([_this.getVPCId()]); + }); } getVPCId() { diff --git a/test_create_instances.sh b/test_create_instances.sh new file mode 100644 index 0000000..9a5f33c --- /dev/null +++ b/test_create_instances.sh @@ -0,0 +1,58 @@ +#!/bin/bash + +##### NOTE +### Requirements: +### - AWS CLI +### - jsawk +### - key-pair 'test' + +# If you need, set your default aws profile to create the resources on the right account +##export AWS_DEFAULT_PROFILE= + + +## Create EC2 instances +echo Creating EC2 instance +INSTANCE_ID=`aws --region ap-northeast-1 ec2 run-instances --image-id ami-936d9d93 --instance-type t2.micro --key-name test | jsawk "return this.Instances[0].InstanceId;"` +echo INSTANCE_ID=$INSTANCE_ID + +## Create S3 bucket +echo Creating S3 bucket +aws --region ap-northeast-1 s3 mb s3://test-bucket-gs-autotag-1 + +# ## Create ELB +echo Creating ELB +aws --region ap-northeast-1 elb create-load-balancer \ + --load-balancer-name testLoadBalancer \ + --availability-zones ap-northeast-1a ap-northeast-1c \ + --listeners Protocol=HTTP,LoadBalancerPort=80,InstanceProtocol=HTTP,InstancePort=80 + + +## Create RDS bucket +echo Creating RDS instance +aws --region ap-northeast-1 rds create-db-instance \ + --db-instance-identifier mytestrdsinstance \ + --db-instance-class db.t1.micro \ + --allocated-storage 5 \ + --engine MySQL \ + --master-username testtest \ + --master-user-password testtest + +## Create VPC +echo Creating VPC +VPC_ID=`aws --region ap-northeast-1 ec2 create-vpc --cidr-block 10.0.0.0/16 | jsawk "return this.Vpc.VpcId;"` +echo VPC_ID=$VPC_ID + +## Create VPC Subnet +echo Creating VPC Subnet +SUBNET_ID=`aws --region ap-northeast-1 ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.0.12.0/24 | jsawk "return this.Subnet.SubnetId;"` +echo SUBNET_ID=$SUBNET_ID + +## Create Internet Gateway for VPC +echo Creating Internet Gateway +INTERNET_GATEWAY_ID=`aws --region ap-northeast-1 ec2 create-internet-gateway | jsawk "return this.InternetGateway.InternetGatewayId;"` +echo INTERNET_GATEWAY_ID=$INTERNET_GATEWAY_ID + +## Create Elastic Map Reduce cluster +echo Creating EMR cluster +CLUSTER_ID`aws --region ap-northeast-1 emr create-cluster --release-label emr-4.0.0 --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m1.medium InstanceGroupType=CORE,InstanceCount=2,InstanceType=m1.medium --use-default-roles | jsawk "return this.ClusterId;"` +echo CLUSTER_ID=$CLUSTER_ID