Help with Documentation #2
Comments
|
Hi! Thanks for trying out minotaur. |
|
Definately making more sense now with needing the infrastructure components. I am getting an error on the public VPC CIDR address. Any ideas? minotaur infrastructure deploy all -e test -r us-east-1 -z us-east-1a -i m1.small -c 10.0.0.0/24 The CIDR '10.0.2.0/27' is invalid. thanks! |
|
Recommended subnetting scheme is as follows: for each vpc of, for example, 10.0.0.0/21 there is 2 public 10.0.0.0/23, 10.0.0.4/23 subnets, 2 private 10.0.0.2/24, 10.0.0.6/24 subnets and 2 reserved 10.0.0.3/24, 10.0.0.7/24 subnets. So there is 3 subnets(public, private and reserved) per one availability zone. |
|
Thanks. I'm trying to do this will the all option. The /21 subnet mask worked. The nat creation failed next. cancelled |
|
Thanks for your patience, I will try to guide you further through deployment procedure. |
|
Thanks for the help. Sorry if I'm just not following the instructions correctly. Anything else prior to that command? minotaur infrastructure deploy iampolicies errors with: File "/deploy/infrastructure/aws/infrastructure.py", line 73, in deploy |
|
Ohh, it's a bug, thanks. I've done quick patch for it - please use my development fork(https://github.com/alexgear/minotaur/tree/hadoop) to get updated version. Make sure to use hadoop branch - it is where patch was dropped. |
|
Also don't forget to populate this directory with json file where your credentials will be stored. You can read more about this procedure in bastion readme. |
|
I had to create a 'trusted' group and an 'administrators' group to get the iampolicies stack to work. I just made these two with full admin privileges to get past that part. Let's see how the deploy all goes this time. I'm confused about this bastion part. It's just an ssh server in the public sub-domain. I have my pem file, but I don't have a public key. Where do I get that? |
|
You can generate public key from your privatekey.pem using following command: |
|
Closer. The nat creation still partially completes. 11:47:49 UTC-0800 CREATE_FAILED AWS::CloudFormation::Stack nat-test-us-east-1-us-east-1a The following resource(s) failed to create: [WaitForInstance]. |
|
May you write here the content of /var/log/user-data.log file located on nat instance. If it's too long - write me an email. |
|
Error indicated there wasn't a 0.0.0.0/0 route in my route table. Trying to work around it, I added it manually. I think I'm getting further now. Lets see! |
|
Awesome! Route table entry with destination 0.0.0.0/0 must point to the network interface of the nat instance and must be automaticly created by user-data script. But to investigate your issue I really need the content of /var/log/user-data.log file located on nat instance, please send it to me. |
|
Ok, I wanted to start over to better test because I was still having some trouble. Ran this: It hangs on the nat creation. Here is the /var/log/user-data.log file + export AWS_DEFAULT_REGION=us-east-1
+ AWS_DEFAULT_REGION=us-east-1
+ ROUTE_TABLE_ID=rtb-e8473a8d
+ INSTANCE_WAIT_HANDLE_URL='https://cloudfor...'
++ curl http://169.254.169.254/latest/meta-data/instance-id
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 10 100 10 0 0 1502 0 --:--:-- --:--:-- --:--:-- 1666
+ INSTANCE_ID=i-163cc7e7
++ aws ec2 describe-network-interfaces --filters '{"Name":"attachment.instance-id", "Values":["i-163cc7e7"]}'
+ DESCRIBE_INTERFACES_RESPONSE='{
"NetworkInterfaces": [
{
"Status": "in-use",
"MacAddress": "12:4b:e4:7f:a5:72",
"SourceDestCheck": true,
"VpcId": "vpc-4d224e28",
"Description": null,
"Association": {
"PublicIp": "5...",
"IpOwnerId": "amazon"
},
"NetworkInterfaceId": "eni-04548229",
"PrivateIpAddresses": [
{
"Association": {
"PublicIp": "5...",
"IpOwnerId": "amazon"
},
"Primary": true,
"PrivateIpAddress": "10.0.2.226"
}
],
"RequesterManaged": false,
"AvailabilityZone": "us-east-1a",
"Attachment": {
"Status": "attached",
"DeviceIndex": 0,
"AttachTime": "2015-01-16T00:09:50.000Z",
"InstanceId": "i-163cc7e7",
"DeleteOnTermination": true,
"AttachmentId": "eni-attach-cb2a29b6",
"InstanceOwnerId": "876410254869"
},
"Groups": [
{
"GroupName": "nat-test-us-east-1-us-east-1a-SecurityGroup-1QGV5HGUTBZ71",
"GroupId": "sg-a011c5c4"
}
],
"SubnetId": "subnet-209e480b",
"OwnerId": "876410254869",
"TagSet": [],
"PrivateIpAddress": "10.0.2.226"
}
]
}'
+ yum -y install jq
Loaded plugins: priorities, update-motd, upgrade-helper
Resolving Dependencies
--> Running transaction check
---> Package jq.x86_64 0:1.4-1.0.amzn1 will be installed
--> Processing Dependency: jq-libs(x86-64) = 1.4-1.0.amzn1 for package: jq-1.4-1.0.amzn1.x86_64
--> Processing Dependency: libjq.so.1()(64bit) for package: jq-1.4-1.0.amzn1.x86_64
--> Running transaction check
---> Package jq-libs.x86_64 0:1.4-1.0.amzn1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
jq x86_64 1.4-1.0.amzn1 amzn-main 28 k
Installing for dependencies:
jq-libs x86_64 1.4-1.0.amzn1 amzn-main 90 k
Transaction Summary
================================================================================
Install 1 Package (+1 Dependent package)
Total download size: 118 k
Installed size: 238 k
Downloading packages:
--------------------------------------------------------------------------------
Total 553 kB/s | 118 kB 00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : jq-libs-1.4-1.0.amzn1.x86_64 1/2
Installing : jq-1.4-1.0.amzn1.x86_64 2/2
Verifying : jq-libs-1.4-1.0.amzn1.x86_64 1/2
Verifying : jq-1.4-1.0.amzn1.x86_64 2/2
Installed:
jq.x86_64 0:1.4-1.0.amzn1
Dependency Installed:
jq-libs.x86_64 0:1.4-1.0.amzn1
Complete!
++ jq --raw-output '.["NetworkInterfaces"][0]["NetworkInterfaceId"]'
++ echo '{
"NetworkInterfaces": [
{
"Status": "in-use",
"MacAddress": "12:4b:e4:7f:a5:72",
"SourceDestCheck": true,
"VpcId": "vpc-4d224e28",
"Description": null,
"Association": {
"PublicIp": "5...",
"IpOwnerId": "amazon"
},
"NetworkInterfaceId": "eni-04548229",
"PrivateIpAddresses": [
{
"Association": {
"PublicIp": "5...",
"IpOwnerId": "amazon"
},
"Primary": true,
"PrivateIpAddress": "10.0.2.226"
}
],
"RequesterManaged": false,
"AvailabilityZone": "us-east-1a",
"Attachment": {
"Status": "attached",
"DeviceIndex": 0,
"AttachTime": "2015-01-16T00:09:50.000Z",
"InstanceId": "i-163cc7e7",
"DeleteOnTermination": true,
"AttachmentId": "eni-attach-cb2a29b6",
"InstanceOwnerId": "876410254869"
},
"Groups": [
{
"GroupName": "nat-test-us-east-1-us-east-1a-SecurityGroup-1QGV5HGUTBZ71",
"GroupId": "sg-a011c5c4"
}
],
"SubnetId": "subnet-209e480b",
"OwnerId": "876410254869",
"TagSet": [],
"PrivateIpAddress": "10.0.2.226"
}
]
}'
+ NETWORK_INTERFACE_ID=eni-04548229
++ aws ec2 describe-route-tables --filters '{"Name":"route-table-id", "Values":["rtb-e8473a8d"]}'
+ DESCRIBE_ROUTE_TABLES_RESPONSE='{
"RouteTables": [
{
"Associations": [
{
"SubnetId": "subnet-d39147f8",
"RouteTableAssociationId": "rtbassoc-924010f7",
"Main": false,
"RouteTableId": "rtb-e8473a8d"
}
],
"RouteTableId": "rtb-e8473a8d",
"VpcId": "vpc-4d224e28",
"PropagatingVgws": [],
"Tags": [
{
"Value": "RouteTable",
"Key": "aws:cloudformation:logical-id"
},
{
"Value": "arn:aws:cloudformation:us-east-1:876410254869:stack/subnet-test-us-east-1-us-east-1a-private/6933a810-9d13-11e4-b6d3-5001b491380a",
"Key": "aws:cloudformation:stack-id"
},
{
"Value": "subnet-test-us-east-1-us-east-1a-private",
"Key": "aws:cloudformation:stack-name"
}
],
"Routes": [
{
"GatewayId": "local",
"DestinationCidrBlock": "10.0.0.0/21",
"State": "active"
}
]
}
]
}'
+ [[ {
"RouteTables": [
{
"Associations": [
{
"SubnetId": "subnet-d39147f8",
"RouteTableAssociationId": "rtbassoc-924010f7",
"Main": false,
"RouteTableId": "rtb-e8473a8d"
}
],
"RouteTableId": "rtb-e8473a8d",
"VpcId": "vpc-4d224e28",
"PropagatingVgws": [],
"Tags": [
{
"Value": "RouteTable",
"Key": "aws:cloudformation:logical-id"
},
{
"Value": "arn:aws:cloudformation:us-east-1:876410254869:stack/subnet-test-us-east-1-us-east-1a-private/6933a810-9d13-11e4-b6d3-5001b491380a",
"Key": "aws:cloudformation:stack-id"
},
{
"Value": "subnet-test-us-east-1-us-east-1a-private",
"Key": "aws:cloudformation:stack-name"
}
],
"Routes": [
{
"GatewayId": "local",
"DestinationCidrBlock": "10.0.0.0/21",
"State": "active"
}
]
}
]
} == *\0\.\0\.\0\.\0* ]]
+ aws ec2 delete-route --route-table-id rtb-e8473a8d --destination-cidr-block 0.0.0.0/0
A client error (InvalidRoute.NotFound) occurred: no route with destination-cidr-block 0.0.0.0/0 in route table rtb-e8473a8d
|
|
Ok it's a bug. You can get rid of it by redeploying your infrastructure with -c 10.0.8.0/21 option.(10.0.0.0/21 will cause this error). Fixes will be dropped shortly. |
This seems like it could be very useful, but I'm having trouble understanding how to setup the sueprvisor container. First, it is great to provide a docker image, to save on all the details a new user like me isn't interested in.
But I don't know what kind of setup I need once my container is up. I end up getting
Traceback (most recent call last):
File "/usr/local/bin/minotaur", line 98, in
Minotaur().deploy()
File "/usr/local/bin/minotaur", line 87, in deploy
exec("{0}.main()".format(sys.argv[3]))
File "", line 1, in
File "/deploy/labs/kafka/kafka.py", line 53, in main
str(args.num_nodes), args.instance_type, args.zk_version, args.kafka_url)
File "/deploy/labs/kafka/kafka.py", line 23, in init
vpc_id = self.get_vpc(environment).id
The text was updated successfully, but these errors were encountered: