Skip to content

civichacker/aws-formula

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

aws-formula

Configure the non-server parts of an AWS datacenter without using CloudFormation. Multiple VPCs in multiple regions can be configured with one pillar.

Requires: Saltstack 2016.11 ( 2015.8.2 if not using NAT Gateway ) and Boto3

Available states

Create VPC objects, including:

  • VPC
  • Internet Gateway
  • NAT Gateway
  • Subnets
  • Routing Tables

Create ec2 components including:

  • Key pairs

Create and update Security Groups rules. Servers must be added to the Security Groups as part of the salt-cloud creation proces.

Limitations and Workarounds

The salt boto states used in this formula have some limitations that do not allow for the one pass full creation of a VPC datacenter. The current issues are:

  • NAT Gateways can not be added to a routing table by name. ( Found the undocumented nat_gateway_subnet_name option that would resolve this issue, but it is not working per Salt Issue 38791. )
  • Inter-Region VPN connections can not be added to routing table

NAT Gateway Workaround

The workaround for the nat gateway is to do an initial salt state run to create the gateways, then add the NAT Gateway's nat_gateway_id to the routing tables in the pillar and re-run the state. In the sample pillar the default Gateways are commented out to indicate where the nat_gateway_id should go.

Inter-Region VPN

AWS currently does not offer a service for VPC peering between regions. This means the recommended solution is to create a VPN instance and then use an ipsec VPN tunnel between regions. Unfortunately, when using VPN instances you can not use the Route Propagation functionality, so those routes have to be manually added to all routing tables for a VPC. VPN routing is shown commented out in the pillar example. Like above, run the states once to create all of the objects. Then when the VPN tunnel is created/active, add the IPs to the pillar and rerun the states.

Configuration

All configuration is done through the AWS pillar. The hierarchy of this pillar is:

aws:
  region:
    us-east-2:
      key_pairs:
      profile:
      vpc:
        myvpc:
          vpc:
          internet_gateway:
          subnets:
          routing_tables:
          routing_global_routes:
          security_groups:

In this hierarchy, the 3rd level ( us-east-2 ) is a region name and the 5th level ( myvpc ) is a vpc name. These are the names that will be used for the region and the name of the VPC. All items besides internet_gateway can have multiple values.

The below examples and the sample pillar uses single quotes in many places to ensure data is interepreted correctly. Not using quotes per the examples is done at your own risk.

Key pairs are included under at the region level since they are not generally VPC specific. Key pair format is a key pair with the name and RSA publi key.

key_pairs:
  mykey: 'ssh-rsa XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [email protected]'

This formula uses an AWS profile for all states instead of the individual fields. The key and keyid should be gpg encrypted using the Saltstack gpg renderer. Example below shows it in unencrypted format.

profile:
  region: us-east-2
  keyid: ASDFASDFASDFASDFASDF
  key: AB12Cd3Efg45hIjk67lMNop8q9RST0uvwXyz

VPC contains vpcs for a given region. Each vpc will have data for all VPC specific states, even if they are not in the vpc.sls. The vpc pillar name is the name that will be used for the VPC in AWS. The only data directly under the vpc name is the CIDR block for the VPC. This Formula is designed using a class B network for the VPC and class C for all subnets.

vpc:
  myvpc:
    cidr_prefix: '10.10'

the VPC subsection contains the data needed to create the VPC. The names on the left are the configuration item names from the boto_vpc.present states. The vpc pillar name should always match the name in the vpc section beneath. The cidr_block should start with the same two octets as the cidr_prefix above.

vpc:
  myvpc:
    vpc:
      name: myvpc
      cidr_block: 10.10.0.0/16
      instance_tennancy: default
      dns_support: 'true'
      dns_hostnames: 'true'

An internet gateway is needed for most use cases.

vpc:
  myvpc:
    internet_gateway:
      name: internet_gateway

Subnets are named by their subnet ID ( assumes we are using class C subnets). The subnet ID will be appended to the cidr_prefix above to create the CIDR or the subnet. Every subnet has to at least have a subnet name and availability zone. if nat_gateway is specified, then a NAT Gateway will be created in that subnet. Subnet associations are done in the Routing Table section below.

vpc:
  myvpc:
    subnets:
      1:
        name: subWebA
        az: a
        nat_gateway: true
      11:
        name: appwebA
        az: a

The above example would create two subnets:

  • subWebA with CIDR 10.10.1.0/24
  • subAppA with CIDR 10.10.1.0/24

Both are in Availability Zone A and a NAT Gateway would be created in subWebA.

Routing tables will create the tables, add routes, and assign subnets to routing tables. The below example include the interface_id of a already created NAT Gateway.

vpc:
  myvpc:
    routig_tables:
      publicA:
        routes:
          default:
            destination_cidr_block: 0.0.0.0/0
            internet_gateway_name: internet_gateway
        subnet_names:
          - subWebA
      privateA:
        routes:
          default:
            destination_cidr_block: 0.0.0.0/0
            nat_gateway_id: 'nat-0abcdef123546ghi'
        subnet_names:
          - subAppA

Routes that will be added to all routing tables. Use this for adding vpn routes.

vpc:
  myvpc:
    routing_global_routes:
      vpnPROD:
        destination_cidr_block: '10.10.0.0/16'
        instance_id: 'i-xxxxxxxxxxxxxxx'

Create security groups and rules. Usage notes:

  • If a single port is being specified, the from_port and to port can be replace with just port.
  • source_group_name and cidr_ip can be either a single item or a list.
  • Use port: -1 to specify all ports
  • A rules pillar name is for information purposes only and is not used in the actual rule creation.s
vpc:
  myvpc:
    security_groups:
      sgApp-myvpc:
        description: SG for all App servers
        rules:
          http:
            ip_protocol: tcp
            port: 80
            source_group_name:
              - sgWeb-myvpc
              - sgApp-myvpc
        rules_egress:
          all:
            ip_protocol: all
            port: -1
            cidr_ip: '0.0.0.0/0'
      sgSalt-myvpc:
        description: SG for all Salt servers
        rules:
          salt-master:
            ip_protocol: tcp
            from_port: 4505
            to_port: 4506
            cidr_ip: '10.10.0.0/16'
          salt-api:
            ip_protocol: tcp
            port: 443
            cidr_ip:
              - '10.10.0.0/16'
              - '10.20.0.0/16'

About

Saltstack AWS creates all the non-server components of a AWS hosted datacenter

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • SaltStack 78.4%
  • HTML 21.6%