Better ways to store TLS files explained in more detail

[fernando-villalba]

In your private tls cert module you make some interesting recommendations:

We strongly recommend encrypting the private key file while it's in transit to the servers that will use it. Here are some of the ways you could do this:

Encrypt the certificate using KMS and include the encrypted files in the AMI for your Vault servers. Give those servers an IAM role that lets them access the same KMS key and decrypt their certs just before booting.
Put your TLS cert in a secure S3 Bucket with encryption enabled. Give your Vault servers an IAM role that allows them to download the certs from the S3 bucket just before booting.
Manually upload the certificate to each EC2 Instance with scp

I am interested in learning more about the options you discuss. For example. How do you have KMS decrypt the key on the fly? Vault doesn't seem to provide these options (and in my opinion it should) as it has for auto-unseal. Can you point me to some documentation that explains how to do this with examples and concisely? (This may be a good idea to include in that README as well.)

Again, the s3 bucket idea also seems like a good idea, what's the best practice here? Do you copy the file with a line in user data script the first time the machine boots? Or somehow dynamically do it this every time vault runs? Again, tutorials for this would be enormously helpful.

In my opinion this is super important information to have because the current method described (creating keys and storing in AMI) doesn't feel very secure at all as you rightly mention in the README.

I did some google searches before posting this, but I couldn't find resources that were specific enough for this.

[fernando-villalba]

I solved this by doing the following in case it helps anyone else save time:

Encrypting the TLS key using the aws_kms_ciphertext resource from terraform and writing it to a file. The file resides in a subfolder of vault-consul-ami and because it is encrypted and base64, I am okay with uploading this to a private repository and keep it versioned controlled.

When creating private ami images (for which I also encrypted the boot as it is not done by default in the example provided here) I add the encrypted key to it, this is later decrypted with aws cli command in user data.

Here is an example of the code block used to encrypt the key

resource "aws_kms_ciphertext" "tls_key" {
  key_id = "alias/your-tls-key"

  plaintext = tls_private_key.cert.private_key_pem

    # Store the certificate's private key in a file.
  provisioner "local-exec" {
    command = "echo -n '${aws_kms_ciphertext.tls_key.ciphertext_blob}' > '${var.private_key_file_path}' && chmod ${var.permissions} '${var.private_key_file_path}' && chown ${var.owner} '${var.private_key_file_path}'"
    interpreter = ["bash", "-c"]
  }

}

And here is an example of the command used to decrypt the key in user data file that runs when booting the instance for the first time:

aws kms decrypt --region ${aws_region} --ciphertext-blob fileb://<(base64 --decode $CONSUL_ENCRYPTED_KEY_FILE) \
--output text --query Plaintext | base64 --decode > $CONSUL_TLS_KEY_FILE