Merge upstream changes (#2)

* Allows username validation against IAM groups

This change gives the option to validate the remote username against
the IAM groups containing the user invoking the lambda function. This
is an optional feature which is used in conjunction with kmsauth.

For example, if there were two groups of users, you could put your
admins in the ssh-admin IAM group to allow them to generate certificates
with a remote_username of 'admin'. Users with fewer permissions could be
in the ssh-user group to allow them to generate certificates for the 'user'
account.

The group name is configurable, however they must all be in a consistent
format, and must all contain the relevant remote_username once.

* Compressed CA private key support

* Fixing Netflix#72 thanks @Immortalin and @tuxinaut .

* Add support for loading ED25519 public keys

* Add certificate builder and test ED25519 signed by RSA

* Allowing BLESS lambda to accept ed25519 keys, completing https://gith… (Netflix#74)

* Allowing BLESS lambda to accept ed25519 keys, completing Netflix#71 .  Thanks @jnewbigin .

* Moving BLESS to python 3.6. (Netflix#75)

* Moving BLESS to python 3.6.
You just need to rebuild, publish, and switch your lambda runtime from 2.7 to 3.6.

* Moving TravisCI to Python3.6 as well.

* bless_client.py: fix argv unpacking when using a kmsauth token (Netflix#63)

* Add the FileSync flag to the zip command (Netflix#76)

* Make lambda_configs dir optional for publish make target (Netflix#69)

* Adding a blacklisted remote_usernames option.  This would prevent particular SSH Authorized Principals from being included in a BLESS certificate.

* Refactored BLESS to cache KMS decrypt results for the ca private key password.

* Bumping to Release v.0.3.0

Features include:
Python 3.6 Lambda support
Caching of the KMS decrypted CA Private Key Password.
Compressed CA Private Key support, allowing RSA 4096 keys to be set in the Lambda Environment.
Issue certificates for ED25519 public keys (RSA CA).
New option to validate the remote username against the IAM groups of the calling user.
Updated dependencies.

.gitignore

@@ -7,3 +7,6 @@ htmlcov/
 libs/
 publish/
 venv/
+aws_lambda_libs/
+lambda_configs/
+.pytest_cache/

.travis.yml

@@ -6,7 +6,7 @@ addons:
 
 matrix:
   include:
-    - python: "2.7"
+    - python: "3.6"
 
 install:
   - pip install coveralls

Makefile

@@ -36,18 +36,18 @@ publish:
 	mv ./publish/bless_lambda/bless/aws_lambda/* ./publish/bless_lambda/
 	cp -r ./aws_lambda_libs/. ./publish/bless_lambda/
 	if [ -d ./lambda_configs/ ]; then cp -r ./lambda_configs/. ./publish/bless_lambda/; fi
-	cd ./publish/bless_lambda && zip -r ../bless_lambda.zip .
+	cd ./publish/bless_lambda && zip -FSr ../bless_lambda.zip .
 
 compile:
-	yum install -y gcc libffi-devel openssl-devel python27-virtualenv
-	virtualenv /tmp/venv
+	yum install -y gcc libffi-devel openssl-devel python36 python36-virtualenv
+	virtualenv-3.6 /tmp/venv
 	/tmp/venv/bin/pip install --upgrade pip setuptools
 	/tmp/venv/bin/pip install -e .
-	cp -r /tmp/venv/lib/python2.7/site-packages/. ./aws_lambda_libs
-	cp -r /tmp/venv/lib64/python2.7/site-packages/. ./aws_lambda_libs
+	cp -r /tmp/venv/lib/python3.6/site-packages/. ./aws_lambda_libs
+	cp -r /tmp/venv/lib64/python3.6/site-packages/. ./aws_lambda_libs
 
 lambda-deps:
 	@echo "--> Compiling lambda dependencies"
-	docker run --rm -it -v ${CURDIR}:/src -w /src amazonlinux make compile
+	docker run --rm -it -v ${CURDIR}:/src -w /src amazonlinux:1 make compile
 
 .PHONY: develop dev-docs clean test lint coverage publish

README.md

@@ -33,7 +33,7 @@ Cd to the bless repo:
 
 Create a virtualenv if you haven't already:
 
-    $ virtualenv venv
+    $ python3.6 -m venv venv
 
 Activate the venv:
 
@@ -114,6 +114,16 @@ def lambda_handler(event, context):
 - Provide your desired ./lambda_configs/ca_key_name.pem prior to Publishing a new Lambda .zip
 - Set the permissions of ./lambda_configs/ca_key_name.pem to 444.
 
+You can now provide your private key and/or encrypted private key password via the lambda environment or config file.
+In the `[Bless CA]` section, you can set `ca_private_key` instead of the `ca_private_key_file` with a base64 encoded
+version of your .pem (e.g. `cat key.pem | base64` ).
+
+Because every config file option is supported in the environment, you can also just set `bless_ca_default_password`
+and/or `bless_ca_ca_private_key`.  Due to limits on AWS Lambda environment variables, you'll need to compress RSA 4096
+private keys, which you can now do by setting `bless_ca_ca_private_key_compression`. For example, set 
+`bless_ca_ca_private_key_compression = bz2` and `bless_ca_ca_private_key` to the output of 
+`cat ca-key.pem | bzip2 | base64`.
+
 ### BLESS Config File
 - Refer to the the [Example BLESS Config File](bless/config/bless_deploy_example.cfg) and its
 included documentation.

bless/__about__.py

@@ -1,5 +1,3 @@
-from __future__ import absolute_import, division, print_function
-
 __all__ = [
     "__title__", "__summary__", "__uri__", "__version__", "__author__",
     "__email__", "__license__", "__copyright__",
@@ -11,7 +9,7 @@
     "sign SSH public keys.")
 __uri__ = "https://github.com/Netflix/bless"
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"
 
 __author__ = "The BLESS developers"
 __email__ = "[email protected]"

bless/aws_lambda/bless_lambda.py

@@ -3,18 +3,15 @@
     :copyright: (c) 2016 by Netflix Inc., see AUTHORS for more
     :license: Apache, see LICENSE for more details.
 """
-import base64
+
 import logging
-import os
-import time
 
 import boto3
-from botocore.exceptions import ClientError
-from kmsauth import KMSTokenValidator, TokenValidationError
-from marshmallow.exceptions import ValidationError
+import os
+import time
+from bless.cache.bless_lambda_cache import BlessLambdaCache
 
-from bless.config.bless_config import BlessConfig, \
-    BLESS_OPTIONS_SECTION, \
+from bless.config.bless_config import BLESS_OPTIONS_SECTION, \
     CERTIFICATE_VALIDITY_BEFORE_SEC_OPTION, \
     CERTIFICATE_VALIDITY_AFTER_SEC_OPTION, \
     ENTROPY_MINIMUM_BITS_OPTION, \
@@ -24,20 +21,25 @@
     KMSAUTH_SECTION, \
     KMSAUTH_USEKMSAUTH_OPTION, \
     KMSAUTH_REMOTE_USERNAMES_ALLOWED_OPTION, \
+    VALIDATE_REMOTE_USERNAMES_AGAINST_IAM_GROUPS_OPTION, \
     KMSAUTH_SERVICE_ID_OPTION, \
     TEST_USER_OPTION, \
     CERTIFICATE_EXTENSIONS_OPTION, \
-    REMOTE_USERNAMES_VALIDATION_OPTION
+    REMOTE_USERNAMES_VALIDATION_OPTION, \
+    IAM_GROUP_NAME_VALIDATION_FORMAT_OPTION, \
+    REMOTE_USERNAMES_BLACKLIST_OPTION
 from bless.request.bless_request import BlessSchema
 from bless.ssh.certificate_authorities.ssh_certificate_authority_factory import \
     get_ssh_certificate_authority
 from bless.ssh.certificates.ssh_certificate_builder import SSHCertificateType
 from bless.ssh.certificates.ssh_certificate_builder_factory import get_ssh_certificate_builder
+from kmsauth import KMSTokenValidator, TokenValidationError
+from marshmallow.exceptions import ValidationError
+
+global_bless_cache = None
 
 
-def lambda_handler(event, context=None, ca_private_key_password=None,
-                   entropy_check=True,
-                   config_file=os.path.join(os.path.dirname(__file__), 'bless_deploy.cfg')):
+def lambda_handler(event, context=None, ca_private_key_password=None, entropy_check=True, config_file=None):
     """
     This is the function that will be called when the lambda function starts.
     :param event: Dictionary of the json request.
@@ -46,16 +48,25 @@ def lambda_handler(event, context=None, ca_private_key_password=None,
     :param ca_private_key_password: For local testing, if the password is provided, skip the KMS
     decrypt.
     :param entropy_check: For local testing, if set to false, it will skip checking entropy and
-    won't try to fetch additional random from KMS
-    :param config_file: The config file to load the SSH CA private key from, and additional settings
+    won't try to fetch additional random from KMS.
+    :param config_file: The config file to load the SSH CA private key from, and additional settings.
     :return: the SSH Certificate that can be written to id_rsa-cert.pub or similar file.
     """
+    # For testing, ignore the static bless_cache, otherwise fill the cache one time.
+    global global_bless_cache
+    if ca_private_key_password is not None or config_file is not None:
+        bless_cache = BlessLambdaCache(ca_private_key_password, config_file)
+    elif global_bless_cache is None:
+        global_bless_cache = BlessLambdaCache(config_file=os.path.join(os.path.dirname(__file__), 'bless_deploy.cfg'))
+        bless_cache = global_bless_cache
+    else:
+        bless_cache = global_bless_cache
+
     # AWS Region determines configs related to KMS
-    region = os.environ['AWS_REGION']
+    region = bless_cache.region
 
     # Load the deployment config values
-    config = BlessConfig(region,
-                         config_file=config_file)
+    config = bless_cache.config
 
     logging_level = config.get(BLESS_OPTIONS_SECTION, LOGGING_LEVEL_OPTION)
     numeric_level = getattr(logging, logging_level.upper(), None)
@@ -72,14 +83,15 @@ def lambda_handler(event, context=None, ca_private_key_password=None,
     entropy_minimum_bits = config.getint(BLESS_OPTIONS_SECTION, ENTROPY_MINIMUM_BITS_OPTION)
     random_seed_bytes = config.getint(BLESS_OPTIONS_SECTION, RANDOM_SEED_BYTES_OPTION)
     ca_private_key = config.getprivatekey()
-    password_ciphertext_b64 = config.getpassword()
     certificate_extensions = config.get(BLESS_OPTIONS_SECTION, CERTIFICATE_EXTENSIONS_OPTION)
 
     # Process cert request
     schema = BlessSchema(strict=True)
     schema.context[USERNAME_VALIDATION_OPTION] = config.get(BLESS_OPTIONS_SECTION, USERNAME_VALIDATION_OPTION)
     schema.context[REMOTE_USERNAMES_VALIDATION_OPTION] = config.get(BLESS_OPTIONS_SECTION,
                                                                     REMOTE_USERNAMES_VALIDATION_OPTION)
+    schema.context[REMOTE_USERNAMES_BLACKLIST_OPTION] = config.get(BLESS_OPTIONS_SECTION,
+                                                                   REMOTE_USERNAMES_BLACKLIST_OPTION)
 
     try:
         request = schema.load(event).data
@@ -92,15 +104,11 @@ def lambda_handler(event, context=None, ca_private_key_password=None,
         request.public_key_to_sign,
         request.kmsauth_token))
 
-    # decrypt ca private key password
-    if ca_private_key_password is None:
-        kms_client = boto3.client('kms', region_name=region)
-        try:
-            ca_password = kms_client.decrypt(
-                CiphertextBlob=base64.b64decode(password_ciphertext_b64))
-            ca_private_key_password = ca_password['Plaintext']
-        except ClientError as e:
-            return error_response('ClientError', str(e))
+    # Make sure we have the ca private key password
+    if bless_cache.ca_private_key_password is None:
+        return error_response('ClientError', bless_cache.ca_private_key_password_error)
+    else:
+        ca_private_key_password = bless_cache.ca_private_key_password
 
     # if running as a Lambda, we can check the entropy pool and seed it with KMS if desired
     if entropy_check:
@@ -112,6 +120,7 @@ def lambda_handler(event, context=None, ca_private_key_password=None,
                     'System entropy was {}, which is lower than the entropy_'
                     'minimum {}.  Using KMS to seed /dev/urandom'.format(
                         entropy, entropy_minimum_bits))
+                kms_client = boto3.client('kms', region_name=bless_cache.region)
                 response = kms_client.generate_random(
                     NumberOfBytes=random_seed_bytes)
                 random_seed = response['Plaintext']
@@ -143,9 +152,30 @@ def lambda_handler(event, context=None, ca_private_key_password=None,
                 if allowed_users != ['*'] and not all([u in allowed_users for u in requested_remotes]):
                     return error_response('KMSAuthValidationError',
                                           'unallowed remote_usernames [{}]'.format(request.remote_usernames))
+
+                # Check if the user is in the required IAM groups
+                if config.get(KMSAUTH_SECTION, VALIDATE_REMOTE_USERNAMES_AGAINST_IAM_GROUPS_OPTION):
+                    iam = boto3.client('iam')
+                    user_groups = iam.list_groups_for_user(UserName=request.bastion_user)
+
+                    group_name_template = config.get(KMSAUTH_SECTION, IAM_GROUP_NAME_VALIDATION_FORMAT_OPTION)
+                    for requested_remote in requested_remotes:
+                        required_group_name = group_name_template.format(requested_remote)
+
+                        user_is_in_group = any(
+                            group
+                            for group in user_groups['Groups']
+                            if group['GroupName'] == required_group_name
+                        )
+
+                        if not user_is_in_group:
+                            return error_response('KMSAuthValidationError',
+                                                  'user {} is not in the {} iam group'.format(request.bastion_user,
+                                                                                              required_group_name))
+
             elif request.remote_usernames != request.bastion_user:
-                    return error_response('KMSAuthValidationError',
-                                          'remote_usernames must be the same as bastion_user')
+                return error_response('KMSAuthValidationError',
+                                      'remote_usernames must be the same as bastion_user')
             try:
                 validator = KMSTokenValidator(
                     None,

bless/cache/bless_lambda_cache.py

@@ -0,0 +1,44 @@
+import base64
+import os
+
+import boto3
+from bless.config.bless_config import BlessConfig
+from botocore.exceptions import ClientError
+
+
+class BlessLambdaCache:
+    region = None
+    config = None
+    ca_private_key_password = None
+    ca_private_key_password_error = None
+
+    def __init__(self, ca_private_key_password=None,
+                 config_file=None):
+        """
+
+        :param ca_private_key_password: For local testing, if the password is provided, skip the KMS
+        decrypt.
+        :param config_file: The config file to load the SSH CA private key from, and additional settings.
+        """
+        # AWS Region determines configs related to KMS
+        if 'AWS_REGION' in os.environ:
+            self.region = os.environ['AWS_REGION']
+        else:
+            self.region = 'us-west-2'
+
+            # Load the deployment config values
+        self.config = BlessConfig(self.region, config_file=config_file)
+
+        password_ciphertext_b64 = self.config.getpassword()
+
+        # decrypt ca private key password
+        if ca_private_key_password is None:
+            kms_client = boto3.client('kms', region_name=self.region)
+            try:
+                ca_password = kms_client.decrypt(
+                    CiphertextBlob=base64.b64decode(password_ciphertext_b64))
+                self.ca_private_key_password = ca_password['Plaintext']
+            except ClientError as e:
+                self.ca_private_key_password_error = str(e)
+        else:
+            self.ca_private_key_password = ca_private_key_password