Merge pull request #39 from lyft/refresh-netflix-code

Pull upstream changes in netflix/bless into lyft's fork lyft/bless

.gitignore

@@ -7,3 +7,6 @@ htmlcov/
 libs/
 publish/
 venv/
+aws_lambda_libs/
+lambda_configs/
+.pytest_cache/

.travis.yml

@@ -1,12 +1,10 @@
-sudo: false
-
 language: python
 
 addons:
 
 matrix:
   include:
-    - python: "2.7"
+    - python: "3.7"
 
 install:
   - pip install coveralls

Makefile

@@ -5,6 +5,7 @@ test: lint
 
 develop:
 	@echo "--> Installing dependencies"
+	pip install --upgrade pip setuptools
 	pip install -r requirements.txt
 	pip install "file://`pwd`#egg=bless[tests]"
 	@echo ""
@@ -33,21 +34,16 @@ publish:
 	rm -rf ./publish/bless_lambda/
 	mkdir -p ./publish/bless_lambda
 	cp -r ./bless ./publish/bless_lambda/
-	mv ./publish/bless_lambda/bless/aws_lambda/* ./publish/bless_lambda/
+	cp ./publish/bless_lambda/bless/aws_lambda/bless* ./publish/bless_lambda/
 	cp -r ./aws_lambda_libs/. ./publish/bless_lambda/
-	cp -r ./lambda_configs/. ./publish/bless_lambda/
-	cd ./publish/bless_lambda && zip -r ../bless_lambda.zip .
+	if [ -d ./lambda_configs/ ]; then cp -r ./lambda_configs/. ./publish/bless_lambda/; fi
+	cd ./publish/bless_lambda && zip -FSr ../bless_lambda.zip .
 
 compile:
-	yum install -y gcc libffi-devel openssl-devel python27-virtualenv
-	virtualenv /tmp/venv
-	/tmp/venv/bin/pip install --upgrade pip setuptools
-	/tmp/venv/bin/pip install -e .
-	cp -r /tmp/venv/lib/python2.7/site-packages/. ./aws_lambda_libs
-	cp -r /tmp/venv/lib64/python2.7/site-packages/. ./aws_lambda_libs
+	./lambda_compile.sh
 
 lambda-deps:
 	@echo "--> Compiling lambda dependencies"
-	docker run --rm -it -v ${CURDIR}:/src -w /src amazonlinux make compile
+	docker run --rm -v ${CURDIR}:/src -w /src amazonlinux:2 ./lambda_compile.sh
 
 .PHONY: develop dev-docs clean test lint coverage publish

README.md

@@ -2,10 +2,10 @@
 # BLESS - Bastion's Lambda Ephemeral SSH Service
 [![Build Status](https://travis-ci.org/Netflix/bless.svg?branch=master)](https://travis-ci.org/Netflix/bless) [![Test coverage](https://coveralls.io/repos/github/Netflix/bless/badge.svg?branch=master)](https://coveralls.io/github/Netflix/bless) [![Join the chat at https://gitter.im/Netflix/bless](https://badges.gitter.im/Netflix/bless.svg)](https://gitter.im/Netflix/bless?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![NetflixOSS Lifecycle](https://img.shields.io/osslifecycle/Netflix/bless.svg)]()
 
-BLESS is an SSH Certificate Authority that runs as a AWS Lambda function and is used to sign ssh
+BLESS is an SSH Certificate Authority that runs as an AWS Lambda function and is used to sign SSH
 public keys.
 
-SSH Certificates are an excellent way to authorize users to access a particular ssh host,
+SSH Certificates are an excellent way to authorize users to access a particular SSH host,
 as they can be restricted for a single use case, and can be short lived.  Instead of managing the
 authorized_keys of a host, or controlling who has access to SSH Private Keys, hosts just
 need to be configured to trust an SSH CA.
@@ -33,7 +33,7 @@ Cd to the bless repo:
 
 Create a virtualenv if you haven't already:
 
-    $ virtualenv venv
+    $ python3.7 -m venv venv
 
 Activate the venv:
 
@@ -52,42 +52,29 @@ Run the tests:
 To deploy an AWS Lambda Function, you need to provide a .zip with the code and all dependencies.
 The .zip must contain your lambda code and configurations at the top level of the .zip.  The BLESS
 Makefile includes a publish target to package up everything into a deploy-able .zip if they are in
-the expected locations.
+the expected locations.  You will need to setup your own Python 3.7 lambda to deploy the .zip to.
+
+Previously the AWS Lambda Handler needed to be set to `bless_lambda.lambda_handler`, and this would generate a user 
+cert.  `bless_lambda.lambda_handler` still works for user certs.  `bless_lambda_user.lambda_handler_user` is a handler 
+that can also be used to issue user certificates.
+
+A new handler `bless_lambda_host.lambda_handler_host` has been created to allow for the creation of host SSH certs.
+
+All three handlers exist in the published .zip.
 
 ### Compiling BLESS Lambda Dependencies
-AWS Lambda has some limitations, and to deploy code as a Lambda Function, you need to package up
-all of the dependencies.  AWS Lambda only supports Python 2.7 and BLESS depends on
-[Cryptography](https://cryptography.io/en/latest/), which must be compiled.  You will need to
+To deploy code as a Lambda Function, you need to package up all of the dependencies.  You will need to
 compile and include your dependencies before you can publish a working AWS Lambda.
 
-You can use a docker container running amazon linux:
+BLESS uses a docker container running [Amazon Linux 2](https://hub.docker.com/_/amazonlinux) to package everything up:
 - Execute ```make lambda-deps``` and this will run a container and save all the dependencies in ./aws_lambda_libs
 
-Alternatively you can:
-- Deploy an [Amazon Linux AMI](http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html)
-- SSH onto that instance
-- Copy BLESS' `setup.py` to the instance
-- Copy BLESS' `bless/__about__.py` to the instance at `bless/__about__.py`
-- Install BLESS' dependencies:
-```
-$ sudo yum install gcc libffi-devel openssl-devel
-$ virtualenv venv
-$ source venv/bin/activate
-(venv) $ pip install --upgrade pip setuptools
-(venv) $ pip install -e .
-```
-- From that instance, copy off the contents of:
-```
-$ cp -r venv/lib/python2.7/site-packages/. aws_lambda_libs
-$ cp -r venv/lib64/python2.7/site-packages/. aws_lambda_libs
-```
-- put those files in: ./aws_lambda_libs/
-
 ### Protecting the CA Private Key
-- Generate a password protected RSA Private Key:
+- Generate a password protected RSA Private Key in the PEM format:
 ```
-$ ssh-keygen -t rsa -b 4096 -f bless-ca- -C "SSH CA Key"
+$ ssh-keygen -t rsa -b 4096 -m PEM -f bless-ca- -C "SSH CA Key"
 ```
+- **Note:** OpenSSH Private Key format is not supported.
 - Use KMS to encrypt your password.  You will need a KMS key per region, and you will need to
 encrypt your password for each region.  You can use the AWS Console to paste in a simple lambda
 function like this:
@@ -114,13 +101,23 @@ def lambda_handler(event, context):
 - Provide your desired ./lambda_configs/ca_key_name.pem prior to Publishing a new Lambda .zip
 - Set the permissions of ./lambda_configs/ca_key_name.pem to 444.
 
+You can now provide your private key and/or encrypted private key password via the lambda environment or config file.
+In the `[Bless CA]` section, you can set `ca_private_key` instead of the `ca_private_key_file` with a base64 encoded
+version of your .pem (e.g. `cat key.pem | base64` ).
+
+Because every config file option is supported in the environment, you can also just set `bless_ca_default_password`
+and/or `bless_ca_ca_private_key`.  Due to limits on AWS Lambda environment variables, you'll need to compress RSA 4096
+private keys, which you can now do by setting `bless_ca_ca_private_key_compression`. For example, set 
+`bless_ca_ca_private_key_compression = bz2` and `bless_ca_ca_private_key` to the output of 
+`cat ca-key.pem | bzip2 | base64`.
+
 ### BLESS Config File
 - Refer to the the [Example BLESS Config File](bless/config/bless_deploy_example.cfg) and its
 included documentation.
 - Manage your bless_deploy.cfg files outside of this repo.
 - Provide your desired ./lambda_configs/bless_deploy.cfg prior to Publishing a new Lambda .zip
 - The required [Bless CA] option values must be set for your environment.
-- Every option can be changed in the environment. The environment variable name is contructed
+- Every option can be changed in the environment. The environment variable name is constructed
 as section_name_option_name (all lowercase, spaces replaced with underscores).
 
 ### Publish Lambda .zip
@@ -152,6 +149,8 @@ random from kms (kms:GenerateRandom) and permissions for logging to CloudWatch L
 ## Using BLESS
 After you have [deployed BLESS](#deployment) you can run the sample [BLESS Client](bless_client/bless_client.py)
 from a system with access to the required [AWS Credentials](http://boto3.readthedocs.io/en/latest/guide/configuration.html).
+This client is really just a proof of concept to validate that you have a functional lambda being called with valid
+IAM credentials. 
 
     (venv) $ ./bless_client.py region lambda_function_name bastion_user bastion_user_ip remote_usernames bastion_source_ip bastion_command <id_rsa.pub to sign> <output id_rsa-cert.pub>
 
@@ -162,11 +161,11 @@ You can inspect the contents of a certificate with ssh-keygen directly:
     $ ssh-keygen -L -f your-cert.pub
 
 ## Enabling BLESS Certificates On Servers
-Add the following line to /etc/ssh/sshd_config:
+Add the following line to `/etc/ssh/sshd_config`:
 
     TrustedUserCAKeys /etc/ssh/cas.pub
 
-Add a new file, owned by and only writable by root, at /etc/ssh/cas.pub with the contents:
+Add a new file, owned by and only writable by root, at `/etc/ssh/cas.pub` with the contents:
 
     ssh-rsa AAAAB3NzaC1yc2EAAAADAQ…  #id_rsa.pub of an SSH CA
     ssh-rsa AAAAB3NzaC1yc2EAAAADAQ…  #id_rsa.pub of an offline SSH CA

bless/__about__.py

@@ -1,5 +1,3 @@
-from __future__ import absolute_import, division, print_function
-
 __all__ = [
     "__title__", "__summary__", "__uri__", "__version__", "__author__",
     "__email__", "__license__", "__copyright__",
@@ -11,7 +9,7 @@
     "sign SSH public keys.")
 __uri__ = "https://github.com/Netflix/bless"
 
-__version__ = "0.2.0"
+__version__ = "0.4.0"
 
 __author__ = "The BLESS developers"
 __email__ = "[email protected]"

bless/aws_lambda/bless_lambda.py

@@ -3,191 +3,11 @@
     :copyright: (c) 2016 by Netflix Inc., see AUTHORS for more
     :license: Apache, see LICENSE for more details.
 """
-import base64
-import logging
-import os
-import time
+from bless.aws_lambda.bless_lambda_user import lambda_handler_user
 
-import boto3
-from botocore.exceptions import ClientError
-from kmsauth import KMSTokenValidator, TokenValidationError
-from marshmallow.exceptions import ValidationError
 
-from bless.config.bless_config import BlessConfig, \
-    BLESS_OPTIONS_SECTION, \
-    CERTIFICATE_VALIDITY_BEFORE_SEC_OPTION, \
-    CERTIFICATE_VALIDITY_AFTER_SEC_OPTION, \
-    ENTROPY_MINIMUM_BITS_OPTION, \
-    RANDOM_SEED_BYTES_OPTION, \
-    LOGGING_LEVEL_OPTION, \
-    KMSAUTH_SECTION, \
-    KMSAUTH_USEKMSAUTH_OPTION, \
-    KMSAUTH_SERVICE_ID_OPTION, \
-    TEST_USER_OPTION, \
-    CERTIFICATE_EXTENSIONS_OPTION
-from bless.request.bless_request import BlessSchema
-from bless.ssh.certificate_authorities.ssh_certificate_authority_factory import \
-    get_ssh_certificate_authority
-from bless.ssh.certificates.ssh_certificate_builder import SSHCertificateType
-from bless.ssh.certificates.ssh_certificate_builder_factory import get_ssh_certificate_builder
-
-
-def lambda_handler(event, context=None, ca_private_key_password=None,
-                   entropy_check=True,
-                   config_file=os.path.join(os.path.dirname(__file__), 'bless_deploy.cfg')):
+def lambda_handler(*args, **kwargs):
     """
-    This is the function that will be called when the lambda function starts.
-    :param event: Dictionary of the json request.
-    :param context: AWS LambdaContext Object
-    http://docs.aws.amazon.com/lambda/latest/dg/python-context-object.html
-    :param ca_private_key_password: For local testing, if the password is provided, skip the KMS
-    decrypt.
-    :param entropy_check: For local testing, if set to false, it will skip checking entropy and
-    won't try to fetch additional random from KMS
-    :param config_file: The config file to load the SSH CA private key from, and additional settings
-    :return: the SSH Certificate that can be written to id_rsa-cert.pub or similar file.
+    Wrapper around lambda_handler_user for backwards compatibility
     """
-    # AWS Region determines configs related to KMS
-    region = os.environ['AWS_REGION']
-
-    # Load the deployment config values
-    config = BlessConfig(region,
-                         config_file=config_file)
-
-    logging_level = config.get(BLESS_OPTIONS_SECTION, LOGGING_LEVEL_OPTION)
-    numeric_level = getattr(logging, logging_level.upper(), None)
-    if not isinstance(numeric_level, int):
-        raise ValueError('Invalid log level: {}'.format(logging_level))
-
-    logger = logging.getLogger()
-    logger.setLevel(numeric_level)
-
-    certificate_validity_before_seconds = config.getint(BLESS_OPTIONS_SECTION,
-                                                        CERTIFICATE_VALIDITY_BEFORE_SEC_OPTION)
-    certificate_validity_after_seconds = config.getint(BLESS_OPTIONS_SECTION,
-                                                       CERTIFICATE_VALIDITY_AFTER_SEC_OPTION)
-    entropy_minimum_bits = config.getint(BLESS_OPTIONS_SECTION, ENTROPY_MINIMUM_BITS_OPTION)
-    random_seed_bytes = config.getint(BLESS_OPTIONS_SECTION, RANDOM_SEED_BYTES_OPTION)
-    ca_private_key = config.getprivatekey()
-    password_ciphertext_b64 = config.getpassword()
-    certificate_extensions = config.get(BLESS_OPTIONS_SECTION, CERTIFICATE_EXTENSIONS_OPTION)
-
-    # Process cert request
-    schema = BlessSchema(strict=True)
-    try:
-        request = schema.load(event).data
-    except ValidationError as e:
-        return error_response('InputValidationError', str(e))
-
-    logger.info('Bless lambda invoked by [user: {0}, bastion_ips:{1}, public_key: {2}, kmsauth_token:{3}]'.format(
-        request.bastion_user,
-        request.bastion_user_ip,
-        request.public_key_to_sign,
-        request.kmsauth_token))
-
-    # decrypt ca private key password
-    if ca_private_key_password is None:
-        kms_client = boto3.client('kms', region_name=region)
-        try:
-            ca_password = kms_client.decrypt(
-                CiphertextBlob=base64.b64decode(password_ciphertext_b64))
-            ca_private_key_password = ca_password['Plaintext']
-        except ClientError as e:
-            return error_response('ClientError', str(e))
-
-    # if running as a Lambda, we can check the entropy pool and seed it with KMS if desired
-    if entropy_check:
-        with open('/proc/sys/kernel/random/entropy_avail', 'r') as f:
-            entropy = int(f.read())
-            logger.debug(entropy)
-            if entropy < entropy_minimum_bits:
-                logger.info(
-                    'System entropy was {}, which is lower than the entropy_'
-                    'minimum {}.  Using KMS to seed /dev/urandom'.format(
-                        entropy, entropy_minimum_bits))
-                response = kms_client.generate_random(
-                    NumberOfBytes=random_seed_bytes)
-                random_seed = response['Plaintext']
-                with open('/dev/urandom', 'w') as urandom:
-                    urandom.write(random_seed)
-
-    # cert values determined only by lambda and its configs
-    current_time = int(time.time())
-    test_user = config.get(BLESS_OPTIONS_SECTION, TEST_USER_OPTION)
-    if (test_user and (request.bastion_user == test_user or
-            request.remote_usernames == test_user)):
-        # This is a test call, the lambda will issue an invalid
-        # certificate where valid_before < valid_after
-        valid_before = current_time
-        valid_after = current_time + 1
-        bypass_time_validity_check = True
-    else:
-        valid_before = current_time + certificate_validity_after_seconds
-        valid_after = current_time - certificate_validity_before_seconds
-        bypass_time_validity_check = False
-
-    # Authenticate the user with KMS, if key is setup
-    if config.get(KMSAUTH_SECTION, KMSAUTH_USEKMSAUTH_OPTION):
-        if request.kmsauth_token:
-            try:
-                validator = KMSTokenValidator(
-                    None,
-                    config.getkmsauthkeyids(),
-                    config.get(KMSAUTH_SECTION, KMSAUTH_SERVICE_ID_OPTION),
-                    region
-                )
-                # decrypt_token will raise a TokenValidationError if token doesn't match
-                validator.decrypt_token(
-                    "2/user/{}".format(request.remote_usernames),
-                    request.kmsauth_token
-                )
-            except TokenValidationError as e:
-                return error_response('KMSAuthValidationError', str(e))
-        else:
-            return error_response('InputValidationError', 'Invalid request, missing kmsauth token')
-
-    # Build the cert
-    ca = get_ssh_certificate_authority(ca_private_key, ca_private_key_password)
-    cert_builder = get_ssh_certificate_builder(ca, SSHCertificateType.USER,
-                                               request.public_key_to_sign)
-    for username in request.remote_usernames.split(','):
-        cert_builder.add_valid_principal(username)
-
-    cert_builder.set_valid_before(valid_before)
-    cert_builder.set_valid_after(valid_after)
-
-    if certificate_extensions:
-        for e in certificate_extensions.split(','):
-            if e:
-                cert_builder.add_extension(e)
-    else:
-        cert_builder.clear_extensions()
-
-    # cert_builder is needed to obtain the SSH public key's fingerprint
-    key_id = 'request[{}] for[{}] from[{}] command[{}] ssh_key[{}]  ca[{}] valid_to[{}]'.format(
-        context.aws_request_id, request.bastion_user, request.bastion_user_ip, request.command,
-        cert_builder.ssh_public_key.fingerprint, context.invoked_function_arn,
-        time.strftime("%Y/%m/%d %H:%M:%S", time.gmtime(valid_before)))
-    cert_builder.set_critical_option_source_addresses(request.bastion_ips)
-    cert_builder.set_key_id(key_id)
-    cert = cert_builder.get_cert_file(bypass_time_validity_check)
-
-    logger.info(
-        'Issued a cert to bastion_ips[{}] for remote_usernames[{}] with key_id[{}] and '
-        'valid_from[{}])'.format(
-            request.bastion_ips, request.remote_usernames, key_id,
-            time.strftime("%Y/%m/%d %H:%M:%S", time.gmtime(valid_after))))
-    return success_response(cert)
-
-
-def success_response(cert):
-    return {
-        'certificate': cert
-    }
-
-
-def error_response(error_type, error_message):
-    return {
-        'errorType': error_type,
-        'errorMessage': error_message
-    }
+    return lambda_handler_user(*args, **kwargs)