Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IAM based auth for S3 policy repo #691

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

kiesenverseist
Copy link

Changes proposed

This pull request introduces an additional method for retrieving credentials to access an s3 bucket. When using Amazon's EKS it is possible to create a service account that corresponds to an IAM role. That role ARN and a web identity token file are injected into running pods and can be used with STS to generate temporary credentials.

This PR makes the following changes to accommodate this:

  • Add the role name and web token file environment variables to be read in by the config. As these are injected by EKS, they do not have the OPAL_ prefix, which is why the default for those fields directly reads from the environment. I'm not sure if this is the best way to do this.
  • Change the build_auth_headers function. I've added an extra branch where if the ARN and token file are present, it will attempt to use that. I have also made that function async as it now has to read a file, and make requests. I've also added additional logging to indicate which authentication method is being used.
  • Add the get_temporary_sts_credentials function. This handles the reading of the token file, and sending the request to STS. It also parses the response. This makes use of a time based cache decorator so that the temporary credentials are refreshed on a timer.
  • Addition of a cache decorator. This is a wrapper around lru_cache adding a time based cache invalidator, as well as code to make it work with async functions.
  • Change the build_aws_rest_auth_headers function to include an optional session token. Temporary credentials also include a session token which needs to be included in auth headers when they are used.

Check List (Check all the applicable boxes)

  • I sign off on contributing this submission to open-source
  • My code follows the code style of this project.
  • My change requires changes to the documentation.
  • I have updated the documentation accordingly.
  • All new and existing tests passed.
  • This PR does not contain plagiarized content.
  • The title of my pull request is a short description of the requested changes.

Note to reviewers

  • I'm not certain I'm reading from the environment in a way that matches the style of the rest of the project.
  • I've added the async decorator, but there do exist libraries that provide this functionality, maybe more robustly. I've erred on the side of not including dependencies, but let me know if it'd be better to use a dependency here.
  • I'm not sure how testable these changes are :/
  • The in code documentation has been updated, is there anywhere else documentation should be updated?

Copy link

netlify bot commented Nov 4, 2024

Deploy Preview for opal-docs canceled.

Name Link
🔨 Latest commit d6b71c1
🔍 Latest deploy log https://app.netlify.com/sites/opal-docs/deploys/67283328392edd00081ae30c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant