Possible to pass a JWT token to tasks?

[gigi-at-zymergen]

We are trying out Flyte in our microservices ecosystem. Our desired flow would be the following:

We have edge applications with UIs in front of them, which authenticate end users and obtain a JWT.

That edge application would then use the Flyte API to trigger a registered task/workflow.
Many of our tasks need to make requests against other services which require presenting that JWT to access data on behalf of the end user.

We are currently passing the token as an input to the workflows in our test environment. This has the disadvantage of being visible in plaintext as part of the execution. These tokens do expire, and we can hide direct Flyte access behind other layers, but we would generally want to expose most of the inputs used for an execution to the end users.

We haven't yet configured Oauth2 for the Flyte admin API itself, though I have read https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html .

My understanding is that this addresses auth for the Flyte admin API itself, but I don't see any discussion of how to pass a JWT to tasks, so that they can make calls against other services on behalf of an edge authenticated user?

[kumare3]

@gigi-at-zymergen firstly welcome to the community. Thank you for the question. So to understand correctly,
Option 1 - you have temporary jwt tokens that you would want to inject into each Flyte task dynamically per execution? OR
Option 2 - you would like to use jwt tokens in your flyte tasks that would limit the access to your other services (like service auth).

Flyte supports Using Secrets in a flyte task. But these secrets are more like service auth tokens, that you provision (and periodically rotate).

If you want a new token for execution, then you will have to 1. use some encryption system and pass the decryption key using the flyte secrets system, or you will have to write your own Admission webhook that injects secrets somehow (seems hard). you can leverage Flyte secrets to do that - as it is a webhook, as long as you have different secrets for each.

But. I may be misunderstanding. cc @EngHabu what do you think?

BTW please join our slack community and we could probably help you even faster

[EngHabu]

Hey @gigi-at-zymergen,
Thank you for your question. I agree with Ketan. I would like to go a couple of steps back and revisit the security model...

JWT tokens are bearer token. They offer no inherit security mechanism that proves ownership of the JWT token. The best way you secure them is by NOT sharing them. They are issued by an identity provider (IdP) to be used by X to talk to Y. They should not be passed by X to Z because that just breaks the security model fundamentally... If they are intended for Z to use them, then IdP should deliver them to Z directly (through AuthCode OAuth2 flow or encryption or similar).

In your scenario, how does the edge service generate/issue such a JWT token? does it authenticate to an IdP to do so? does it issue it itself (essentially acting as an IdP) ?
JWT tokens have expiration, what happens if they expire while the WF is still running? Is there a refresh mechanism? WFs should just fail?

Encrypting these tokens and using Secrets to decrypt them will solve the proof issue because then only the intended consumer (Pods) can use them... using Flyte Secrets is the right way for that. This, however, won't solve the renewal/refresh issue.

If you can consider that:

1. You trust the task code
2. You trust that only the edge service can update particular inputs for a launchplan,

Then you can do something like in this diagram:

Essentially you are provisioning a secret that Pods can use to issue the JWT tokens themselves... This alleviates the renewal/refresh issue. Keeps JWT tokens themselves secure (they are not passed around). But has the required assumptions above.

[gigi-at-zymergen]

Thanks, I really appreciate the answers! You've given me a lot to think about in terms of secret injections and how we might be able to work with that. Trying to think through automating the creation and management of the secrets which would get injected to the pods.

We are a bit odd in our trust model. Our "edge" applications are behind our firewall and only used by users within our organization in our AD/LDAP store. We have a custom service sitting in front of AD which proxies LDAP authentication, but also handles authorization grants and issues JWT tokens. We have run into expiry and refresh/renewal issues with long running jobs, so that is a very valid concern. We do have some unfortunate system semantic assumptions about delegation and doing things on behalf of the edge user in many of our workflows. We can probably figure out a service-account/m2m solution. We probably do generally trust the task code, but do want visibility into who is making the original requests at the edge, but may be able to just pass along a user id while using workflow serviceAccounts stored in secrets.
We're currently doing a "poor-man's delegation" AKA impersonation. But ideal would probably be Oauth 2.0 token exchange. (influenced heavily by this article about delegation patterns https://www.scottbrady91.com/OAuth/Delegation-Patterns-for-OAuth-20 ). Still trying to think through a delegation pattern where the task code itself vs. the flyte API would be doing token exchange and making calls to other services on behalf of the workflow triggering user.

[kumare3]

Hi Gigi, let's sync up over VC? Can you please join slack?

[gigi-at-zymergen]

I did join Flyte Slack, and if you're willing, would love to chat further! I'll send you a DM in the Flyte workspace.