Google Artifacts repository support

[InCogNiTo124]

Hey,

I'm trying to use hatch to upload packages to GAR.

GAR uses keyrings and google auth keyring. Twine seems to be using them correctly as twine upload works, but I can't set hatch to use them. Any ideas?

[InCogNiTo124]

Well it seems I have found a solution:

HATCH_INDEX_REPO=https://${LOCATION}-python.pkg.dev/${PROJECT}/${REGISTRY} \
HATCH_INDEX_USER=_json_key_base64 \
HATCH_INDEX_AUTH=< base64 encoded service account key > \
hatch publish

However this feels like a workaround. Any other suggestions for utilizing keyrings.google-artifactregistry-auth?

[ofek]

• https://hatch.pypa.io/latest/publish/#repository
• https://hatch.pypa.io/latest/plugins/publisher/package-index/#repositories

[InCogNiTo124]

• hatch.pypa.io/latest/publish/#repository
• hatch.pypa.io/latest/plugins/publisher/package-index/#repositories

Thank you for the reply.

I may be missing something, but I can't understand how these links you provided help with setting up the Google's keyring authentication.

[ofek]

Sorry I haven't had much time to look into this. What kind of solution or configuration are you expecting?

[owenhaynes]

Yeah looking for the same would like to use keyring keyrings.google-artifactregistry-auth

[owenhaynes]

Ok hatch does not use the user returned from the keyring when keyrings.google-artifactregistry-auth is used. A workaround is to pass the user it returns which is oauth2accesstoken.

so the hatch publish call would look something like

hatch publish -r https://europe-west2-python.pkg.dev/my-gcp-project/foo -u=oauth2accesstoken

[InCogNiTo124]

Sorry for the delay.

What kind of solution or configuration are you expecting?

I expected that the behaviour would be similar to twine. For twine, a simple twine upload ... works because it uses the info from keyrings and GooglePythonAuth. There is no need for any additional config. We use https://github.com/google-github-actions/auth in our workflow and this actually suffices. Really practical.

as per @owenhaynes hatch doesn't seem to be using that info from keyrings so it requires additional config. Although there are two workarounds in this very thread, user experience would be greatly increased if hatch publish ... did all the work 😄

[InCogNiTo124]

Ok hatch does not use the user returned from the keyring when keyrings.google-artifactregistry-auth is used. A workaround is to pass the user it returns which is oauth2accesstoken.

so the hatch publish call would look something like

hatch publish -r https://europe-west2-python.pkg.dev/my-gcp-project/foo -u=oauth2accesstoken

I like this more as I don't have to manually base64 encode the SA

[owenhaynes]

@InCogNiTo124 do you know how to get hatch to pull dependencies from private indexes using the keyring or is this a missing feature?

[InCogNiTo124]

@owenhaynes haven't yet had the chance to test it out

[owenhaynes]

Looks like you need to have the env PIP_NO_INPUT=true set for it to use the keyring

[ofek]

Hello! I would like to better support this use case. Can one of you write down an enumeration of the current issues and a potential remedy?

[InCogNiTo124]

Context

We use GAR to publish private Python packages.
For authenticating with the GAR, Google uses keyrings, and more specifically keyrings.google-artifactregistry-auth package.

The CI does it for us. This is an example of a typical GitHub CI:

twine CI example

jobs:
  build-and-publish:
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
      - uses: google-github-actions/auth@v1
        with:
          credentials_json: <some json key>

      - ... other steps ...
      - run: python3 -m build
      - run: python3 -m pip install keyring keyrings.google-artifactregistry-auth
      - run: python3 -m twine upload --repository-url https://europe-python.pkg.dev/my-gcp-project/foo/ dist/*

Twine uses GooglePythonAuth keyring, which uses GOOGLE_DEFAULT_CREDENTIALS, which in turn is set by the auth step.

Problem statement

Simply switching python3 -m twine upload ... for hatch publish ... does not work. There are two workarounds in this thread, one with setting the appropriate environment variables, and the other with setting the auth user as the oauth response.

Possible explanations

It seems that hatch doesn't use all the data from the keyring as twine does. For example:

https://github.com/pypa/twine/blob/9ac14fcb68b3ff06eb2c0056a4006c2ea6a1aa10/twine/auth.py#L55,L67

vs

https://github.com/pypa/hatch/blob/2f685c7de285eed72b0bb0f4b01ef251e9ab033d/src/hatch/publish/index.py#L92,L97

Possible solutions

I guess hatch should introduce keyring probing for usernames as well, not just for passwords. I don't know myself how would I incorporate this change in the codebase.

[InCogNiTo124]

@ofek any update on this?

[ischaojie]

I am trying to solve this problem~

[Darsstar]

Looks like you need to have the env PIP_NO_INPUT=true set for it to use the keyring

I contributed the --keyring-provider flag to Pip 23.1, which should help a bit.
https://pip.pypa.io/en/stable/topics/authentication/#keyring-support

[ofek]

Resolved (I think) by #836