Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable OAuth2 authentication. #646

Open
wants to merge 131 commits into
base: master
Choose a base branch
from
Open

Conversation

ojecborec
Copy link

I've created the new branch to enable reviewing changes for #602.

Copy link

netlify bot commented Aug 16, 2024

Deploy Preview for opal-docs canceled.

Name Link
🔨 Latest commit 6ea4078
🔍 Latest deploy log https://app.netlify.com/sites/opal-docs/deploys/672a4226544adb00080d614c

@roekatz roekatz self-requested a review August 21, 2024 09:55
@roekatz
Copy link
Collaborator

roekatz commented Aug 21, 2024

Hi @ojecborec,

Thanks again for the help & contribution.

  1. Can you explain what is the use case / use cases that this PR aims to enable? Why have you found it needed? (Is it having opal-server authenticate opal-client using a third party identity provider?).
  2. Please fix the pre-commit checks and the failing tests.
  3. This new feature requires additional tests (probably for DataUpdater/PolicyUpdater).

@ojecborec
Copy link
Author

Following PR is enabling OAuth2 third party authentication for both server and client. Client would send access token generated by Client Credentials grant and server would validate this token by either calling introspect endpoint or reading JWT signature. I'll have a look at pre-commit checks and failing tests.

@ojecborec
Copy link
Author

If I understand correctly some pre-commit checks are failing on code like this

-        callbacks_router = init_callbacks_api(self.authenticator, self._callbacks_register)
+        callbacks_router = init_callbacks_api(
+            self.authenticator, self._callbacks_register
+        )

Why is callbacks_router = init_callbacks_api(self.authenticator, self._callbacks_register) being rejected? I see a lot of code like this all over the place.

@ojecborec
Copy link
Author

Can you help me understand how is this test failure related to code I'm submitting (I'm not a Python developer) ?

    @pytest.mark.asyncio
    async def test_data_updater_with_report_callback(server):
        ...
    
        try:
            ...
    
            proc2 = multiprocessing.Process(target=trigger_update_patch, daemon=True)
            proc2.start()
            ...
    
        # cleanup
        finally:
            await updater.stop()
            proc.terminate()
>           proc2.terminate()
E           UnboundLocalError: local variable 'proc2' referenced before assignment

@roekatz
Copy link
Collaborator

roekatz commented Aug 21, 2024

@ojecborec I'll help with Python you'll help with OAuth2 :) (As I'm not an expert, although willing to make myself more familiar of course).

I still don't understand what's the missing use case for OPAL (from user's high level perspective) - Do you want opal-client to authenticate to opal-server using a specific Google account? (OAuth2 is usually used for authenticating web applications on behalf of actual people, isn't it?).

Regarding pre-commit - it enforces the "Black" formatting. You can install pre-commit locally (brew install pre-commit if you're on mac) and run it within the repo with pre-commit run --all-files. It would fix the formatting for you (not only show errors). You can also install/enable Black extension in your IDE so files are formatted on save / on some key combo.

Regarding the tests - Yeah seems like the stack trace is misleading. I believe the test raised an exception before proc2 was defined, finally is always executed (exception or not) for clean up but then proc2 isn't defined. I guess the exception has to do with the changes in DataUpdater because it actually tests data updates. Maybe fixing the proc2 issue (define it at the function's head and check if it's not None before calling .terminate()) would reveal the actual exception.
You can run tests locally using:

pip install -r requirements.txt 
pytest

@roekatz
Copy link
Collaborator

roekatz commented Aug 22, 2024

@ojecborec Apologies - I see now similar failures on other PRs, so those probably don't have to do with you changes.
I'll take care of fixing them in master

roekatz and others added 18 commits August 23, 2024 16:39
docs(obtain-jwt-token.mdx): curl request data-raw should be valid json
…ng-opal-tests-in-github-actions

Fix data updater tests
…l-application-tests

Introduce docker & bash based application tests
Bumps [micromatch](https://github.com/micromatch/micromatch) from 4.0.7 to 4.0.8.
- [Release notes](https://github.com/micromatch/micromatch/releases)
- [Changelog](https://github.com/micromatch/micromatch/blob/master/CHANGELOG.md)
- [Commits](micromatch/micromatch@4.0.7...4.0.8)

---
updated-dependencies:
- dependency-name: micromatch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [axios](https://github.com/axios/axios) from 1.7.3 to 1.7.5.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](axios/axios@v1.7.3...v1.7.5)

---
updated-dependencies:
- dependency-name: axios
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [webpack](https://github.com/webpack/webpack) from 5.91.0 to 5.94.0.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](webpack/webpack@v5.91.0...v5.94.0)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
@roekatz
Copy link
Collaborator

roekatz commented Aug 29, 2024

@ojecborec - You can rebase onto master as tests should be fixed there now

asafc and others added 30 commits November 5, 2024 17:09
…unner-piping-logs-does-not-always-work

fix _pipe_log_stream not running consistently for monitored subprocess
# Conflicts:
#	packages/requires.txt

# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-client/opal_client/data/updater.py
#	packages/opal-client/opal_client/policy/updater.py
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/jwk.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/data/api.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-client/opal_client/data/updater.py
#	packages/opal-client/opal_client/policy/fetcher.py
#	packages/opal-client/opal_client/policy/updater.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-common/opal_common/fetcher/providers/http_fetch_provider.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/jwk.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/data/api.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-client/opal_client/data/updater.py
#	packages/opal-client/opal_client/policy/fetcher.py
#	packages/opal-client/opal_client/policy/updater.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-common/opal_common/fetcher/providers/http_fetch_provider.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/jwk.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/data/api.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-client/opal_client/data/updater.py
#	packages/opal-client/opal_client/policy/fetcher.py
#	packages/opal-client/opal_client/policy/updater.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-common/opal_common/fetcher/providers/http_fetch_provider.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/jwk.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/data/api.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-client/opal_client/data/updater.py
#	packages/opal-client/opal_client/policy/fetcher.py
#	packages/opal-client/opal_client/policy/updater.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-common/opal_common/fetcher/providers/http_fetch_provider.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/jwk.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/data/api.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-client/opal_client/data/updater.py
#	packages/opal-client/opal_client/policy/fetcher.py
#	packages/opal-client/opal_client/policy/updater.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-common/opal_common/fetcher/providers/http_fetch_provider.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/jwk.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/data/api.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-client/opal_client/data/updater.py
#	packages/opal-client/opal_client/policy/fetcher.py
#	packages/opal-client/opal_client/policy/updater.py
#	packages/opal-common/opal_common/authentication/authenticator.py
#	packages/opal-common/opal_common/authentication/oauth2.py
#	packages/opal-common/opal_common/config.py
#	packages/opal-common/opal_common/fetcher/providers/http_fetch_provider.py
#	packages/opal-server/opal_server/authentication/authenticator.py
#	packages/opal-server/opal_server/server.py
#	packages/requires.txt
# Conflicts:
#	packages/opal-client/opal_client/client.py
#	packages/opal-client/opal_client/data/updater.py
#	packages/opal-client/opal_client/policy/updater.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants