OPAL Client Redirected to GitLab Sign-In Page Despite Access Token in OPAL_DATA_CONFIG_SOURCES

[keyan4858]

I have set up GitLab on an EC2 instance and created a repository that includes policy.rego and data.json files. On another EC2 instance, I installed the OPAL server, OPAL client, and OPA using pip. I configured the OPAL server and client by providing the necessary environment variables within executable shell files. Specifically, for the OPAL server, I set the OPAL_DATA_CONFIG_SOURCES environment variable as follows:
export OPAL_DATA_CONFIG_SOURCES='{"config": {"entries": [{"url": "http://34.234.67.23/Naruto/OPA/raw/main/data.json", "topics": ["policy_data"], "config": {"headers": {"Authorization": "Bearer glpat-tx5AxRcPgT1VKt37HPrz"}}, "dst_path": "/v1/data/example", "save_method": "PUT"}]}}'

Here are my settings:
opal-server.sh file contains:
#!/bin/bash
export OPAL_POLICY_REPO_URL="http://3.129.51.36/Naruto/OPA.git"
export OPAL_POLICY_REPO_TOKEN="glpat-tx5AxRcPgT1VKt37HPrz"
export OPAL_POLICY_REPO_WEBHOOK_SECRET="k@Rthi143"
export OPAL_OPA_HEALTH_CHECK_POLICY_ENABLED="True"
export OPAL_POLICY_REPO_MAIN_BRANCH="main"
export OPAL_POLICY_REPO_POLLING_INTERVAL="60"
export OPAL_DATA_CONFIG_SOURCES='{"config": {"entries": [{"url": "http://3.129.51.36/Naruto/OPA/raw/main/data.json", "topics": ["policy_data"], "config": {"headers": {"Authorization": "Bearer glpat-tx5AxRcPgT1VKt37HPrz"}},"dst_path": "/v1/data/example","save_method": "PUT"}]}}'
opal-server run

opal-client.sh file contains:
#!/bin/bash
export OPAL_SERVER_URL="http://localhost:7002"
export OPA_SERVER_URL="http://localhost:8181"
opal-client run

Despite including the access token in the Authorization header, the OPAL client encounters an error where it is redirected to the GitLab sign-in page when attempting to fetch the data.json file. It appears that the URL provided in the OPAL_DATA_CONFIG_SOURCES variable leads to a login page instead of directly accessing the JSON file. I need assistance in resolving this issue so that the OPAL client can successfully fetch the data.json file from the GitLab repository without encountering a redirection to the sign-in page. Any advice on proper configuration or troubleshooting steps would be greatly appreciated.

[maya-barak]

Hi @keyan4858 !
Thank you for the information, I will check with the team and get back to you.

[maya-barak]

@keyan4858 Could you please share a docker-compose file with us that reproduces it ?
(you can send it on private if you prefer)

[keyan4858]

Hi @maya-barak am not using docker, I used python packages to install both opal-server and opal-client.

[orweis]

Hi @keyan4858 -

1. Usually when OPAL is accessing Git for policy, including for policy data in the form of a data.json file that is not done via the data-sources config but rather by setting up the repository as a policy source - please consider that alternative path.

2. Are you able to access http://3.129.51.36/Naruto/OPA/raw/main/data.json directly from the same machine as the OPAL-client , with that same secret - if not the problem is likely not OPAL related.

3. If 1,2 don't work for you - please include the logs for the OPAL client trying to fetch the file; so we can see the error.

[keyan4858]

Hi @orweis I have followed your suggestion and the first point is helped to solve the issue.

[keyan4858]

Hi, I would like to know if OPAL supports Gitlab Webhooks for automatic updates.if it does, could you please give step-by-step guidance on how to configure it?

[orweis]

Hi @keyan4858 , glad to hear.

Follow the guide here: https://opal.ac/tutorials/track_a_git_repo#option-2-using-a-webhook

You'd see specific configuration for Gitlab a little below