Check whether the opal-client has latest policy

[hongbo-miao]

Hi team, I am trying to know whether the opal-client has latest policy.

• One method in my mind is checking opal-client logs and get the last time when it pulled the policy based on the logs. For this method, we need write some our own codes to parse the log.
• Another method in my mind is letting opal-client expose one API for when did it pull the policy last time. Does it make sense to expose one?

Thanks!

[asafc]

Hi @hongbo-miao, welcome back from vacaction :)

You can either use the update callbacks feature or the healthcheck policy feature to achieve this.
Go over this document which explains about both of these features.

In essence:

The healthcheck policy is special OPA policy that (if activated) is loaded into OPA as the system.opal rego package. This special policy can be used to make sure that OPA is ready to accept authorization queries, and than its state is not out of sync due to failed data updates.

The update callback feature will trigger a callback (HTTP call to a configurable url) after every successful data update or policy update. It allows you to track which updates completed successfully and were correctly saved to OPA cache.

Example configuration

We have an example docker compose configuration with healthcheck policy and callbacks already configured.

Pointers for the example configuration:

callbacks

If you replace the URL on line 55 of the config - you can hit your own server with update callbacks.
If you leave line 55 unchanged - OPAL server will be called instead and you will see update "confirmations" on OPAL server logs.

healthcheck policy

Healthcheck policy is also configured, check out the urls in lines 59-60 of the example config (OPA api urls you can hit to check the policy).

[hongbo-miao]

Hi @asafc thanks! You are always ahead.

A following question, in the document, it says

healthy, checks that:

• OPA is ready (as defined above)
• and latest policy update bundle synced correctly
• and last published data update was fetched and synced correctly

Regarding "latest policy update bundle synced correctly", how does "latest" define here like policy that got checked in 10 seconds? Thanks!

[asafc]

it's a very simple check.

Regarding policy, we require that the last try to push a bundle to OPA cache succeeded:

1. opal server will continuously notify opal client on the latest policy bundle hash
2. opal client will ask for the latest bundle hash upon being notified
3. if for some reason the bundle is not synced correctly - the "ready" state will still be true, but the "healthy" state will be false.
4. the next time opal server will publish a bundle, opal client will fetch the bundle again and try to apply it to OPA state.
5. if the policy bundle is synced successfully to OPA cache, "healthy" will be back to true.

Bundle sync issues are not very common if your bundles contain valid rego.
OPAL currently cannot deal with invalid rego if you push it by mistake to your policy repo.

We did not observe any policy syncing issues with our own internal use of OPAL.

Data updates follow similar pattern - we require that the last data update attempt was successful.
Data updates can fail more often - since the fetcher will call arbitrary urls that you provide (3rd party services, your own apis).

We internally use update callbacks to make sure the data update succeeded, an attempt to republish if no callback is received.

[asafc]

also cc @orweis as i am leaving for short PTO until Sunday! 😎