Proposal for opa version 0.43.1

[mlajkim]

Hello,

Our team has been using opa and kube-mgmt for a long time with the following fixed versions:

• opa 0.43.0
• kube-mgmt 7.0.6

Since both versions have vulnerabilities, we want to update the package with no-found-vulnerabilities to meet our standards.
Yet it seems like there are some changes in:

• Rego grammar
• opa configurations
• etc that we are not aware of

between the current image 0.43.0 and the latest image 1.0.0.It is our fault that we haven't upgraded periodically, but yeah our team was a bit busy with other stuff.

Correct me if I am wrong, but our system indicates that every opa version below 0.64.0 has vulnerabilities.
So if we upgrade to an already-released version, it will most likely be 0.64.0.

Upgrading from 0.43.0 and 0.64.0 will not only take us some time to gradually upgrade
but also since we need to modify our Rego policies, there's a risk that these changes could unknowingly disrupt our production environments.

therefore, if possible, we would like to maintain our current policies while addressing the vulnerability issue with the following steps:

1. We create a new branch named 0.43.1-release based on 0.43.0.
2. We make a PR that fixes the vulnerabilities for 0.43.0-fix-vuln based on the branch 0.43.1-release.
3. Have the OPA team review/merge the PR into the 0.43.1-release branch.
4. Have the OPA team release 0.43.1 as a patch version.

This way, we can temporarily upgrade our internal OPA usage from 0.43.0 to 0.43.1 and later upgrade to the latest version 1.0.0 once we've completed some internal tasks we have in our organization.

Do you think this approach is feasible? Is there anything else I might be overlooking?

We would greatly appreciate your assistance with this matter.
Thank you in advance for considering our request.

Have a lovely day.

[charlieegan3]

Hi @mlajkim, thank you for starting the discussion. I’m sorry to hear about your situation and will do my best to assist.

First, in case you were unaware, there is a --v0-compatible mode for OPA 1.0. You can find details here. This is the recommended approach for those needing a quick upgrade without modifying policies.

That said, I encourage you and the team to consider this point:

Upgrading from 0.43.0 and 0.64.0 will not only take us some time to gradually upgrade.

With the tools available now, the process may be less arduous than you expect.

Using an OPA 1.0 binary locally, you can run:
• opa check --v0-v1 --strict to identify issues that are now compiler errors.
• opa fmt --write --v0-v1 to automatically upgrade Rego to the new syntax.
• regal lint to use the Regal linter for detecting additional bugs.

More details here: https://www.styra.com/blog/renovating-rego/

If you haven’t already, consider adding test coverage to confirm the behavior remains consistent before and after the process.

This effort is likely more valuable than delaying the upgrade. You’ll not only benefit from modernized Rego but also from the significantly faster and improved OPA, which has seen substantial progress over the past two years.

If you choose the 0.43.0-fix-vuln path, note that this will need to be done in a private fork, as it’s not something we can support in the public OPA repository.

[mlajkim]

@charlieegan3
Hi, thank you for the kind response!

Our team will begin the operation check for OPA's "v0 backwards compatibility" as you've shared starting on Jan 24 JST.
I will post some questions in this discussion if we have any during the operation check.

Thank you in advance.

[mlajkim]

Sometimes it does not restart:

[charlieegan3]

If data.system.authz.unverified_jwt[0].kid is missing, my guess is that the authz policy for the instance is not being loaded in the same way it was before. has that changed at all in the upgrade?

[anderseknert]

The system namespace is reserved by OPA, and while this predates OPA 1.0, there's now by default an extended type check based on the known input the system authz policies receive. So if you have a policy like that referencing input.foo or whatever, OPA knows that can never be defined and will fail. You can disable this with --skip-known-schema-check passed to opa run... but it would be better to fix those references.

[mlajkim]

(Still under investigation but the --skip-known-schema-check worked in our setup as well)

[mlajkim]

@anderseknert @charlieegan3
Hi,

We've concluded that the --skip-known-schema-check flag indeed resolves the problem. However, as you mentioned, we also believe it's essential to address the underlying issues, such as error handling.

The NullReferenceError occurs because of the following logic:

default unverified_jwt = [null, null]
unverified_jwt = io.jwt.decode(jwt)

This code relies on io.jwt.decode. Our Rego policy then attempts something like:

unverified_jwt[0].kid == json.unmarshal(jwks_cached).keys[_].kid

This operation causes the NullReferenceError if json.unmarshal(jwks_cached) is null. To fix this, we'll modify the code as follows:

unverified_jwt = decoded_jwt {
    decoded_jwt := io.jwt.decode(jwt)
} else = [{ "kid": null }, { "exp": null }]

This ensures that the keys kid or exp are always present in the JWT.

I'm still puzzled about why the restarts occur. Even when it "restarts," it loads afterward successfully (when it is still a initial booting stage). The results are inconsistent, making it difficult to pinpoint the exact cause of the flakiness.

We also sometimes get the following log, but again randomly:

/etc/opa/policy/authorization_policy.rego:151: rego_type_error: io.jwt.decode_verify: invalid argument(s)
	have: (string, null, ???)
	want: (jwt: string, constraints: object[string: any], output: array<boolean, object[any: any], object[any: any]>)

Is there any docs/tutorials that I should learn so that I can truly understand the issue I am facing? This week I am fully committed to address this issue.
Thanks in advance.

p.s)
I believe the feature is introduced in the following PR: open-policy-agent/opa#6141