Confluent Kafka Policy Push -- Seeking integration advice

[stevejhall]

I have taken a brief look at the Kafka plugin approach, for OPA

Assuming we were interested in moving toward Confluent Kafka SAAS offering, where they don’t intend to support plugins, what is the advice around integrating for a policy push approach using Confluent Kafka REST APIs to enforce OPA policies?

Some background information...

The Metadata Server (MDS) is Confluent's recommended integration. The integration is largely via one call to a role-binding endpoint, where we combine one of Confluent's fixed roles, with an LDAP group or user, and a resource such as a Kafka topic.

see https://docs.confluent.io/platform/current/security/rbac/mds-api.html#rbac-rolebinding-crud

Here is an example:

POST /security/1.0/principals/{principal}/roles/{roleName}/bindings 
HTTP/1.1
Host: example.com
Content-Type: application/json

{
	"scope": {
		"clusterName": "us-east",
	},
	"resourcePatterns": [{
		"resourceType": "Topic",
		"name": "erp_us_bom",
		"patternType": "Literal"
	}]
}

Principal and role may look something like these examples:

POST /security/1.0/principals/User:bill/roles/DeveloperWrite
POST /security/1.0/principals/Group:erp_data_consumers/roles/DeveloperRead

As there are no roles to write, and as OPA does not appear to offer policy push or policy synch, the use case does not seem well suited to OPA.

However, those are the observations of a novice.

Before I say OPA is not a match I am interested in other's ideas around this. Those who have more experience and vision around OPA.

Is this well suited, or a bad idea?

Is anyone else working on this?

If I was to propose such an integration, would the community back it or reject it?

Thanks in advance for any input.

[anderseknert]

Hey @stevejhall 👋 :)

I think we've talked somewhere before about this, but good to see you again!

I could be reading those docs wrong, but it looks like all that their API provides is a way to do a role-binding between LDAP groups and some pre-defined roles/resources, correct?

as OPA does not appear to offer policy push or policy synch, the use case does not seem well suited to OPA

I'm not sure what you mean by policy push/sync in this context, but I see nothing in their API that looks like it would allow OPA to push policy, even if translated from Rego to something like JSON. Or would that kind of sync simply entail that the rolebindings fram LDAP groups -> Kafka roles was stored in OPA first and then forwarded to the metadata server? OPA isn't really meant to be the datasource of policy or policy data, but rather to consume and use data for policy decisions it makes itself. I don't think OPA would have much to add in this scenario, and limited options around authorizatios like the one described in the docs here are commonly what drives people to OPA in the first place :) I'd be happy to be proven wrong though, and there could definitely be more to it than what I can discern from their API docs.

Is anyone else working on this?

Not that I'm aware of.

If I was to propose such an integration, would the community back it or reject it?

We're happy to back anything OPA related that has a chance of being useful to others! Feel free to reach out if you need any advice on OPA here. I'm not sure if we have anyone with experience working with the commercial Confluent offerings before though.

[stevejhall]

Thank you, @anderseknert

I posted this here per a group member's request, in response the the same question I posted on Slack a few weeks ago.

I could be reading those docs wrong, but it looks like all that their API provides is a way to do a role-binding between LDAP groups and some pre-defined roles/resources, correct? [Steve Hall] yes, that is correct. RBAC is then natively enforced within Confluent Kafka via its integration with its Metadata Server, and LDAP.

[Steve Hall] I should have said push role bindings. However, I think you have clearly stated the gist. The goal is to define role bindings somewhere external to Kafka, and forward them into Confluent Kafka by a POST to the MDS REST API. Our architects are hoping to TAG Kafka Topics in our Metadata Catalog, where we categorize all our Data Lake and Data Warehouse assets, then use the tag to automate pushing role bindings into Confluent Kafka, where they would be natively enforced. The Confluent Kafka MDS, and the Metadata Catalog are independent systems with similar names, but different purposes.

I don't think OPA would have much to add in this scenario [Steve Hall] I agree, seems like a use case that OPA is not designed to support.

Thanks for your time here, it's appreciated.