-
Notifications
You must be signed in to change notification settings - Fork 608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add support for externalized consumers #7657
base: main
Are you sure you want to change the base?
Conversation
✅ Deploy Preview for kongdocs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
- geo: null | ||
id: null |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
geo
and id
should be optional (default null
) imo
|
||
-- Dummy/Link that points to Konnect Docs when ready. | ||
|
||
With the `pool_id` you obtained from the previous step, you can configure the key-auth plugin to validate API keys against the Identity Service. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jschmid1 where does the pool_id
come from? Konnect?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to change all references of pools to realms, and update the field terminology
@@ -0,0 +1,71 @@ | |||
--- | |||
nav_title: | |||
title: How to configure pools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change to Realms
--- | ||
|
||
|
||
With `pool_id` you can configure the key-auth plugin to validate API keys against the Identity Service. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change to "realm_id"
|
||
With `pool_id` you can configure the key-auth plugin to validate API keys against the Identity Service. | ||
|
||
### Configuring Multiple Pools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
change to "Realms"
|
||
### Configuring Multiple Pools | ||
|
||
In the key-auth plugin configuration, add the `pools` option as shown below: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
change to "realms"
In the key-auth plugin configuration, add the `pools` option as shown below: | ||
|
||
```yaml | ||
pools: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
change to "identity_realms"
type: remote | ||
``` | ||
|
||
In this configuration, the dataplane will initially check the local pool (LMDB) before querying the remote Identity Service. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cp-scoped consumers first
|
||
In this configuration, the dataplane will initially check the local pool (LMDB) before querying the remote Identity Service. | ||
|
||
If a matching key is found in any of these pools, the request will be authenticated. If the key is not found in any of the configured pools, the request will be blocked. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"realms"
|
||
If a matching key is found in any of these pools, the request will be authenticated. If the key is not found in any of the configured pools, the request will be blocked. | ||
|
||
### Configuring Single Pools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Single Realm
|
||
### Configuring Single Pools | ||
|
||
It is also possible to configure only a single pool, either local or remote. However, only one of each type can be configured. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
change pool to realm
change "either local or remote" to "consumers can be scoped to the geo or cp".
|
||
It is also possible to configure only a single pool, either local or remote. However, only one of each type can be configured. | ||
|
||
To configure only a remote pool: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
change "remote pool" to "realm"
Testing edit function
Updated the terminology used for realms and the yaml file examples
…in to add realms section Signed-off-by: Diana <[email protected]>
@brentos Hi! I just wanted to let you know I'll be working on external consumers and I just added some changes to the PR, including a new Konnect page that explains what they are and how to configure them, as well as a revised section to key auth. I had a few questions while I was working on it:
|
Signed-off-by: Diana <[email protected]>
Signed-off-by: Diana <[email protected]>
Broken sales link was a false failure, I manually approved link validation for now |
Signed-off-by: Diana <[email protected]>
@smritikjaggi and @brentos The externalized consumers pages are ready for review if you have the chance to look them over: |
app/konnect/centralized-consumers.md
Outdated
|
||
The following diagram shows how you can configure centralized Consumers in realms as well as scope Consumers to a Control Plane: | ||
|
||
<!--vale off --> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really understand what this diagram is showing me.
For example:
Consumer: Ira
Consumer Groups: Gold for appointments
Does this consumer only have access to the appointments control plane? Or does it have access to both control planes, but since the consumer group Gold only exists on appointments
, it can only access appointments? Or something else?
Are control planes located inside realms, or alongside them?
app/konnect/centralized-consumers.md
Outdated
* The Consumer identity can be setup centrally instead of being defined across multiple control planes. | ||
* Share Consumers across multiple control planes. Users don't need to replicate changes to Consumer identity in multiple control planes and Consumer configuration doesn't conflict. | ||
* Reduces configuration sync issues between the control plane and the dataplane. Centralized Consumers aren't part of the configuration that is pushed down from the control plane to the dataplane, so it reduces config size and latency. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add the required Gateway version here as a note, somewhere.
* **Realm is listed first:** The dataplane will first reach out to the realm. If the API key is not found in the realm, the dataplane will look for the API key in the control plane config. | ||
* **Control Plane scope listed first:** The dataplane will initially check the control plane configuration (LMDB) for the API key before looking up the API Key in the realm. | ||
* **Realm isn't associated with a control plane:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the dataplane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked. | ||
* **Realm is only associated with a Control Plane:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the dataplane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* **Realm is listed first:** The dataplane will first reach out to the realm. If the API key is not found in the realm, the dataplane will look for the API key in the control plane config. | |
* **Control Plane scope listed first:** The dataplane will initially check the control plane configuration (LMDB) for the API key before looking up the API Key in the realm. | |
* **Realm isn't associated with a control plane:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the dataplane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked. | |
* **Realm is only associated with a Control Plane:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the dataplane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked. | |
* **Realm is listed first:** The data plane will first reach out to the realm. If the API key is not found in the realm, the data plane will look for the API key in the control plane config. | |
* **Control Plane scope listed first:** The data plane will initially check the control plane configuration (LMDB) for the API key before looking up the API Key in the realm. | |
* **Realm isn't associated with a control plane:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the data plane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked. | |
* **Realm is only associated with a Control Plane:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the data plane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked. |
Should this info actually be on the Konnect page instead, except genericized to "credentials" instead of "API key"?
app/konnect/centralized-consumers.md
Outdated
<!--vale off --> | ||
{% mermaid %} | ||
flowchart TD | ||
A(Consumer: Cruz<br>Consumer Groups: Gold for appointments, Silver for billing) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Each consumer can be in multiple consumer groups, but you can't scope the consumer group to a service.
If billing
respected the Gold
group, then Cruz would be Gold for billing too.
app/konnect/centralized-consumers.md
Outdated
* {region}: Region for your {{site.konnect_short_name}} instance. | ||
* $KONNECT_TOKEN: Replace with your {{site.konnect_short_name}} personal access token. | ||
* $CONTROL_PLANE_UUID: (Optional) Replace with your Control Plane UUID. | ||
* `ttl` and `negative_ttl`: (Optional) Time-to-live in minutes of the consumer or negative consumer for this realm in the {{site.base_gateway}} cache. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's a negative consumer
? I don't think that this is what negative_ttl
means. @zekth can you help us out here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there's no concept of negative_consumer
. negative_ttl
represents the TTL of a bad login cache entry. Meaning api key foobar
is sent to the gateway; gateway reaches the service to authenticate it; if it fails the api key foobar
will be labelled as non working in gateway cache to the time of the TTL. During that time it won't try to reache the authentication server.
app/konnect/centralized-consumers.md
Outdated
* $KONNECT_TOKEN: Replace with your {{site.konnect_short_name}} personal access token. | ||
* {realmId}: The ID of the realm you created previously. | ||
* $CONSUMER_NAME: Replace with the name of the consumer. | ||
* $CONSUMER_GROUP: (Optional) Replace with the name of the consumer groups you want to associate with the consumer. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I specify consumer_groups
on a consumer does that replace consumer_groups
from the realm, or are they added together?
If I specify consumer_groups: []
does that mean no consumer groups, or use the groups from the realm?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is additive here, I'll add that info to the doc.
* **Realm isn't associated with a control plane:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the dataplane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked. | ||
* **Realm is only associated with a Control Plane:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the dataplane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* **Realm isn't associated with a control plane:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the dataplane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked. | |
* **Realm is only associated with a Control Plane:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the dataplane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked. | |
* **Realm only:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the dataplane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked. | |
* **Control Plane only:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the dataplane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked. |
@@ -287,19 +295,55 @@ Response: | |||
} | |||
``` | |||
|
|||
{% if_version gte: 3.10.x %} | |||
|
|||
## Configure realms for centralized Consumers in {{site.konnect_short_name}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be an example rather than on the overview?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, I'll convert it to an example.
Co-authored-by: lena-larionova <[email protected]>
Signed-off-by: Diana <[email protected]>
Signed-off-by: Diana <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The content looks good Diana! Can we talk through the naming and placement of Consumers please? This doc outlines that we decided to go with Consumers as the official naming of this initiative: https://docs.google.com/document/d/1k-d9FqAk4Jqo-e9-19IekqKQlNWfaFdgBNBuJGBw80k/edit?tab=t.0#heading=h.tn4g33ltt5z7
app/_data/docs_nav_konnect.yml
Outdated
@@ -15,6 +15,8 @@ | |||
url: /private-connections/aws-privatelink/ | |||
- text: Geographic Regions | |||
url: /geo/ | |||
- text: Regional Consumers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are going with the naming of "Consumers" as we are not introducing a new entity, but are expanding on the existing Consumer entity such that they can be defined in a central location per region and referenced across multiple control planes. Doc: https://docs.google.com/document/d/1k-d9FqAk4Jqo-e9-19IekqKQlNWfaFdgBNBuJGBw80k/edit?tab=t.0#heading=h.tn4g33ltt5z7
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good! I'll revise this to call out that this is a new type of consumer management.
title: Configure realms for regional Consumers in {{site.konnect_short_name}} | ||
--- | ||
|
||
Starting in {{site.base_gateway]] 3.10.x, credentials that are stored centrally in {{site.konnect_short_name}} that are shared across multiple {{site.base_gateway}} can be validated by configuring `identity_realms` field in the Key Auth plugin. You can use these realms when you create [regional Consumers](/konnect/regional-consumers/). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to add some language that says "today a DP can only reach out to realms in the same region as they are deployed"?
app/konnect/regional-consumers.md
Outdated
@@ -0,0 +1,78 @@ | |||
--- | |||
title: Regional Consumers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Our official name for this initiative is "Consumers". What is the best way for us to reflect that this initiative is an expansion of the existing consumer entity such that consumers can be stored in a centralized space per region?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my revision, I'll call them "centralized consumer management" or "centrally managed consumers" if I need to call out the difference between them and "original" consumers. Otherwise, I'll use "consumers" when the context makes it clear we're talking about the central ones.
app/konnect/regional-consumers.md
Outdated
* `$CONSUMER_GROUP`: (Optional) Replace with the name of the consumer groups you want to associate with the consumer. Consumer groups set here are additive. This means that if you configure the realm with consumer groups A and B, and then configure the regional consumer with consumer group C, the authenticated consumer will be assigned to consumer groups A, B, and C. | ||
1. Consumers require authentication. Configure authentication using the [Key Auth plugin](/hub/kong-inc/key-auth/how-to/). | ||
|
||
The order in which you configure the `identity_realms` dictates the priority in which the data plane attempts to authenticate the provided API keys: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can also list out that the default is scope:cp
for Key Auth plugin.
Signed-off-by: Diana <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Left some comments
Save the ID of the realm. | ||
|
||
Be sure to replace the following with your own values: | ||
* `{region}`: Region for your {{site.konnect_short_name}} instance. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Today DPs can only reach out to the realms defined in the same region as the DP. Should we add this info somewhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep, I can add that somewhere!
{:.important} | ||
> Centralized consumer management is only available for {{site.base_gateway}} 3.10 data planes. Make sure your control plane uses 3.10 data planes at minimum when configuring these consumers. | ||
|
||
1. Use the `/realms` endpoint to create a realm and optionally associate it with allowed control planes and time-to-live values: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We define time-to-live further down the document. Should we define it here as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll leave this one as-is since we typically follow this format in docs where we explain it later in the bullets and it could bloat the step instruction if I put it here.
|
||
{:.note} | ||
> **Note:** If you are using KIC to manage your data plane nodes in {{site.konnect_short_name}}, ensure that you configure the `telemetry_endpoint` in the data plane. You can find the `telemetry_endpoint` in the {{site.konnect_short_name}} UI in [Gateway Manager](https://cloud.konghq.com/gateway-manager/) in the data plane node instructions. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to add a section around changing the scope of consumers from cp to realm? The user will need to do the following call:
POST https://us.api.konghq.com/identity/pools/%7BpoolID%7D/consumers)/Alice-123/keys
{
"type": "legacy" // Migrating Alice's existing keys
"secret": "Alice-api-key"
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since that's optional, I'd rather link them to the API spec so they can see what else they can do. Do you have a predicted link to the public API spec?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am adding a new step to create new keys for the consumer, since I think that was missing.
Co-authored-by: smritikjaggi <[email protected]>
Signed-off-by: Diana <[email protected]>
Description
Testing instructions
Preview link:
Checklist