Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Meta-service cluster is gated by Enterprise Edition(Do NOT merge) #16231

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

drmingdrmer
Copy link
Member

@drmingdrmer drmingdrmer commented Aug 12, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

feat: Meta-service cluster is gated by Enterprise Edition

By default meta-service disallows clustering.
Meta-service cluster is only enabled when raft-config
databend_enterprise_license is set and is valid.
No feature in the jwt claim is examined.

The EE gate check when a meta node initiate raft-protocol network
instance. Thus without a valid EE token, all raft-protocol are disabled,
including RequestVote, AppendEntries, InstallSnapshot and internal
request forward.

If EE token is not set, an error will be outputed to log file.

  • New config databend_enterprise_license:

    [raft_config]
    databend_enterprise_license = "<token>"
    

    This token is same as the one used by databend-query.

  • When testing, a temp key pair and jwt claim is created to pass
    integration tests, this is enabled by fake_ee_license config entry.

  • Other changes: Add DisplaySlice and DisplayUnixTimeStampExt to
    display slice of Display instance and unix timestamp.

feat: Add API to update license to meta-service in flight

Update EE license token with the following API:

curl -qs '127.0.0.1:28101/v1/ctrl/update_license?license=<license_token>'

On success, i.e., the token is valid and not expired, meta-service
respond with the 200 OK with token info, such as:

{"Success":"JWTClaims{issuer: databend, issued_at: 2024-05-13T05:57:24.000000Z+0000, expires_at: 2025-05-13T05:57:24.000000Z+0000, custom: LicenseInfo{ type: enterprise, org: databend-interval-test, tenants: None, features: [Unlimited] }}"}

If the token is invalid, or expired, it responds with 400 Bad Request
and a reason that cause the failure, such as:

Invalid license: JWT compact encoding error

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Other

Related Issues


This change is Reviewable

@drmingdrmer drmingdrmer changed the title change: Meta-service cluster is gated by Enterprise Edition feat: Meta-service cluster is gated by Enterprise Edition Aug 12, 2024
@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Aug 12, 2024
@drmingdrmer drmingdrmer force-pushed the 109-meta-service-ee branch 2 times, most recently from 9a49614 to 5618fb1 Compare August 12, 2024 06:01
@drmingdrmer drmingdrmer marked this pull request as ready for review August 13, 2024 03:25
@drmingdrmer drmingdrmer changed the title feat: Meta-service cluster is gated by Enterprise Edition feat: Meta-service cluster is gated by Enterprise Edition(Do NOT merge) Aug 13, 2024
}

impl MetaServiceEnterpriseGate {
const LICENSE_PUBLIC_KEY: &'static str = r#"-----BEGIN PUBLIC KEY-----
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why use a hard code public key?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This way only the the license token published by us with our private key will be correctly decoded.
If to let user config public key in config, user will be able to use a self signed token with its private key and let databend-meta decoded it with the public key it provided.

@ZhiHanZ
Copy link
Collaborator

ZhiHanZ commented Aug 13, 2024

I think we should provide a robust way to let user config their license on flight instead of update license on environment and restart meta server.

@drmingdrmer drmingdrmer force-pushed the 109-meta-service-ee branch 4 times, most recently from 1c88099 to d3ab07e Compare August 14, 2024 12:47
@drmingdrmer
Copy link
Member Author

The API to update license in flight is added, Please review again guys

Copy link
Collaborator

@ZhiHanZ ZhiHanZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, some ideas:
should we update meta server license when query cluster trigger
SET GLOBAL LICENSE command? it could simplify the meta service license setup, and support to view license from meta service by using license info system call command.
Ref:
https://docs.databend.com/guides/overview/editions/dee/license#verifying-a-license

@drmingdrmer
Copy link
Member Author

LGTM, some ideas: should we update meta server license when query cluster trigger SET GLOBAL LICENSE command? it could simplify the meta service license setup, and support to view license from meta service by using license info system call command. Ref: https://docs.databend.com/guides/overview/editions/dee/license#verifying-a-license

If databend-query accepted an invalid license and forwarded it to meta-services, then meta-service cluster may run into problem because it can not form a cluster anymore. And then any further setting can not be saved by databend-query.

Therefore it is a little bit dangerous, but it would help in most cases.

@drmingdrmer drmingdrmer force-pushed the 109-meta-service-ee branch 2 times, most recently from 3efa7e5 to 2350e06 Compare August 20, 2024 05:55
By default meta-service disallows clustering.
Meta-service cluster is only enabled when raft-config
`databend_enterprise_license` is set and is valid.
No feature in the jwt claim is examined.

The EE gate check when a meta node initiate raft-protocol network
instance. Thus without a valid EE token, all raft-protocol are disabled,
including `RequestVote`, `AppendEntries`, `InstallSnapshot` and internal
request forward.

If EE token is not set, an error will be outputed to log file.

- New config `databend_enterprise_license`:
  ```
  [raft_config]
  databend_enterprise_license = "<token>"
  ```

  This token is same as the one used by databend-query.

- When testing, a temp key pair and jwt claim is created to pass
  integration tests, this is enabled by `fake_ee_license` config entry.

- Other changes: Add `DisplaySlice` and `DisplayUnixTimeStampExt` to
  display slice of `Display` instance and unix timestamp.
Update EE license token with the following API:

```
curl -qs '127.0.0.1:28101/v1/ctrl/update_license?license=<license_token>'
```

On success, i.e., the token is valid and not expired, meta-service
respond with the 200 OK with token info, such as:

```
{"Success":"JWTClaims{issuer: databend, issued_at: 2024-05-13T05:57:24.000000Z+0000, expires_at: 2025-05-13T05:57:24.000000Z+0000, custom: LicenseInfo{ type: enterprise, org: databend-interval-test, tenants: None, features: [Unlimited] }}"}
```

If the token is invalid, or expired, it responds with 400 Bad Request
and a reason that cause the failure, such as:

```
Invalid license: JWT compact encoding error
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants