Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MINOR] docs: correct credential vending document #6248

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

[MINOR] docs: correct credential vending document #6248

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7e5a911
fix credential vending document
FANNG1 Jan 15, 2025
8d4ecf8
fix credential vending document
FANNG1 Jan 15, 2025
12c2fe5
fix credential vending document
FANNG1 Jan 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/hadoop-catalog.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ Refer to [Schema operation](./manage-fileset-metadata-using-gravitino.md#schema-
| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication for the fileset. | The parent(schema) value | No | 0.6.0-incubating |
| `credential-providers` | The credential provider types, separated by comma. | (none) | No | 0.8.0-incubating |

Credential providers can be specified in several places, as listed below. Gravitino checks the `credential-provider` setting in the following order of precedence:
Credential providers can be specified in several places, as listed below. Gravitino checks the `credential-providers` setting in the following order of precedence:

1. Fileset properties
2. Schema properties
Expand Down
20 changes: 11 additions & 9 deletions docs/security/credential-vending.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ An S3 token is a token credential with scoped privileges, by leveraging STS [Ass
| `s3-access-key-id` | `gravitino.iceberg-rest.s3-access-key-id` | The static access key ID used to access S3 data. | (none) | Yes | 0.6.0-incubating |
| `s3-secret-access-key` | `gravitino.iceberg-rest.s3-secret-access-key` | The static secret access key used to access S3 data. | (none) | Yes | 0.6.0-incubating |
| `s3-role-arn` | `gravitino.iceberg-rest.s3-role-arn` | The ARN of the role to access the S3 data. | (none) | Yes | 0.7.0-incubating |
| `s3-region` | `gravitino.iceberg-rest.s3-region` | The region of the S3 service, like `us-west-2`. | (none) | No | 0.6.0-incubating |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the region required to provide credential service? Can it be replaced with the endpoint?

Copy link
Contributor Author

@FANNG1 FANNG1 Jan 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

region is an optional properties to get credential, there is another configuration for STS endpoint

| `s3-external-id` | `gravitino.iceberg-rest.s3-external-id` | The S3 external id to generate token. | (none) | No | 0.7.0-incubating |
| `s3-token-expire-in-secs` | `gravitino.iceberg-rest.s3-token-expire-in-secs` | The S3 session token expire time in secs, it couldn't exceed the max session time of the assumed role. | 3600 | No | 0.7.0-incubating |
| `s3-token-service-endpoint` | `gravitino.iceberg-rest.s3-token-service-endpoint` | An alternative endpoint of the S3 token service, This could be used with s3-compatible object storage service like MINIO that has a different STS endpoint. | (none) | No | 0.8.0-incubating |
Expand All @@ -73,14 +74,15 @@ A credential with static OSS access key id and secret access key.

An OSS token is a token credential with scoped privileges, by leveraging STS [Assume Role](https://www.alibabacloud.com/help/en/oss/developer-reference/use-temporary-access-credentials-provided-by-sts-to-access-oss). To use an OSS token credential, you should create a role and grant it proper privileges.

| Gravitino server catalog properties | Gravitino Iceberg REST server configurations | Description | Default value | Required | Since Version |
|-------------------------------------|---------------------------------------------------|-------------------------------------------------------------------------------|---------------|----------|------------------|
| `credential-providers` | `gravitino.iceberg-rest.credential-providers` | `oss-token` for s3 token credential. | (none) | Yes | 0.8.0-incubating |
| `oss-access-key-id` | `gravitino.iceberg-rest.oss-access-key-id` | The static access key ID used to access OSS data. | (none) | Yes | 0.7.0-incubating |
| `oss-secret-access-key` | `gravitino.iceberg-rest.oss-secret-access-key` | The static secret access key used to access OSS data. | (none) | Yes | 0.7.0-incubating |
| `oss-role-arn` | `gravitino.iceberg-rest.oss-role-arn` | The ARN of the role to access the OSS data. | (none) | Yes | 0.8.0-incubating |
| `oss-external-id` | `gravitino.iceberg-rest.oss-external-id` | The OSS external id to generate token. | (none) | No | 0.8.0-incubating |
| `oss-token-expire-in-secs` | `gravitino.iceberg-rest.oss-token-expire-in-secs` | The OSS security token expire time in secs. | 3600 | No | 0.8.0-incubating |
| Gravitino server catalog properties | Gravitino Iceberg REST server configurations | Description | Default value | Required | Since Version |
|-------------------------------------|---------------------------------------------------|--------------------------------------------------------------------------------------------------------------|---------------|----------|------------------|
| `credential-providers` | `gravitino.iceberg-rest.credential-providers` | `oss-token` for s3 token credential. | (none) | Yes | 0.8.0-incubating |
| `oss-access-key-id` | `gravitino.iceberg-rest.oss-access-key-id` | The static access key ID used to access OSS data. | (none) | Yes | 0.7.0-incubating |
| `oss-secret-access-key` | `gravitino.iceberg-rest.oss-secret-access-key` | The static secret access key used to access OSS data. | (none) | Yes | 0.7.0-incubating |
| `oss-role-arn` | `gravitino.iceberg-rest.oss-role-arn` | The ARN of the role to access the OSS data. | (none) | Yes | 0.8.0-incubating |
| `oss-region` | `gravitino.iceberg-rest.oss-region` | The region of the OSS service, like `oss-cn-hangzhou`, only used when `credential-providers` is `oss-token`. | (none) | No | 0.8.0-incubating |
| `oss-external-id` | `gravitino.iceberg-rest.oss-external-id` | The OSS external id to generate token. | (none) | No | 0.8.0-incubating |
| `oss-token-expire-in-secs` | `gravitino.iceberg-rest.oss-token-expire-in-secs` | The OSS security token expire time in secs. | 3600 | No | 0.8.0-incubating |

### ADLS credentials

Expand Down Expand Up @@ -161,7 +163,7 @@ The classpath of the server:

Suppose the Iceberg table data is stored in S3, follow the steps below:

1. Download the [Gravitino AWS bundle jar](https://mvnrepository.com/artifact/org.apache.gravitino/gravitino-aws-bundle), and place it to the classpath of Iceberg REST server.
1. Download the [Gravitino AWS bundle jar without hadoop packages](https://mvnrepository.com/artifact/org.apache.gravitino/gravitino-aws), and place it to the classpath of Iceberg REST server.

2. Add s3 token credential configurations.

Expand Down
Loading