Skip to content

Latest commit

 

History

History
343 lines (256 loc) · 9.68 KB

manage-tags-in-gravitino.md

File metadata and controls

343 lines (256 loc) · 9.68 KB
Raw
title slug date keyword license
Manage tags in Gravitino
/manage-tags-in-gravitino
2024-07-24
tag management, tag, tags, Gravitino
This software is licensed under the Apache License version 2.

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

Introduction

Starting from 0.6.0, Gravitino introduces a new tag system that allows you to manage tags for metadata objects. Tags are a way to categorize and organize metadata objects in Gravitino.

This document briefly introduces how to use tags in Gravitino by both Gravitino Java client and REST APIs. If you want to know more about the tag system in Gravitino, please refer to the Javadoc and REST API documentation.

Note that current tag system is a basic implementation, some advanced features will be added in the future versions.

:::info

  1. Metadata objects are objects that are managed in Gravitino, such as CATALOG, SCHEMA, TABLE, COLUMN, FILESET, TOPIC, COLUMN, etc. A metadata object is combined by a type and a comma-separated name. For example, a CATAGLOG object has a name "catalog1" with type "CATALOG", a SCHEMA object has a name "catalog1.schema1" with type "SCHEMA", a TABLE object has a name "catalog1.schema1.table1" with type "TABLE".
  2. Currently, only CATALOG, SCHEMA, TABLE, FILESET, TOPIC objects can be tagged, tagging on COLUMN will be supported in the future.
  3. Tags in Gravitino is inheritable, so listing tags of a metadata object will also list the tags of its parent metadata objects. For example, listing tags of a Table will also list the tags of its parent Schema and Catalog.
  4. Same tag can be associated to both parent and child metadata objects. When you list the associated tags of a child metadata object, this tag will be included twice in the result list with different inherited values. :::

Tag operations

Create new tags

The first step to manage tags is to create new tags. You can create a new tag by providing a tag name, optional comment and properties.

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" -d '{
  "name": "tag1",
  "comment": "This is a tag",
  "properties": {
    "key1": "value1",
    "key2": "value2"
  }
}' http://localhost:8090/api/metalakes/test/tags
GravitinoClient client = ...
Tag tag =
    client.createTag("tag1", "This is a tag", ImmutableMap.of("key1", "value1", "key2", "value2"));

List created tags

You can list all the created tag names as well as tag objects in a metalake in Gravitino.

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags?details=true
GravitinoClient client = ...
String[] tagNames = client.listTags();

Tag[] tags = client.listTagsInfo();

Get a tag by name

You can get a tag by its name.

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/tag1
GravitinoClient client = ...
Tag tag = client.getTag("tag1");

Update a tag

Gravitino allows you to update a tag by providing a new tag name, comment and properties.

curl -X PUT -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" -d '{
  "updates": [
    {
      "@type": "rename",
      "newName": "tag2"
    },
    {
      "@type": "updateComment",
      "newComment": "This is an updated tag"
    },
    {
      "@type": "setProperty",
      "property": "key3",
      "value": "value3"
    },
    {
      "@type": "removeProperty",
      "property": "key1"
    }
  ]
}' http://localhost:8090/api/metalakes/test/tags/tag1
GravitinoClient client = ...
Tag tag = client.alterTag(
    "tag1",
    TagChange.rename("tag2"),
    TagChange.updateComment("This is an updated tag"),
    TagChange.setProperty("key3", "value3"),
    TagChange.removeProperty("key1"));

Currently, Gravitino support the following tag changes:

Supported modification JSON Java
Rename a tag {"@type":"rename","newName":"tag_renamed"} TagChange.rename("tag_renamed")
Update a comment {"@type":"updateComment","newComment":"new_comment"} TagChange.updateComment("new_comment")
Set a tag property {"@type":"setProperty","property":"key1","value":"value1"} TagChange.setProperty("key1", "value1")
Remove a tag property {"@type":"removeProperty","property":"key1"} TagChange.removeProperty("key1")

Delete a tag

You can delete a tag by its name.

curl -X DELETE -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/tag2
GravitinoClient client = ...
client.deleteTag("tag2");

Tag associations

Gravitino allows you to associate and disassociate tags with metadata objects. Currently, only CATALOG, SCHEMA, TABLE, FILESET, TOPIC objects can be tagged.

Associate and disassociate tags with a metadata object

You can associate and disassociate tags with a metadata object by providing the object type, object name and tag names.

The request path for REST API is /api/metalakes/{metalake}/tags/{metadataObjectType}/{metadataObjectName}.

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" -d '{
  "tagsToAdd": ["tag1", "tag2"],
  "tagsToRemove": ["tag3"]
}' http://localhost:8090/api/metalakes/test/tags/catalog/catalog1

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" -d '{
  "tagsToAdd": ["tag1"]
}' http://localhost:8090/api/metalakes/test/tags/schema/catalog1.schema1
Catalog catalog1 = ...
catalog1.supportsTags().associateTags(
    new String[] {"tag1", "tag2"},
    new String[] {"tag3"});

Schema schema1 = ...
schema1.supportsTags().associateTags(new String[] {"tag1"}， null);

List associated tags for a metadata object

You can list all the tags associated with a metadata object. The tags in Gravitino are inheritable, so listing tags of a metadata object will also list the tags of its parent metadata objects.

The request path for REST API is /api/metalakes/{metalake}/tags/{metadataObjectType}/{metadataObjectName}.

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/catalog/catalog1

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/schema/catalog1.schema1

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/catalog/catalog1?details=true

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/schema/catalog1.schema1?details=true
Catalog catalog1 = ...
String[] tags = catalog1.supportsTags().listTags();
Tag[] tagsInfo = catalog1.supportsTags().listTagsInfo();

Schema schema1 = ...
String[] tags = schema1.supportsTags().listTags();
Tag[] tagsInfo = schema1.supportsTags().listTagsInfo();

Get an associated tag by name for a metadata object

You can get an associated tag by its name for a metadata object.

The request path for REST API is /api/metalakes/{metalake}/tags/{metadataObjectType}/{metadataObjectName}/{tagName}.

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/catalog/catalog1/tag1

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/schema/catalog1.schema1/tag1
Catalog catalog1 = ...
Tag tag = catalog1.supportsTags().getTag("tag1");

Schema schema1 = ...
Tag tag = schema1.supportsTags().getTag("tag1");

List metadata objects associated with a tag

You can list all the metadata objects associated with a tag.

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/tag1/objects
Tag tag = ...
MetadataObject[] objects = tag.associatedObjects().objects();
int count = tag.associatedObjects().count();