Clearer definition of NameIdentifier

[xunliu]

Gratiton uses NameIdentifier to describe all Graviton managed objects like Database, Datalake, Stream, Files, and more.

But NameIdentifier has a flexible namespace definition inside to support metalake, schema, multi-level namespace, and table names. Flexibility also confuses us when we use it.

NameIdentifier Struct

NameIdentifier {
  Namespace {
      String[] levels,
  } namespace,
  String name,
}

issues 1

NameIdentifier[] nameIdentifiers = metalake.listCatalogs(NameIdentifier.of(metalakeNameVar).namespace()); // throw exception in gravtion-0.1.0 version

NameIdentifier[] nameIdentifiers = metalake.listCatalogs(Namespace.of(metalakeNameVar)); // it ok

NameIdentifier Structure

Clearer definition 1

NameIdentifier {
  String metalake,
  String catalog,
  Namespace {
      String[] levels,
  } namespace,
  String name,
}

1. metalake: is required, It spilt each different user.
2. catalog: is required, It describes data source type and connect and more.
3. namespace: an option, that may exist or not exist in the different data sources.
4. name: is required, have different mean in the different data sources.

NameIdentifier definition in the different data source

Datasource Type	Metalake	Catalog	Namespace	Name	Desc
Database	metalake	catalog	schema(database name)		Database expression
Database	metalake	catalog	schema(database name)	table name	Table expresstion
Datalake	metalake	catalog	Datalake {namespaces}
Stream	metalake	catalog		topic
S3	metalake	catalog	dir1.dir2...		partition expression
S3	metalake	catalog	dir1.dir2...	file	storage file expression

Clearer definition 2

NameIdentifier {
  String metalake,
  String catalog,
  Namespace {
      String[] levels,
  } namespace,
}

1. metalake: is required, It spilt each different user.
2. catalog: is required, It describes data source type and connect and more.
3. namespace: an option, that may exist or not exist in the different data sources.

NameIdentifier definition in the different data source

Datasource Type	Metalake	Catalog	Namespace	Desc
Database	metalake	catalog	schema {database name}	Database expression
Database	metalake	catalog	schema{database name}.{table name}	Table expresstion
Datalake	metalake	catalog	Datalake {namespaces}
Stream	metalake	catalog	topic
S3	metalake	catalog	{dir1}.{dir2}...{dirZ}	partition expression
S3	metalake	catalog	{dir1}.{dir2}...{file}	storage file expression

Related issue

• [Bug report] Unable to get explicit catalog identifier in the NameIdentifier #268