Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#5057] Added first part of CLI code #5058

Merged
merged 51 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
b5e08c0
Added firest part of CLI code.
justinmclean Oct 2, 2024
655d133
update CLI option with output
justinmclean Oct 2, 2024
4a29c8b
Add missing license header
justinmclean Oct 2, 2024
1b12eff
Add default
justinmclean Oct 2, 2024
75de71e
Spotless on Java 11
justinmclean Oct 2, 2024
5172afa
Ignore warnings in test with Java 11
justinmclean Oct 2, 2024
0dc8856
Disable two tests on JAVA 17
justinmclean Oct 2, 2024
24dee9e
disabling two tests for now
justinmclean Oct 2, 2024
0ddbb7d
fix disabled tests
justinmclean Oct 2, 2024
4b95dd3
Still having issues on Java 17
justinmclean Oct 2, 2024
f76fa16
remove two tests as Java 17 is still giving issues
justinmclean Oct 2, 2024
8cbc605
No need to copy CLI for IT tests
justinmclean Oct 2, 2024
fa2cdaf
remove unneeded call
justinmclean Oct 9, 2024
68f7fa2
change version command into client and server version commands
justinmclean Oct 11, 2024
e2ebb7a
list entities and commands
justinmclean Oct 11, 2024
0e82c8f
Cache the metalake environment variable
justinmclean Oct 11, 2024
7b97750
use libs
justinmclean Oct 11, 2024
a562ed7
Merge branch 'main' into CLI
justinmclean Oct 11, 2024
d678b99
Trino Connector uses old deprecated CLI
justinmclean Oct 12, 2024
10af3d2
remove duplication and clarify
justinmclean Oct 16, 2024
f955db2
use client shadow jar
justinmclean Oct 16, 2024
80aef91
wrong file
justinmclean Oct 16, 2024
6d897c0
new -> 19
justinmclean Oct 17, 2024
76aa1fe
fix example command
justinmclean Oct 17, 2024
affd6ad
revert as the suggested name is not value
justinmclean Oct 17, 2024
abc2da8
simplify command options
justinmclean Oct 17, 2024
9808022
update command options
justinmclean Oct 17, 2024
423219c
Merge branch 'main' into CLI
justinmclean Oct 17, 2024
954b351
Make executable and runnable from bin directory. update command format.
justinmclean Oct 22, 2024
808fffc
revert back to using two args and fix command format
justinmclean Oct 22, 2024
848a381
remove meatlake form name and add back metalake option
justinmclean Oct 22, 2024
2fe9a80
add Gravitino URL as an environment variable
justinmclean Oct 22, 2024
526d305
update README to mention environment variables
justinmclean Oct 22, 2024
d2ded4c
improved description of met lake name and add to options docs
justinmclean Oct 22, 2024
cb5a5d7
remove version number from alias
justinmclean Oct 22, 2024
71107fd
order in alphabetical order
justinmclean Oct 22, 2024
a6aeb42
add final where needed
justinmclean Oct 22, 2024
ce3ddc1
made private
justinmclean Oct 22, 2024
70aaaf9
use Joiner instead of StringBuilder
justinmclean Oct 22, 2024
22ca191
update commands
justinmclean Oct 23, 2024
ab52cf2
improve command line error handling
justinmclean Oct 23, 2024
bd57944
add executable shell script
justinmclean Oct 23, 2024
445cd41
put back README and make it developer focused
justinmclean Oct 23, 2024
7ea7125
support ignore option to ignore client/server version mismatch
justinmclean Oct 23, 2024
0a8844e
move list commands up one entity
justinmclean Oct 23, 2024
c342d7e
add comment
justinmclean Oct 23, 2024
d2b28af
no need for class
justinmclean Oct 23, 2024
038e1f6
fix command error handling and unwanted output
justinmclean Oct 23, 2024
ce5fa18
remove example scripts
justinmclean Oct 23, 2024
069f425
add CLI documentation
justinmclean Oct 23, 2024
0accd4d
no message
justinmclean Oct 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions clients/cli/bin/errors.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -21,28 +21,29 @@

# Some of these examples assume you have the Apache Gravitino playground running.

alias gcli='java -jar clients/cli/build/libs/gravitino-cli-0.7.0-incubating-SNAPSHOT.jar'
shopt -s expand_aliases
alias gcli='java -jar ../../cli/build/libs/gravitino-cli-*-incubating-SNAPSHOT.jar'

# No such command
gcli unknown

# unknown command and entiry
gcli unknown --unknown
gcli unknown unknown

# unknown command
gcli metalake --unknown
gcli metalake unknown

# unknown entity
gcli unknown --list
gcli unknown list

# Name not specified
gcli metalake --details
gcli metalake details

# Unknown metalake name
gcli metalake --details --name unknown
gcli metalake details --name unknown

# Unknown catalog name
gcli catalog --details --name metalake_demo.unknown
gcli catalog details --name metalake_demo.unknown

# Missing catalog name
gcli catalog --details --name metalake_demo
gcli catalog details --name metalake_demo
87 changes: 45 additions & 42 deletions clients/cli/bin/examples.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#!/bin/bash

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
Expand All @@ -17,12 +19,11 @@
# under the License.
#

#!/bin/bash

# These examples assume you have the Apache Gravitino playground running.

unset GRAVITINO_METALAKE
alias gcli='java -jar clients/cli/build/libs/gravitino-cli-0.7.0-incubating-SNAPSHOT.jar'
shopt -s expand_aliases
alias gcli='java -jar ../../cli/build/libs/gravitino-cli-*-incubating-SNAPSHOT.jar'

# display help
gcli --help
Expand All @@ -31,73 +32,75 @@ gcli --help
gcli --version

# metalake details
gcli --details
gcli details

# metalake list
gcli --list
gcli list

# list all catalogs in a metalake
gcli metalake --list --name metalake_demo
gcli metalake list --metalake metalake_demo

# list catalog schema
gcli catalog --list --name metalake_demo.catalog_iceberg
gcli catalog --list --name metalake_demo.catalog_mysql
gcli catalog --list --name metalake_demo.catalog_postgres
gcli catalog --list --name metalake_demo.catalog_hive
gcli catalog list --metalake metalake_demo --name catalog_iceberg
gcli catalog list --metalake metalake_demo --name catalog_mysql
gcli catalog list --metalake metalake_demo --name catalog_postgres
gcli catalog list --metalake metalake_demo --name catalog_hive

# list catalog details
gcli catalog --details --name metalake_demo.catalog_iceberg
gcli catalog --details --name metalake_demo.catalog_mysql
gcli catalog --details --name metalake_demo.catalog_postgres
gcli catalog --details --name metalake_demo.catalog_hive
gcli catalog details --metalake metalake_demo --name catalog_iceberg
gcli catalog details --metalake metalake_demo --name catalog_mysql
gcli catalog details --metalake metalake_demo --name catalog_postgres
gcli catalog details --metalake metalake_demo --name catalog_hive

# list schema tables
gcli schema --list --name metalake_demo.catalog_postgres.hr
gcli schema --list --name metalake_demo.catalog_mysql.db
gcli schema --list --name metalake_demo.catalog_hive.sales
gcli schema list --metalake metalake_demo --name catalog_postgres.hr
gcli schema list --metalake metalake_demo --name catalog_mysql.db
gcli schema list --metalake metalake_demo --name catalog_hive.sales

# list schema details
gcli schema --details --name metalake_demo.catalog_postgres.hr
gcli schema --details --name metalake_demo.catalog_mysql.db
gcli schema --details --name metalake_demo.catalog_hive.sales
gcli schema details --metalake metalake_demo --name catalog_postgres.hr
gcli schema details --metalake metalake_demo --name catalog_mysql.db
gcli schema details --metalake metalake_demo --name catalog_hive.sales

# list table details
gcli table --list --name metalake_demo.catalog_postgres.hr.departments
gcli table --list --name metalake_demo.catalog_mysql.db.iceberg_tables
gcli table --list --name metalake_demo.catalog_hive.sales.products
gcli table list --metalake metalake_demo --name catalog_postgres.hr.departments
gcli table list --metalake metalake_demo --name catalog_mysql.db.iceberg_tables
gcli table list --metalake metalake_demo --name catalog_hive.sales.products

# Exmaples where metalake is set in an evironment variable
export GRAVITINO_METALAKE=metalake_demo

# metalake details
gcli metalake --details
gcli metalake details

# list all catalogs in a metalake
gcli metalake --list
gcli metalake list

# list catalog schema
gcli catalog --list --name catalog_iceberg
gcli catalog --list --name catalog_mysql
gcli catalog --list --name catalog_postgres
gcli catalog --list --name catalog_hive
gcli catalog list --name catalog_iceberg
gcli catalog list --name catalog_mysql
gcli catalog list --name catalog_postgres
gcli catalog list --name catalog_hive

# list catalog details
gcli catalog --details --name catalog_iceberg
gcli catalog --details --name catalog_mysql
gcli catalog --details --name catalog_postgres
gcli catalog --details --name catalog_hive
gcli catalog details --name catalog_iceberg
gcli catalog details --name catalog_mysql
gcli catalog details --name catalog_postgres
gcli catalog details --name catalog_hive

# list schema tables
gcli schema --list --name catalog_postgres.hr
gcli schema --list --name catalog_mysql.db
gcli schema --list --name catalog_hive.sales
gcli schema list --name catalog_postgres.hr
gcli schema list --name catalog_mysql.db
gcli schema list --name catalog_hive.sales

# list schema details
gcli schema --details --name catalog_postgres.hr
gcli schema --details --name catalog_mysql.db
gcli schema --details --name catalog_hive.sales
gcli schema details --name catalog_postgres.hr
gcli schema details --name catalog_mysql.db
gcli schema details --name catalog_hive.sales

# list table details
gcli table --list --name catalog_postgres.hr.departments
gcli table --list --name catalog_mysql.db.iceberg_tables
gcli table --list --name catalog_hive.sales.products
gcli table list --name catalog_postgres.hr.departments
gcli table list --name catalog_mysql.db.iceberg_tables
gcli table list --name catalog_hive.sales.products

unset GRAVITINO_METALAKE
40 changes: 27 additions & 13 deletions clients/cli/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
under the License.
-->

# Apache Gravitino CLI
# Apache Gravitino Command Line Interface

Apache Gravitino CLI is a command-line tool that interacts with the Gravitino server to manage and query entities like metalakes, catalogs, schemas, and tables. The tool provides options for listing information about Gravitino entities and in future versions support creating, deleting, and updating these entities.

Expand Down Expand Up @@ -76,46 +76,60 @@ To run the Gravitino CLI, use the following command structure:
```bash
usage: gcli [metalake|catalog|schema|table] [list|details|create|delete|update] [options]
Options
-C,--create create an entity
-D,--details list details about an entity
-f,--name <arg> full entity name (dot separated)
-h,--help command help information
-L,--list list entity children
justinmclean marked this conversation as resolved.
Show resolved Hide resolved
-r,--server Gravitino server version
-R,--delete delete an entity
-u,--url <arg> Gravitino URL (default: http://localhost:8090)
-U,--update update an entity
-v,--version Gravitino client version
```

## Commands
The following commands are available for entity management:

--list: List available entities
--details: Show detailed information about an entity
--create: Create a new entity
--delete: Delete an existing entity
--update: Update an existing entity
- list: List available entities
- details: Show detailed information about an entity
- create: Create a new entity
- delete: Delete an existing entity
- update: Update an existing entity

### Examples
List All Metalakes

```bash
gcli --list
gcli list
```

Get Details of a Specific Metalake

```bash
gcli metalake --details -name my-metalake
gcli metalake details --metalake my-metalake
```

List Tables in a Catalog

```bash
gcli metalake --list -name my-metalake
gcli metalake list --metalake my-metalake
justinmclean marked this conversation as resolved.
Show resolved Hide resolved
```

### Setting the Metalake name

As dealing with one Metalake is a typical scenario, you can set the Metalake name in several ways.

1. Passed in on the command line via the `--metalake` parameter.
2. Set via the 'GRAVITINO_METALAKE' environment variable.

The command line option overrides the environment variable.

### Setting the Gravitino URL

As you need to set the Gravitino URL for every command, you can set the URL in several ways.

1. Passed in on the command line via the `--url` parameter.
2. Set via the 'GRAVITINO_URL' environment variable.

The command line option overrides the environment variable.

## Running Tests

This project includes a suite of unit tests to verify its functionality.
Expand Down
26 changes: 11 additions & 15 deletions clients/cli/src/main/java/org/apache/gravitino/cli/FullName.java
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,17 @@ public FullName(CommandLine line) {
}

/**
* Retrieves the metalake name from the command line options, environment variables, or the first
* part of the full name.
* Retrieves the metalake name from the command line options, the GRAVITINO_METALAKE environment
* variable.
*
* @return The metalake name, or null if not found.
*/
public String getMetalakeName() {
// If specified on the command line use that
if (line.hasOption(GravitinoOptions.METALAKE)) {
return line.getOptionValue(GravitinoOptions.METALAKE);
}

// Cache the metalake environment variable
if (metalakeEnv == null) {
metalakeEnv = System.getenv("GRAVITINO_METALAKE");
Expand All @@ -53,9 +58,6 @@ public String getMetalakeName() {
// Check if the metalake name is set as an environment variable
if (metalakeEnv != null) {
return metalakeEnv;
// Extract the metalake name from the full name option
} else if (line.hasOption(GravitinoOptions.NAME)) {
return line.getOptionValue(GravitinoOptions.NAME).split("\\.")[0];
}

return null;
justinmclean marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -67,7 +69,7 @@ public String getMetalakeName() {
* @return The catalog name, or null if not found.
*/
public String getCatalogName() {
return getNamePart(1);
return getNamePart(0);
}

/**
Expand All @@ -76,7 +78,7 @@ public String getCatalogName() {
* @return The schema name, or null if not found.
*/
public String getSchemaName() {
return getNamePart(2);
return getNamePart(1);
}

/**
Expand All @@ -85,7 +87,7 @@ public String getSchemaName() {
* @return The table name, or null if not found.
*/
public String getTableName() {
return getNamePart(3);
return getNamePart(2);
}

/**
Expand All @@ -100,13 +102,7 @@ public String getNamePart(int position) {
if (line.hasOption(GravitinoOptions.NAME)) {
String[] names = line.getOptionValue(GravitinoOptions.NAME).split("\\.");

justinmclean marked this conversation as resolved.
Show resolved Hide resolved
/* Adjust position if metalake is part of the full name. */
String metalakeEnv = System.getenv("GRAVITINO_METALAKE");
if (metalakeEnv != null) {
position = position - 1;
}

if (names.length < position) {
if (names.length <= position) {
System.err.println(ErrorMessages.MALFORMED_NAME);
return null;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ public class GravitinoCommandLine {
private Options options;
private String entity;
private String command;
private String urlEnv;
justinmclean marked this conversation as resolved.
Show resolved Hide resolved

public static final String CMD = "gcli"; // recommended name
public static final String DEFAULT_URL = "http://localhost:8090";
Expand Down Expand Up @@ -182,13 +183,28 @@ protected void handleGeneralCommand() {
}

/**
* Gets the Gravitino URL from the command line options, or returns the default URL.
* Retrieves the Gravitinno URL from the command line options or the GRAVITINO_URL environment
* variable.
*
* @return The Gravitino URL to be used.
* @return The Gravitinno URL, or null if not found.
*/
protected String getUrl() {
return line.hasOption(GravitinoOptions.URL)
? line.getOptionValue(GravitinoOptions.URL)
: DEFAULT_URL;
public String getUrl() {
// If specified on the command line use that
if (line.hasOption(GravitinoOptions.URL)) {
return line.getOptionValue(GravitinoOptions.URL);
}

// Cache the Gravitino URL environment variable
if (urlEnv == null) {
urlEnv = System.getenv("GRAVITINO_URL");
}

// If set return the Gravitino URL environment variable
if (urlEnv != null) {
return urlEnv;
}

// Return the default localhost URL
return DEFAULT_URL;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,7 @@ public class GravitinoOptions {
public static final String SERVER = "server";
public static final String URL = "url";
public static final String NAME = "name";
public static final String LIST = "list";
public static final String DETAILS = "details";
public static final String CREATE = "create";
public static final String UPDATE = "update";
public static final String DELETE = "delete";
public static final String METALAKE = "metalake";

/**
* Builds and returns the CLI options for Gravitino.
Expand All @@ -49,13 +45,7 @@ public Options options() {
options.addOption(createSimpleOption("r", SERVER, "Gravitino server version"));
options.addOption(createArgOption("u", URL, "Gravitino URL (default: http://localhost:8090)"));
options.addOption(createArgOption("f", NAME, "full entity name (dot separated)"));

// specifying the command
options.addOption(createSimpleOption("L", LIST, "list entity children"));
options.addOption(createSimpleOption("D", DETAILS, "list details about an entity"));
options.addOption(createSimpleOption("C", CREATE, "create an entity"));
options.addOption(createSimpleOption("U", UPDATE, "update an entity"));
options.addOption(createSimpleOption("R", DELETE, "delete an entity"));
options.addOption(createArgOption("m", METALAKE, "Metalake"));

return options;
}
Expand Down
Loading
Loading