Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: adding config in CLI #39

Closed
wants to merge 105 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
7cf9dfd
Initial commit
aaravm Jun 7, 2024
0087881
added tes-autogenerated code for all the models
aaravm Jun 7, 2024
4560fc4
corrected workflow
aaravm Jun 7, 2024
4210498
added function create_task from autogenerated, trying some stuff for …
aaravm Jun 7, 2024
61fcb7a
made service.rs class
aaravm Jun 9, 2024
780e7ff
changed return type of service class, added TES class, using service …
aaravm Jun 11, 2024
47e3ca5
removed return type temporarily fortes class
aaravm Jun 11, 2024
df64e6d
added tests for service class
aaravm Jun 12, 2024
b0ea439
added unit test for tes class
aaravm Jun 15, 2024
b54b56f
changed a lot of names
aaravm Jun 16, 2024
4e5c8dc
merged configuration and service class
aaravm Jun 16, 2024
ce11ae1
refactoring
pavelnikonorov Jun 16, 2024
395b00a
tes.rs code moved to mod.rs to avoid import as ::tes::tes
pavelnikonorov Jun 17, 2024
b1c12fe
corrected tests in service.rs
aaravm Jun 18, 2024
432ca96
added service-info(not complete) and made build.sh
aaravm Jun 18, 2024
57406a1
Update Readme.md
aaravm Jun 18, 2024
ab08a44
Update Readme.md
aaravm Jun 18, 2024
02685c0
added serviceinfo class
aaravm Jun 20, 2024
cbba4ae
checking whether correct from servicinfo
aaravm Jun 20, 2024
be2e486
Merge remote-tracking branch 'origin/dev' into dev
aaravm Jun 20, 2024
e9ccf43
checking tests
aaravm Jun 20, 2024
d4faee2
added service-info unit test using funnel
aaravm Jun 22, 2024
f1bd0c5
added check function in TES
aaravm Jun 22, 2024
c9cdbfc
openapi models auto-generation: build script update, rm generated mod…
pavelnikonorov Jun 23, 2024
27bab17
refactoring, fixes, tests improvement
pavelnikonorov Jun 24, 2024
d656d3b
CI/CD workflow update
pavelnikonorov Jun 24, 2024
afbe68c
CI/CD workflow fix
pavelnikonorov Jun 24, 2024
da80b57
CI/CD workflow fix
pavelnikonorov Jun 24, 2024
a655a90
CI/CD workflow fix
pavelnikonorov Jun 24, 2024
6514f6e
CI/CD workflow fix
pavelnikonorov Jun 24, 2024
e0bdcf5
commenting out non-finished Task struct that fails CI/CD
pavelnikonorov Jun 24, 2024
a1cd3bf
applying code formating using
pavelnikonorov Jun 24, 2024
97595e0
rustfmt.toml: ignore code formatting checks of the autogenerated models
pavelnikonorov Jun 24, 2024
a6b63ac
changed return type of create to Task
aaravm Jun 26, 2024
431a831
added status to Task
aaravm Jun 26, 2024
e447b5a
corrected CI/CD
aaravm Jun 26, 2024
bcce6ec
trying out tes_status unit tests
aaravm Jun 26, 2024
f08c9dc
unit test working, with no params
aaravm Jun 27, 2024
d8ef075
completed the unit test of tes_status
aaravm Jun 27, 2024
49a702d
CI/CD framework
aaravm Jun 27, 2024
c110379
CI/CD
aaravm Jun 27, 2024
4d88c44
added cancel function
aaravm Jun 28, 2024
dda27a7
added cancel fn in Task
aaravm Jun 28, 2024
df1fc62
adding cancel unit test
aaravm Jun 28, 2024
a25c499
changed transport struct for optional params and data
aaravm Jun 28, 2024
3c92b70
changed the status fn to Task struct
aaravm Jun 28, 2024
bc07b65
moved cancel to Task, added get in TES
aaravm Jun 28, 2024
0f7decc
added list tasks fn
aaravm Jun 29, 2024
19ece96
correcting get list() fn
aaravm Jun 29, 2024
4e415a5
cargo fmt: code formatting
pavelnikonorov Jul 1, 2024
ac64bb3
tests simplifaction though TES and Task objects reuse
pavelnikonorov Jul 1, 2024
1487961
corrected the list fn
aaravm Jul 1, 2024
976846b
merged changes
aaravm Jul 1, 2024
c96b636
Sorcery-ai suggestion
aaravm Jul 1, 2024
d613722
Sorcery-ai suggestion
aaravm Jul 1, 2024
4f482d3
Apply suggestions from code review
aaravm Jul 1, 2024
ebc9bec
sorcery-ai suggestions
aaravm Jul 1, 2024
e56a623
sorcery-ai suggestions
aaravm Jul 1, 2024
d37b000
sorcery-ai suggestions
aaravm Jul 1, 2024
7725298
added some Pavel's suggestions
aaravm Jul 4, 2024
bad81eb
readding accidently deleted workflows folder
aaravm Jul 4, 2024
c95ac80
Update Readme.md
aaravm Jul 4, 2024
198e0f8
changed Cargo.toml
aaravm Jul 4, 2024
206c6c4
changed the sample file, changed author name
aaravm Jul 4, 2024
2f2b6a8
changed versions of ci.yml
aaravm Jul 5, 2024
4a41548
added parameters in a much easier manner
aaravm Jul 7, 2024
2d48bdd
changing git ignore file
aaravm Jul 7, 2024
a5b484e
changed local and github workflows
aaravm Jul 8, 2024
598646f
correct github workflows
aaravm Jul 8, 2024
1369f7e
correct github workflows
aaravm Jul 8, 2024
8ec57cc
using cargo nextest for tests
aaravm Jul 8, 2024
1913d3d
correcting ci/cd
aaravm Jul 8, 2024
ee0948c
correcting ci/cd
aaravm Jul 8, 2024
44a1d64
removing commented out code
aaravm Jul 8, 2024
5a13d19
moved ListTaskParam to model.rs
aaravm Jul 8, 2024
0e29df7
added unit coverage
aaravm Jul 8, 2024
e02346c
trying to remove installing rust in ci/cd
aaravm Jul 8, 2024
dff1a21
trying to remove installing rust in ci/cd
aaravm Jul 8, 2024
bedf68f
trying to remove installing rust in ci/cd
aaravm Jul 8, 2024
4ee3302
trying to remove installing rust in ci/cd
aaravm Jul 8, 2024
b8ea658
trying to remove installing rust in ci/cd
aaravm Jul 8, 2024
2d24c43
trying to remove installing rust in ci/cd
aaravm Jul 8, 2024
f6ef231
trying to remove installing rust in ci/cd
aaravm Jul 8, 2024
5d9c4bf
remove comments
aaravm Jul 8, 2024
0d8d25c
changed openapi generator to Bash Launcher
aaravm Jul 9, 2024
7fa6bd6
remove nodejs
aaravm Jul 9, 2024
00b6367
changed gitignore
aaravm Jul 9, 2024
84a86c8
taken initial cli from nanopub.rs
aaravm Jul 11, 2024
7fe8709
initial changes
aaravm Jul 11, 2024
a76ddd3
cargo.toml separating
aaravm Jul 15, 2024
433dd90
initial version of CLI
aaravm Jul 15, 2024
1d80b7c
trying to make cli work
aaravm Jul 19, 2024
1ae659a
initial version CLI
aaravm Jul 19, 2024
0011f39
added read from .config file
aaravm Jul 23, 2024
87fb971
can use url in task too
aaravm Jul 23, 2024
1b96865
added list_tasks fn
aaravm Jul 24, 2024
d57bede
corrected list fn
aaravm Jul 25, 2024
ce90baa
added get fn
aaravm Jul 25, 2024
4bb1f59
correct ci/cd
aaravm Jul 25, 2024
215ae5a
added status fn
aaravm Jul 25, 2024
6be3e10
added cancel
aaravm Jul 25, 2024
a6d98b3
added config
aaravm Jul 25, 2024
d44fa38
added docs
aaravm Jul 25, 2024
768b4cf
added example
aaravm Jul 28, 2024
786a855
sorcery suggestions
aaravm Aug 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: GitHub CI/CD Workflow

on: [push, pull_request]

jobs:
build:

runs-on: ubuntu-latest
container:
image: ubuntu:24.04
env:
OPENSSL_DIR: /usr/include/openssl
OPENSSL_LIB_DIR: /usr/lib/x86_64-linux-gnu
OPENSSL_INCLUDE_DIR: /usr/include/openssl

steps:
- name: Install dependencies
run: |
apt-get update
apt-get install -y curl git build-essential libssl-dev

- uses: actions/checkout@v4 # checkout the repository
- name: Install Rust
run: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
echo "$HOME/.cargo/bin" >> $GITHUB_PATH

- name: Set up JDK 11 # required for build-models.sh
uses: actions/setup-java@v3
with:
distribution: 'adopt'
java-version: '11'

- name: Install Go
uses: actions/setup-go@v4
with:
go-version: '^1.22'
- name: Install Funnel
run: |
if [ -d "funnel" ]; then rm -Rf funnel; fi
git clone https://github.com/ohsu-comp-bio/funnel.git
cd funnel && make && make install && cd ..
- uses: actions/checkout@v4 # checkout the repository
- name: Build models
run: |
bash ./build-models.sh
- name: Build
run: |
cargo build --verbose
- name: Run tests
run: |
bash ./run-tests.sh
- name: Lint
run: |
cargo clippy -- -D warnings
- name: Format
run: |
cargo fmt -- ./lib/src/serviceinfo/models/*.rs # workaround to fix autogenerated code formatting
cargo fmt -- ./lib/src/tes/models/*.rs
cargo fmt -- --check
98 changes: 98 additions & 0 deletions .github/workflows/local.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
name: Local CI/CD Workflow

on: workflow_dispatch

jobs:
build:

runs-on: ubuntu-latest
container:
image: ubuntu:24.04
env:
OPENSSL_DIR: /usr/include/openssl
OPENSSL_LIB_DIR: /usr/lib/x86_64-linux-gnu
OPENSSL_INCLUDE_DIR: /usr/include/openssl

steps:

- name: Cache Rust dependencies
uses: actions/cache@v3
with:
path: ~/.cargo
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: ${{ runner.os }}-cargo-

- name: Cache Rust build output
uses: actions/cache@v3
with:
path: target
key: ${{ runner.os }}-build-${{ hashFiles('**/Cargo.lock') }}
restore-keys: ${{ runner.os }}-build-

- name: Cache Maven dependencies
uses: actions/cache@v3
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-maven-

- name: Cache Go modules
uses: actions/cache@v3
with:
path: ~/.cache/go-build
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: ${{ runner.os }}-go-

- name: Cache Funnel dependencies
uses: actions/cache@v3
with:
path: ~/funnel/build
key: ${{ runner.os }}-funnel-${{ hashFiles('**/funnel/*') }}
restore-keys: ${{ runner.os }}-funnel-

- uses: actions/checkout@v4 # checkout the repository
- name: Install Rust
run: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
echo "$HOME/.cargo/bin" >> $GITHUB_PATH

- name: Set up JDK 11 # required for build-models.sh
uses: actions/setup-java@v3
with:
distribution: 'adopt'
java-version: '11'

- name: Install Go
uses: actions/setup-go@v4
with:
go-version: '^1.22'
- name: Install Funnel
run: |
if [ -d "funnel" ]; then rm -Rf funnel; fi
git clone https://github.com/ohsu-comp-bio/funnel.git
cd funnel && make && make install && cd ..
- uses: actions/checkout@v4 # checkout the repository
- name: Build models
run: |
. $HOME/.cargo/env
bash ./build-models.sh
- name: Build
run: |
. $HOME/.cargo/env
cargo build --verbose
- name: Run tests
run: |
. $HOME/.cargo/env
bash ./run-tests.sh
- name: Lint
run: |
. $HOME/.cargo/env
cargo clippy -- -D warnings
- name: Format
run: |
. $HOME/.cargo/env
# rustup install nightly – fails for some reason
# rustup default nightly
cargo fmt -- ./lib/src/serviceinfo/models/*.rs # workaround to fix autogenerated code formatting
cargo fmt -- ./lib/src/tes/models/*.rs
cargo fmt -- --check # --config-path ./rustfmt.toml
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
target/
debug/
Cargo.lock
openapitools.json
*.log
funnel-work-dir/
funnel/
lib/src/*/models
13 changes: 13 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[workspace]
members = [
"cli",
"lib"
]

[workspace.package]
version = "0.1.0"
authors = ["Aarav Mehta <[email protected]> (Google Summer of Code Participant)", "Pavel Nikonorov <[email protected]> (Google Summer of Code Mentor; GENXT)", "ELIXIR Cloud & AAI <[email protected]> (ELIXIR Europe)"]
edition = "2021"
readme = "./README.md"
license-file = "LICENSE.txt"
repository = "https://github.com/elixir-cloud-aai/ga4gh-sdk.git"
36 changes: 36 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# A Generic SDK and CLI for GA4GH API services
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (documentation): Consider capitalizing 'Services' for consistency.

To maintain a consistent title case format, consider changing 'services' to 'Services'.


## Building

First, clone the repository, and then run the following command to automatically generate models using OpenAPI specifications:
```
bash ./build-models.sh
```

To build the project:
```
cargo build
```

## Running the tests

Before running the tests, you need to install Funnel, a task execution system that is compatible with the GA4GH TES API. Follow the instructions in the [Funnel Developer's Guide](https://ohsu-comp-bio.github.io/funnel/docs/development/developers/) to install Funnel.

Once you have installed Funnel, you can run the tests. This will automatically run Funnel as well:

```
bash ./run-tests.sh
```
or, you can run using cargo nextest using
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (documentation): Repetition of the word 'using'.

The phrase 'using cargo nextest using' should be rephrased to 'using cargo nextest'.

```
cargo nextest run
```
For checking the unit converage, you can run:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (documentation): Typo in the word 'converage'.

The word 'converage' should be corrected to 'coverage'.

```
cargo llvm-cov nextest
```

To test the CI/CD workflow locally, install `act` and run the following command:
```
act -j build --container-architecture linux/amd64 -P ubuntu-latest=ubuntu:24.04 --reuse
```
77 changes: 77 additions & 0 deletions build-models.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Exit immediately if a command exits with a non-zero status.
set -e

# Ensure the OpenAPI Generator JAR file is set up
mkdir -p ~/bin/openapitools
OPENAPI_GENERATOR_JAR=~/bin/openapitools/openapi-generator-cli.jar
if [ ! -f "$OPENAPI_GENERATOR_JAR" ]; then
curl -L https://repo1.maven.org/maven2/org/openapitools/openapi-generator-cli/7.7.0/openapi-generator-cli-7.7.0.jar -o "$OPENAPI_GENERATOR_JAR"
fi

get_git_repo_name() {
# Extract the URL of the remote "origin"
url=$(git config --get remote.origin.url)

# Extract the repository name from the URL
repo_name=$(basename -s .git "$url")

echo "$repo_name"
}

SCRIPT_DIR="$(pwd)"

generate_openapi_models() {
# Parameters
OPENAPI_SPEC_PATH="$1"
API_NAME="$2"
DESTINATION_DIR="$3"

# Define the temporary output directory for the OpenAPI generator
TEMP_OUTPUT_DIR=$(mktemp -d)

# Remove the temporary directory at the end of the script
trap 'rm -rf "$TEMP_OUTPUT_DIR"' EXIT

# Run the OpenAPI generator CLI using the JAR file
java -jar "$OPENAPI_GENERATOR_JAR" generate -g rust \
-i "$OPENAPI_SPEC_PATH" \
-o "$TEMP_OUTPUT_DIR" \
--additional-properties=useSingleRequestParameter=true

# Check if the generation was successful
if [ $? -ne 0 ]; then
echo "OpenAPI generation failed. Check the verbose output for details."
exit 1
fi

# Remove the openapitools.json file
rm -f ./openapitools.json

echo "TEMP_OUTPUT_DIR is $TEMP_OUTPUT_DIR"

# Modify the import statements in each generated file
SED_RULE="s/use crate::models;/#![allow(unused_imports)]\n#![allow(clippy::empty_docs)]\nuse crate::$API_NAME::models;/"
for file in $(find "$TEMP_OUTPUT_DIR" -name '*.rs'); do
if [[ "$OSTYPE" == "darwin"* ]]; then
# macOS (BSD) sed syntax
sed -i '' "$SED_RULE" "$file"
else
# Linux (GNU) sed syntax
sed -i "$SED_RULE" "$file"
fi
done

rm -rf "$DESTINATION_DIR/models"
mkdir -p "$DESTINATION_DIR"
cp -r "$TEMP_OUTPUT_DIR/src/models" "$DESTINATION_DIR"

echo "OpenAPI generation complete. Models copied to $DESTINATION_DIR"
}

generate_openapi_models \
"https://raw.githubusercontent.com/ga4gh-discovery/ga4gh-service-info/develop/service-info.yaml" \
"serviceinfo" "$SCRIPT_DIR/lib/src/serviceinfo/"

generate_openapi_models \
"https://raw.githubusercontent.com/ga4gh/task-execution-schemas/develop/openapi/task_execution_service.openapi.yaml" \
"tes" "$SCRIPT_DIR/lib/src/tes/"
24 changes: 24 additions & 0 deletions cli/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
[package]
name = "cli"
description = """
A cross-platform CLI tool written in Rust to use GA4GH-sdk
"""
repository = "https://github.com/elixir-cloud-aai/ga4gh-sdk/tree/main/cli"
version.workspace = true
authors.workspace = true
edition.workspace = true
readme.workspace = true
license-file.workspace = true

[dependencies]
ga4gh-lib = { path = "../lib" }
clap = "3.0"
clap_complete = "3.0"
tokio = { version = "1", features = ["full"] }
serde_json = "^1.0"
tempfile = "3.2"
dirs = "5.0.1"

[[bin]]
name = "cli"
path = "src/main.rs"
Loading
Loading