Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Add feature to emit YAMLs of existing entities #112

Draft
wants to merge 7 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,18 +87,22 @@ export TOWER_ACCESS_TOKEN=<your access token>

## Usage

To confirm the installation of `seqerakit`, configuration of the Seqera Platform CLI and connection is working as expected:
To confirm the installation of `seqerakit`, configuration of the Seqera Platform CLI and connection is working as expected. This will run the `tw info` command under the hood:

```bash
seqerakit --info
```

Use the `-h` or `--help `parameter to list the available commands and their associated options:

Use the `--help` or `-h` parameter to list the available commands and their associated options:
```bash
seqerakit --help
```

Use `--version` or `-v` to retrieve the current version of your seqerakit installation:
```bash
seqerakit --version
```

### Dryrun

To print the commands that would executed with `tw` when using a YAML file, you can run `seqerakit` with the `--dryrun` flag:
Expand Down Expand Up @@ -208,7 +212,18 @@ compute-envs:
file-path: './compute-envs/my_aws_compute_environment.json' # required
overwrite: True
```
## YAML Dump Feature
**Note: This is an experimental feature and is under active development. It may not be fully stable is subject to further change and improvements in future releases. If you encounter issues using this feature or want to provide feedback, please feel free to contribute via [GitHub Issues](https://github.com/seqeralabs/seqera-kit/issues) or [Pull Requests](https://github.com/seqeralabs/seqera-kit/pulls).**

To generate definitions for objects and entities that exist within a Workspace on Seqera Platform in YAML format, you can use the `seqerakit dump` command.

To use this feature, specify the Workspace name or ID for which you want to generate YAML files for, and optionally, a prefix for your YAML filenames (this defaults to the Workspace):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just write a single file? -o output.yml?

```bash
seqerakit dump --workspace <workspace_name/ID> --prefix <optional_prefix>
```
This command will generate YAML definitions for entities (i.e. Pipelines, Compute Environments) in the specified Workspace. These definitions can be used to compare to YAML files initially used to create entities and analyze changes, or to maintain provenance and IaC of the current state of objects in the Platform.

*Note*: This feature currently only supports emitting YAML definitions for Compute Environments and Pipelines.

## Quick start

Expand Down
3 changes: 3 additions & 0 deletions seqerakit/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import importlib.metadata as importlib_metadata

__version__ = importlib_metadata.version(__name__)
83 changes: 67 additions & 16 deletions seqerakit/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,51 +22,89 @@
import sys

from pathlib import Path
from seqerakit import seqeraplatform, helper, overwrite
from seqerakit import seqeraplatform, helper, overwrite, dump
from seqerakit.seqeraplatform import ResourceExistsError, ResourceCreationError
from seqerakit import __version__


logger = logging.getLogger(__name__)


def parse_args(args=None):
parser = argparse.ArgumentParser()
parser.add_argument(
parser = argparse.ArgumentParser(
description="Seqerakit: Python wrapper for the Seqera Platform CLI"
)

# General options
general = parser.add_argument_group("General Options")
general.add_argument(
"-l",
"--log_level",
default="INFO",
choices=("CRITICAL", "ERROR", "WARNING", "INFO", "DEBUG"),
help="The desired log level (default: INFO).",
type=str.upper,
help="Set the logging level.",
)
parser.add_argument(
general.add_argument(
"--info",
"-i",
action="store_true",
help="Display information about the Seqera Platform and exit",
help="Display Seqera Platform information and exit.",
)
parser.add_argument(
general.add_argument(
"--dryrun",
"-d",
action="store_true",
help="Print the commands that would be executed without running them.",
help="Print the commands that would be executed.",
)
parser.add_argument(
general.add_argument(
"--version",
"-v",
action="version",
version=f"%(prog)s {__version__}",
help="Show version number and exit.",
)

# YAML processing options
yaml_processing = parser.add_argument_group("YAML Processing Options")
yaml_processing.add_argument(
"yaml",
type=Path,
nargs="*", # allow multiple YAML paths
help="One or more YAML files with Seqera Platform resources to create",
nargs="*",
help="One or more YAML files with Seqera Platform resource definitions.",
)
parser.add_argument(
yaml_processing.add_argument(
"--delete",
action="store_true",
help="Recursively delete all resources defined in the YAML file(s)",
help="Recursively delete resources defined in the YAML files.",
)
parser.add_argument(
yaml_processing.add_argument(
"--cli",
dest="cli_args",
type=str,
help="Additional arguments to pass to Seqera Platform"
" CLI enclosed in double quotes (e.g. '--cli=\"--insecure\"')",
)

# YAML dumping options
yaml_dumping = parser.add_argument_group("YAML Dumping Options")
yaml_dumping.add_argument(
"--dump", help="Dump Seqera Platform entity definitions to YAML."
)
yaml_dumping.add_argument(
"--workspace",
"-w",
dest="workspace",
type=str,
help="Name/ID of the workspace to dump YAML definitions for",
)
yaml_dumping.add_argument(
"--prefix",
"-p",
dest="prefix",
type=str,
help="Prefix for output YAML files (defaults to workspace name).",
)

return parser.parse_args(args)


Expand Down Expand Up @@ -138,11 +176,24 @@ def main(args=None):
print(sp.info())
return

if not options.yaml:
if not options.yaml and not options.dump:
logging.error(
" No YAML(s) provided. Please provide atleast one YAML configuration file."
)
sys.exit(1)
if options.dump:
if not options.workspace:
logging.error(
" Please provide a workspace name or ID to dump YAML definitions for."
)
sys.exit(1)
else:
sp = seqeraplatform.SeqeraPlatform()
dp = dump.DumpYaml(sp, options.workspace)
if options.prefix:
dp.generate_yaml_dump(options.prefix)
else:
dp.generate_yaml_dump(options.workspace)

# Parse CLI arguments into a list
cli_args_list = options.cli_args.split() if options.cli_args else []
Expand Down
Loading