Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IBCDPE-749] Clarify points in documentation #36

Merged
merged 5 commits into from
Jan 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,9 @@
# IMPORTANT: The ':' and '@' characters before and after
# the access token are required. Do not omit them.
#
# Examples:
# Structure:
# export SEVENBRIDGES_CONNECTION_URI=sbg://:<access-token>@<api-base-endpoint>[/?project=<project-id>]
# Examples:
# export SEVENBRIDGES_CONNECTION_URI=sbg://:f560[...][email protected]/v2
# export SEVENBRIDGES_CONNECTION_URI=sbg://:f560[...][email protected]/v2/?project=bgrande/sandbox

Expand All @@ -68,7 +69,9 @@
# IMPORTANT: The ':' and '@' characters before and after
# the access token are required. Do not omit them.
#
# Examples:
# Structure:
# export NEXTFLOWTOWER_CONNECTION_URI=tower://:<access-token>@<api-base-endpoint>[/?workspace=<organization-name>/<workspace-name>]
# Examples:
# export NEXTFLOWTOWER_CONNECTION_URI=tower://:eyJ0[...][email protected]
# export NEXTFLOWTOWER_CONNECTION_URI=tower://:eyJ0[...][email protected]/api/?workspace=sage-bionetworks/example-project
# export NEXTFLOWTOWER_CONNECTION_URI=tower://:eyJ0[...][email protected]/api/?workspace=sage-bionetworks/example-dev-project
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,18 @@ This Python package provides the components to connect various third-party servi

This repository includes a demonstration script called [`demo.py`](demo.py), which showcases how you can use `py-orca` to launch and monitor your workflows on Nextflow Tower. Specifically, it illustrates how to process an RNA-seq dataset using a series of workflow runs, namely `nf-synstage`, `nf-core/rnaseq`, and `nf-synindex`. `py-orca` can be used with any Python-compatible workflow management system to orchestrate each step (_e.g._ Airflow, Prefect, Dagster). The demonstration script uses [Metaflow](https://metaflow.org/) because it's easy to run locally and has an intuitive syntax.

The script assumes that the following environment variables are set.
Refer to [`.env.example`](.env.example) for the format of their values as well as examples. You can set these environment variables using whatever method you prefer (_e.g._ using an `.env` file, sourcing a shell script).
The script assumes that the following environment variables are set. Before setting them up, ensure that you have an AWS profile configured for a role that has access to the dev/ops tower workspace you plan to launch your workflows from. You can set these environment variables using whatever method you prefer (_e.g._ using an `.env` file, sourcing a shell script, etc).
Refer to [`.env.example`](.env.example) for the format of their values as well as examples.

- `NEXTFLOWTOWER_CONNECTION_URI`
- `SYNAPSE_CONNECTION_URI`
- `AWS_PROFILE` (or another source of AWS credentials)

Once your environment is set, you can create a virtual environment, install the Python dependencies, and run the demonstration script (after downloading it) as follows. Note that you will need to update the `s3_prefix` parameter so that it points to an S3 bucket that is accessible to your Tower workspace.

### Manually creating a virtual environment
### Creating and setting up your py-`orca` virtual environment and executing `demo.py`

Below are the instructions for creating and setting up your virtual environment and executing the `demo.py`. If you would like to set up a developer environment with the relevant dependencies, you can execute the shell script [dev_setup](https://github.com/Sage-Bionetworks-Workflows/py-orca/blob/main/dev_setup.sh) in a clone of this repository stored on your machine.
```bash
# Create and activate a Python virtual environment (tested with Python 3.10)
python3 -m venv venv/
Expand Down