Skip to content

Commit

Permalink
Merge pull request #14 from david4096/updates
Browse files Browse the repository at this point in the history
Updates, HCA feedback
  • Loading branch information
briandoconnor authored Feb 5, 2018
2 parents 2c06c26 + f14a2a9 commit e6962e8
Show file tree
Hide file tree
Showing 12 changed files with 852 additions and 1,004 deletions.
3 changes: 0 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +0,0 @@
[submodule "task-execution-schemas"]
path = task-execution-schemas
url = https://github.com/ga4gh/task-execution-schemas.git
4 changes: 2 additions & 2 deletions CWLFile
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ includes:
- class: Directory
location: proto
- class: Directory
location: task-execution-schemas/proto
location: proto
proto:
class: File
location: proto/workflow_execution.proto
location: proto/workflow_execution_service.proto
cwl:tool: protoc.cwl
49 changes: 17 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,44 @@
![ga4gh logo](http://genomicsandhealth.org/files/logo_ga.png)
<img src="https://www.ga4gh.org/gfx/GA-logo-horizontal-tag-RGB.svg" alt="GA4GH Logo" style="width: 100px;"/>

Schemas for the Workflow Execution Service (WES) API
====================================================

This is used by the Data Working Group - Containers and Workflows Task Team

<img src="swagger_editor.png" width="48">[View in Swagger](http://editor.swagger.io/#/?import=https://raw.githubusercontent.com/ga4gh/workflow-execution-schemas/0.1.0/swagger/proto/workflow_execution.swagger.json)

The [Global Alliance for Genomics and Health](http://genomicsandhealth.org/) is an international
coalition, formed to enable the sharing of genomic and clinical data.

Containers and Workflows Task Team
----------------------------------
Cloud Work Stream
-----------------

The [Data Working Group](http://ga4gh.org/#/) concentrates on data representation, storage,
The [Cloud Work Stream](https://ga4gh/cloud) concentrates on data representation, storage,
and analysis, including working with platform development partners and
industry leaders to develop standards that will facilitate
interoperability. The Containers & Workflows working group is an informal, multi-vendor working group focused on standards for exchanging Docker-based tools and CWL/WDL workflows, execution of Docker-based tools and workflows on clouds, and abstract access to cloud object stores.
interoperability. The Cloud Work Stream is an informal, multi-vendor working group focused on standards for exchanging Docker-based tools and CWL/WDL workflows, execution of Docker-based tools and workflows on clouds, and abstract access to cloud object stores.

What is WES?
============

This is the home of the Workflow Execution Schema proposal. The Workflow
Execution Schema is a minimal common API describing how a user can submit
workflow requests to workflow execution systems in a standardized ways.
The Workflow Execution Schema is a minimal common API describing how a user can submit
workflow requests to workflow execution systems in standardized ways.
Workflow execution engines (SevenBridges, FireCloud, etc) can support this API so users can make workflow requests
programmatically, adding the ability to scale up. In addition, these workflow services could have (and probably do have)
UIs that would (possibly) use this API under the hood to facilitate workflow execution requests.

Having this standard API supported by multiple execution engines will give people options of processing
the same workflow (CWL or WDL) across different workflow execution platforms running across various clouds/environments.
As an example use case, one can find a workflow in CWL on [Dockstore.org](http://dockstore.org), use Dockstore to
generate a JSON parameterization file, and submit this a GA4GH-compliant
generate a JSON parameterization file, and submit this to a GA4GH-compliant
workflow execution service.

Key features of the current API proposal:

* ability to request a workflow run using CWL or WDL (and maybe future formats)
* ability to parameterize that workflow using a JSON schema (ideally a future version would be in common between CWL and WDL)
* ability to get information about running workflows, status, errors, output file locations etc
* ability to get information about running workflows, status, errors, output file locations, etc.
* to search for workflows by arbitrary key/values

Outstanding questions:

* a common JSON parameterization format, see work by Peter, is that checked in?
* a common JSON parameterization format
* standardizing terms, job, workflow, steps, tools, etc
* reference implementation at https://github.com/common-workflow-language/cwltool-service/tree/ga4gh-wes
* validation service for testing WES implementations' conformance to the spec
Expand All @@ -54,9 +49,9 @@ Outstanding questions:
How to view
------------

See the swagger editor to view our [schema in progress](http://editor.swagger.io/#/?import=https://raw.githubusercontent.com/ga4gh/workflow-execution-schemas/develop/src/main/resources/swagger/ga4gh-tool-discovery.yaml).
The file `swagger/workflow_execution_service.swagger.json` contains the API description.

If the current schema fails to validate, visit [debugging](http://online.swagger.io/validator/debug?url=https://raw.githubusercontent.com/ga4gh/workflow-execution-schemas/develop/src/main/resources/swagger/ga4gh-tool-discovery.yaml)
Please visit http://ga4gh.github.io/workflow-execution-schemas to view this document in Swagger UI.

Building Documents
------------------
Expand All @@ -65,24 +60,14 @@ Make sure you have Docker installed for your platform and the `cwltool`.

virtualenv env
source env/bin/activate
pip install setuptools==28.8.0
pip install cwl-runner cwltool==1.0.20161114152756 schema-salad==1.18.20161005190847 avro==1.8.1

Make sure you have the [submodule](http://stackoverflow.com/questions/3939055/submodules-files-are-not-checked-out) checked out:

git submodule update --init --recursive
pip install -r requirements.txt

You can generate the [Swagger](http://swagger.io/) YAML from the Protocol Buffers:
You can generate the [Swagger](http://swagger.io/) from the Protocol Buffers:

cwltool CWLFile
sh tools/prepare_openapi.sh

Find the output in `workflow_execution.swagger.json` and this can be loaded in the [Swagger editor](http://swagger.io/swagger-editor/). Use the GitHub raw feature to generate a URL you can load.

When you're happy with the changes, checkin this file:

mv workflow_execution.swagger.json swagger/proto/

And commit your changes.
The resulting OpenAPI description will be in the `swagger` directory.

How to contribute changes
-------------------------
Expand All @@ -100,4 +85,4 @@ More Information
----------------

* [Global Alliance for Genomics and Health](http://genomicsandhealth.org)
* [Google Forum](https://groups.google.com/forum/#!forum/ga4gh-dwg-containers-workflows)
* [GA4GH Cloud Work Stream)[https://ga4gh.cloud]
190 changes: 0 additions & 190 deletions proto/workflow_execution.proto

This file was deleted.

Loading

0 comments on commit e6962e8

Please sign in to comment.