v0.4.0

miniwdl run

• Improved logging with tty colors and other enhancements
• New subcommand miniwdl run_self_test runs a little workflow to test the installation
• Library functions involving file I/O (e.g. read_string(), write_lines()) are now available in workflows as well as tasks (#204)
• Prevent use of host files neither supplied in the workflow inputs nor generated by the workflow (e.g. /etc/passwd)
• Fix up type coercions in call inputs and struct instances

Remaining limitations:

• Calls run sequentially (#202, #203)
• Task input files are mounted read-only; commands to rename or remove them will fail (#210)
• Task output files must reside in or under the initial working directory (#214)
• Namespace discrepancies may interfere with specifying optional inputs to sub-workflow calls (#193)

Please vote on these and/or report other interoperability problems via Issues!

WDL package

Breaking API changes:

• WDL.Env rewritten with an easier, more-Pythonic interface
• Simplify compound Value constructors so that e.g. Value.Array needs to be told the array item type rather than the full Type.Array(...)

We now plan to minimize further breaking API changes to the WDL AST data structures, on the way to a 1.0 version.

Other changes:

• Add __str__ to Expr classes (#215 @DavyCats)
• String-to-Int/Float coercions allowed, may fail at runtime (#198)
• Statically reject use of stdout(), stderr(), and glob() outside of task output sections
• Set pos : SourcePosition attribute on the subset of WDL.Type objects instantiated by the syntax parser (e.g. Decl.type & StructTypeDef @dinvlad)
• Fix WDL.resolve_file_import() pass-through call to WDL.Tree.resolve_file_import() (@dinvlad)
• Tighten name collision rules for 1.0+ workflow outputs (#196)

Housekeeping

• Parallelized unit test suite using pytest
• Cromwell 45, lark-parser 0.7.3

v0.3.0

miniwdl run

🏁 This is the first release with a credible built-in capability to execute workflows on the local host. It's in early testing, but please try it out and let us know about any interoperability problems via Issues!

Salient limitations:

• Calls run sequentially (#202, #203)
• Task input files are mounted read-only; commands to rename or remove them will fail (#210)
• Task output files must reside in or under the initial working directory (#214)
• Library functions that involve file I/O (e.g. read_string(), write_lines()) are available in tasks but not workflows (#204)
• Namespace discrepancies may interfere with specifying optional inputs to sub-workflow calls (#193)

miniwdl check

• detects cyclic dependencies involving scatter & conditional expressions

miniwdl cromwell

• accept --options with Cromwell workflow options JSON file (#206 @prihoda)
• fix CLI arg completer regression
• Cromwell 44

WDL

• Refactor workflow AST to facilitate runtime dependency analysis, with WorkflowNode and WorkflowSection base classes exposing dependency ID strings, and explicit Gather nodes
• Refactor WDL.load() to permit overriding file-reading logic (#177 @dinvlad)
• Rename WDL.{Workflow,Scatter,Conditional}.elements to .body
• Rename WDL.Expr.Ident.ctx to WDL.Expr.Ident.referee

Housekeeping

• Replace wget subprocess calls with Python standard library functionality (#191 @nh13)
• Make all of package's internal imports relative & clean up naming for readability

v0.2.2

miniwdl cromwell

• disable --path since it doesn't work with Cromwell (see #131, #118)

WDL

• clean up & document programmatic access to miniwdl check lint warnings
• parser: allow comma after last element of map and object literals, as well as arrays

miniwdl run

• new subcommand shares miniwdl cromwell's command-line interface, instead invoking the work-in-progress internal runner. Currently, it can be used to run individual tasks (WDL documents with one task) on the local host.

v0.2.1

miniwdl check

• UnknownRuntimeKey for task runtime entries not on a list of known keys (#150 @pshapiro4broad)

miniwdl cromwell

• accept --input INPUTS.json with a Cromwell-style input JSON file, merging in any additional inputs from the command line (#165 @prihoda)
• allow override of Cromwell JAR and config file through arguments and env vars (#171 @cmarkello)
• Cromwell 42

WDL package

• struct initialization from map literal (where the key type is String-coercible and the value type is coercible to each respective struct member type)
• struct literals (object/map) may omit optional struct members
• parser: remove remaining constraints on the order of workflow elements/sections (#173 @patmagee)

meta

• Add biowdl/aligning and biowdl/expression-quantification test corpi
• miniwdl package is available through conda-forge (@rhpvorderman)

v0.2.0

miniwdl check

• detect call name collisions even if callee has no outputs (#133)
• UnusedDeclaration ignore likely secondary files based on name suffix case-insensitively

miniwdl cromwell

• adds support for shell tab-completion of the available input names, using argcomplete

WDL

• add WDL.values_to_json and WDL.values_from_json for conversion between WDL.Env.Values and Cromwell-style JSON
• WDL.load can take source_text : str of the source code instead of reading it from disk/URI (#154)
• provide pos : SourcePosition for SyntaxError and ImportError (#153 @dinvlad)
• task 'non-input' declarations may interleave anywhere among other task sections (#156)
• workflow output expressions can reference previous workflow outputs (#155 @rhpvorderman)
• exclude output declarations from Workflow.available_inputs (#135 @patmagee)
• StdLib refactoring and several additional function implementations
• Address value coercion loose ends: enforcing optional/nonempty quantifiers, recursive coercions in compound types, refined error objects & messages
• Errors no longer embed source line/column in the textual message; the pos : SourcePosition attribute remains available

meta

• Add biowdl/tasks test corpus
• Improving test suite Mac OS X compatibility although some issues remain

Starting from this release, work-in-progress WDL runner functionality is included in the code base & test suite; but it's still far from usable for general purposes.

v0.1.7

miniwdl

• improved implementation of miniwdl --version (#125 @jtratner)

WDL

• major internal refactoring of StdLib, prepares for runner implementation
• allow input/output sections anywhere in task body (#126 @mwalker174)
• fix resolution of struct types within compound-type struct members (#127 @patmagee)

API changes

• WDL.Expr.{infer_type,eval} and WDL.Tree.*.typecheck can take a specialized StdLib implementation with context-specific functions/types

v0.1.6

miniwdl check

• Fix suppression of UnusedDeclaration on dxWDL task stubs for 'native' applets (#120 @jtratner)

miniwdl cromwell

• Bump to Cromwell 40
• Add --version flag to display miniwdl & Cromwell versions (#121)

WDL

• Add JSON codec for all WDL.Value classes
• Add WDL.Error.RuntimeError

API changes

• rename WDL.StructType to WDL.StructTypeDef to reduce confusion with WDL.Type.StructInstance
• change internal representation of meta and parameter_meta values from string to string_literal (also fix parsing of escaped single-quotes in string_literal)

v0.1.5

miniwdl check

• The nonempty array quantifier Array[T]+ is no longer enforced statically; supplying an Array[T] where Array[T]+ is expected produces a linter warning, but not a validation error.
  • The --no-quant-check command-line option remains available, affecting static enforcement of the optional quantifier T? and implicit coercion of T to Array[T].
  • Outputs from scatters on zip(...) or range(length(...)) are inferred nonempty when possible (for linting purposes)
• Initial support for WDL 1.0 structs, including the old-style object literals (no general Object support)
• Several struct-related linter modules
• QuantityCoercion linter is split into OptionalCoercion and NonemptyCoercion
• NameCollision linter checks scatter variables
• SelectArray linter for questionable invocations of select_all and select_first
• Invalidates use of language keywords in names/identifiers (#74)
• Suppress UnusedDeclaration on dxWDL task stubs for 'native' applets (#109)

API changes

• struct-related additions: WDL.Tree.StructType, WDL.Type.StructInstance, WDL.Expr.Struct
• WDL.Document.imports changes to a NamedTuple as it gets more fields (namely struct aliases)
• {StructType,StructInstance}.type_id changes from an int to a string summarizing the member names & types (better to recognize equivalent struct types in "diamond import" scenarios)
• Add import_max_depth guardrail to WDL.load (#18)

v0.1.4

miniwdl cromwell

• can parse Boolean inputs from command-line

WDL

• Significant internal refactoring on the way to supporting the WDL 1.0 struct features. Struct type definitions and member accesses are supported, but not yet literals and imports.
• Tighten typechecking for select_first and select_all (reject optional inputs)

API changes

• New AST WDL.Expr.Get node used for identifier access, pair left/right access, and struct member accesses.
• Ident nodes are now wrapped in Get nodes (see docs for details)
• WDL.Tree.Call.callee_id is represented as List[str] instead of Ident.
• Internal function for array indexing is renamed from _get to _at.

meta

• Fix Pyre typechecker configuration; for the last few months it seemed to have been exciting successfully but not actually checking anything
• Makefile allows override of user-specific Python package directory (@nkashy1)

v0.1.3

miniwdl check

• suppress UnusedCall when call has no outputs
• improve CommandShellCheck column numbers accuracy

miniwdl cromwell

• after successful workflow execution, create an outputs/ subdirectory tree with symlinks to the output files (but more easily navigated than the cromwell-executions/ tree)

WDL

• simplify Pair parser/code, fixing access to nested pairs p.left.right (#12, #97)
• fix cycle detection on self-references
• refactor internal parsing of types, in preparation for structs

API changes

(none)