v0.2.7

Released on March 22, 2023.

Key Features

• [Beta] Aqueduct now has support for on-demand Kubernetes cluster creation and management on AWS. From the Aqueduct UI, you can connect Aqueduct to your AWS account via the cloud integration feature. Once connected, you can use this cloud integration to ask Aqueduct to automatically create a Kubernetes cluster for you. See the documentation here for how to create an operator that uses on-demand Kubernetes.

Enhancements

• Improves error handling to return more detailed error messages from errors occurring during execution.
• Improves error handling by surfacing errors that occur outside of the execution of an individual function as workflow-level errors; these errors could occur for example if a compute system was misconfigured.
• Improves handling of artifact name conflicts in the Python SDK. Explicitly named artifacts (using either the outputs argument to the @op decorator or the .set_name() function) will immediately flag and prevent conflicts in artifact names. Automatically named artifacts will error if multiple artifacts with the same name are included in a single publish_flow call.
• Displays all compute engines associated with a workflow on the workflows list page.
• Improves efficiency when previewing large objects on the UI by retrieving a sample of the data instead of the full data object and noting that the displayed data is a sample.

Bugfixes

• Fixes bug where an S3 or GCS bucket being used as the Aqueduct artifact store could possibly be deleted from the UI.
• Fixes bug that caused navigation buttons to be misaligned with other buttons on the action bar on the workflow details page.
• Fixes bug where navigating to the next most recent run on the workflow details page would not work correctly.
• Fixes bug pending or errored metric would show as "Unknown" on the UI instead of with the correct status.
• Fixes bug where warning-level checks were being shown as failures on the workflows list page.
• Fixes bug where certain DAG layouts would continue to show a layout with overlapping and crossing edges.

Note

• The parameterization of SQL queries may have unexpected behavior if you accidentally define a
  parameter with the same name twice. The parameter value will be chosen at random in such a case.
  This bug will be fixed in the next release.

All Changes

v0.2.6...v0.2.7

All commits

• ENG 2487: Fix bug where user is allowed to delete an integration that is being used as the storage layer by @eunice-chan in #1086
• Update README.md by @vsreekanti in #1096
• Deprecate FlowConfig in SDK by @jpurusho65 in #1094
• [ENG-2631] Fixes bug that caused navigation buttons to be misaligned by @vsreekanti in #1099
• Change error surfacing to also return inner error messages by @cw75 in #1102
• [ENG-2652] Remove dead code from UI by @vsreekanti in #1104
• Use context.Background() for deferred functions by @hsubbaraj-spiral in #1101
• Add a workflow-level error banner on the UI for errors that occur outside the workflow's context by @kenxu95 in #1107
• Allow creating multiple AWS integrations by @cw75 in #1092
• ENG-2339 Empty String Parameter Rendering Fix by @agiron123 in #1115
• Fix bug where AWS region is hardcoded when configuring kube config and terraform by @cw75 in #1111
• [ENG-2645] Fixes bug where next run button was broken on the UI by @vsreekanti in #1118
• Update the naming scheme for artifacts in our SDK by @kenxu95 in #1093
• ENG 2458 give all history rows a timestamp by @eunice-chan in #1100
• Add automated periodic integration test coverage for SQLite by @kenxu95 in #1119
• Upload the tfstate file at the end of periodic tests by @kenxu95 in #1109
• ENG-2577 Fix bug where warning state is not properly reflected on metadata list views by @agiron123 in #1117
• Fix bug where all errors were surfaced as workflow-level errors by @kenxu95 in #1116
• ENG-2620 DAG Line Crossing Fix by @agiron123 in #1113
• Fix a bug where engine icon is not properly shown in list workflow page by @likawind in #1110
• Improve UI efficiency on previewing large data by @likawind in #1103
• Minor updates to dynamic integration docstring and error message by @cw75 in #1120
• [ENG-2656, WIP] Adds workflow history route, store by @vsreekanti in #1108
• Upload more state and retry when terraform destroy fails by @kenxu95 in #1124
• ENG-2577 follow up by @agiron123 in #1125
• Update automated terraform destroy by @kenxu95 in #1127
• Release 0.2.7 by @cw75 in #1126

New Contributors

• @jpurusho65 made their first contribution in #1094