Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(dagster): Update dagster (patch) #348

Merged
merged 1 commit into from
Nov 30, 2023
Merged

Conversation

plural-renovate[bot]
Copy link
Contributor

This PR contains the following updates:

Package Type Update Change
dagster-azure (source) patch ==0.21.9 -> ==0.21.10
dagster-gcp (source) patch ==0.21.9 -> ==0.21.10
docker.io/dagster/user-code-example final patch 1.5.9 -> 1.5.10

Release Notes

dagster-io/dagster (dagster-azure)

v0.21.10

New
  • Added a new MetadataValue.job metadata type, which can be used to link to a Dagster job from other objects in the UI.
  • [asset backfills] Previously, when partitions definitions were changed after backfill launch, the asset backfill page would be blank. Now, when partitions definitions are changed, the backfill page will display statuses by asset.
  • [dagster-bigquery, dagster-duckdb, dagster-snowflake]. The BigQuery, DuckDB, and Snowflake I/O Managers will now determine the schema (dataset for BigQuery) in the following order of precedence: schema metadata set on the asset or op, I/O manager schema/ dataset configuration, key_prefix set on the asset. Previously, all methods for setting the schema/dataset were mutually exclusive, and setting more than one would raise an exception.
  • [dagster-shell] Added option to exclude the shell command from logs.
  • [dagster-dbt] When running DAGSTER_DBT_PARSE_PROJECT_ON_LOAD=1 dagster dev in a new scaffolded project from dagster-dbt project scaffold, dbt artifacts for loading the project are now created in a static target/ directory.
Bugfixes
  • Problematic inheritance that was causing pydantic warnings to be emitted has been corrected.
  • It's now possible to use the logger of ScheduleEvaluationContext when testing via build_schedule_context.
  • The metadata from a Failure exception is now hoisted up to the failure that culminates when retry limits are exceeded.
  • Fixed bug in which the second instance of an hour partition at a DST boundary would never be shown as “materialized” in certain UI views.
  • Fixed an issue where backfilling an hourly partition that occurred during a fall Daylight Savings Time transition sometimes raised an error.
  • [auto-materialize] Fix issue where assets which were skipped because required parent partitions did not exist would not be materialized once those partitions came into existence.
  • [dagster ecs] The exit code of failed containers is now included in the failure message.
  • [dagster pipes] The PipesK8sClient now correctly raises on failed containers.
  • [dagster pipes] Using pipes within ops instead of assets no longer enforces problematic constraints.
  • [helm] Added maxCatchupRuns and maxTickRetries configuration options for the scheduler in the Helm chart.
  • [embedded-elt] Fixed crashes for non-unicode logs.
  • [UI] Fixed an issue where the test sensor dialog for a sensor that targeted multiple jobs would claim that all of the runs were targeting the same job.
  • [UI] Asset keys, job names, and other strings in Dagster UI no longer truncate unnecessarily in Firefox in some scenarios
  • [UI] A larger “View prior events” button on the Asset > Partitions page makes it easier to see the historical materializations of a specific partition of an asset.
  • [asset-checks, dbt] Fixed a bug that that caused asset checks to not execute when a run was not a subset. As part of the fix, the default dbt selection selection string will not be used for dbt runs, even when not in a subset. Instead we pass the explicit set of models and tests to execute, with DBT_INDIRECT_SELECTION=empty.
  • [asset-checks] Fixed a bug that caused asset checks defined with @asset(check_specs=... to not cooperate with the key_prefix argument of the load_assets_from_modules method and it’s compatriots.
  • [asset-checks] Fixed a bug that caused errors when launching a job from the UI that excluded asset checks.
  • [asset-checks] Fixed a bug that caused UI errors when a check run was deleted.
Deprecations
  • Marked the experimental Airbyte ingestion-as-code feature as deprecated, to be removed in a future release. We suggest users interested in managing their Airbyte connections in code use the Airbyte terraform provider.
Community Contributions
  • define_asset_job now accepts an op_retry_policy argument, which specifies a default retry policies for all of the ops in the job. (thanks Eugenio Contreras!)
  • Fix IOManager not being able to load assets with MultiPartitionsDefinition - thanks @​cyberosa!
  • [dagster-essentials] Three typo fixes in Lesson 8 - thanks Colton @​cmpadden!
Experimental
  • The observable_source_asset decorator now accepts a key argument.
  • [dagster pipes] an implicit_materializations argument has been added to get_results and get_materialize_result to control whether an implicit materialization event is created or not.
  • [embedded-elt] Added a new builder and SlingConnectionResource to allow reusing sources and targets interoperably.
  • [UI] Updated the experimental concurrency limits configuration page to show per-op runtime info and control.
  • [UI] The Auto-materialize history tab for each asset now only includes rows for evaluations where the result of evaluating the policy has changed. Previously, it would also show a row in the table representing periods of time where nothing changed.
  • [asset-checks, dbt] build_dbt_asset_selection now also selects asset checks based on their underlying dbt tests. E.g. build_dbt_asset_selection([my_dbt_assets], dbt_select="tag:data_quality") will select the assets and checks for any models and tests tagged with ‘data_quality’.
Documentation
Dagster Cloud
  • Branch deployments now use the same timeouts for starting and canceling runs that are set for their parent full deployment, instead of a fixed value of 10 minutes.
  • [k8s agent] Setting labels on a code location will now apply those labels to the kubernetes deployment and service for that code location, rather than just applying them to the pod for that code location.

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.


  • If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Signed-off-by: plural-renovate[bot]@users.noreply.github.com
@plural-renovate plural-renovate bot enabled auto-merge (squash) November 30, 2023 23:39
@@ -1,4 +1,4 @@
FROM docker.io/dagster/user-code-example:1.5.9
FROM docker.io/dagster/user-code-example:1.5.10

Check failure

Code scanning / Trivy

Image user should not be 'root' High

Artifact: dagster/Dockerfile
Type: dockerfile
Vulnerability DS002
Severity: HIGH
Message: Specify at least 1 USER command in Dockerfile with non-root user as argument
Link: DS002
@@ -1,4 +1,4 @@
FROM docker.io/dagster/user-code-example:1.5.9
FROM docker.io/dagster/user-code-example:1.5.10

Check notice

Code scanning / Trivy

No HEALTHCHECK defined Low

Artifact: dagster/Dockerfile
Type: dockerfile
Vulnerability DS026
Severity: LOW
Message: Add HEALTHCHECK instruction in your Dockerfile
Link: DS026
@plural-renovate plural-renovate bot merged commit 2ba0bf6 into main Nov 30, 2023
2 of 3 checks passed
@plural-bot
Copy link
Collaborator

🎉 This PR is included in version 1.2.10 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@plural-renovate plural-renovate bot deleted the renovate/patch-dagster branch December 1, 2023 01:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant