Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(dagster): Update dagster (patch) #131

Merged
merged 1 commit into from
Jul 14, 2023
Merged

Conversation

plural-renovate[bot]
Copy link
Contributor

This PR contains the following updates:

Package Type Update Change
dagster-azure (source) patch ==0.19.13 -> ==0.19.14
dagster-gcp (source) patch ==0.19.13 -> ==0.19.14
docker.io/dagster/user-code-example final patch 1.3.13 -> 1.3.14

Release Notes

dagster-io/dagster (dagster-azure)

v0.19.14

New
  • DynamicPartitionsDefinition and SensorResult are no longer marked experimental
  • DagsterInstance now has a get_status_by_partition method, which returns the status of each partition for a given asset. Thanks renzhe-brian!
  • DagsterInstance now has a get_latest_materialization_code_versions method, which returns the code version of the latest materialization for each of the provided (non-partitioned) assets.
  • The error message for when an asset illegally depends on itself is now more informative.
  • Further performance improvements for the Asset Daemon.
  • Performance improvements in the asset graph view for large asset graphs.
  • Pandas 2.x is now supported in all dagster packages.
  • build_asset_context has been added as an asset focused replacement for build_op_context.
  • build_op_context now accepts a partition_key_range parameter.
  • New AssetSelection.upstream_source_assets method allows selecting source assets upstream of the current selection.
  • AssetSelection.key_prefixes and AssetSelection.groups now accept an optional include_sources parameter.
  • The AutoMaterialize evaluations UI now provides more details about partitions and waiting on upstream assets.
  • [dbt] The DbtCli resource is no longer marked experimental.
  • [dbt] The global_config parameter of the DbtCli resource has been renamed to global_config_flags
  • [dbt] load_assets_from_dbt_project and load_assets_from_dbt_manifest now work with the DbtCli resource.
  • [dbt] The manifest argument of the @dbt_assets decorator now additionally can accept a Path argument representing a path to the manifest file or dictionary argument representing the raw manifest blob.
  • [dbt] When invoking DbtCli.cli from inside a @dbt_assets-decorated function, you no longer need to supply the manifest argument as long as you provide the context argument.
  • [dbt] The DbtManifest object can now generate schedules using dbt selection syntax.
dbt_manifest.build_schedule(
      job_name="materialize_dbt_models",
      cron_schedule="0 0 * * *",
      dbt_select="fqn:*"
  )
  • [dbt] When invoking DbtCli.cli and the underlying command fails, an exception will now be raised. To suppress the exception, run the DbtCli.cli(..., raise_on_error=False).
  • [ui] You can now alphabetically sort your partitions on the asset partitions page
  • [ui] A button in the “Run is materializing this asset” and “Run failed to materialize this asset” banners provides direct access to the relevant run logs
Bugfixes
  • Fixed a bug that caused asset metadata to not be available available on the OutputContext when using with_attributes or AssetsDefinition.from_graph.
  • Previously, if a partitioned asset at the root of the graph had more missing partitions than its AutoMaterializePolicy’s max_materializations_per_minute parameter, those older partitions would not be properly discarded from consideration on subsequent ticks. This has been fixed.
  • Fixed a bug that caused AutoMaterializePolicy.lazy() to not materialize missing assets that were downstream of assets without an AutoMaterializePolicy.
  • In rare cases, the AssetDaemon could hit an exception when using a combination of freshness policies and observable source assets. This has been fixed.
  • Previously, string type annotations (most commonly via modules containing from __future__ import annotations) would cause errors in most cases when used with Dagster definitions. This has been fixed for the vast majority of cases.
  • AssetExecutionContext has returned to being a type alias for OpExecutionContext.
  • [ui] Date filtering on the runs page now takes your timezone into consideration
  • [ui] Fixed a bug where selecting partitions in the launchpad dialog cleared out your configuration
  • [ui] In the run Gantt chart, executed steps that follow skipped steps no longer render off the far right of the visualization.
  • [ui] Cancelling a running backfill no longer makes canceled partitions un-selectable on the job partitions page and backfill modal, and cancellation is shown in gray instead of red.
Breaking Changes
  • [experimental] The internal time_window_partition_scope_minutes parameter of the AutoMaterializePolicy class has been removed. Instead, max_materializations_per_minute should be used to limit the number of runs that may be kicked off for a partitioned asset.
Deprecations
  • [dbt] DbtCliResource has been deprecated in favor of DbtCli.
  • The python package dagit has been deprecated in favor of a new package dagster-webserver.
  • OpExecutionContext.asset_partition_key_range has been deprecated in favor of partition_key_range.
Community Contributions
  • The databricks_pyspark_step_launcher will no longer error when executing steps that target a single partition of a DynamicPartitionsDefinition (thanks @​weberdavid!).
  • Increased timeout on readinessProbe for example user code images, which prevents breakages in certain scenarios (thanks @​leehuwuj)!
  • Avoid creation of erroneous local directories by GCS IO manager (thanks @​peterjclaw)!
  • Fixed typo in intro docs (thanks @​adeboyed)!
  • Fix typo in bigquery docs (thanks @​nigelainscoe)!
  • Fix typing on run tag validation (thanks @​yuvalgimmunai)!
  • Allow passing repositoryCredentials arn as config to ecs run launcher (thanks @​armandobelardo)!
Experimental
  • The @observable_source_asset decorator now accepts an auto_observe_interval_minutes parameter. If the asset daemon is turned on, then the observation function will automatically be run at this interval.
  • [dbt] DbtCliTask has been renamed to DbtCliInvocation
  • [dbt] The get_asset_key_by_output_name and get_node_info_by_output_name methods of DbtManifest have been renamed toget_asset_key_for_output_name and get_node_info_for_output_name, respectively.
  • [ui] A new feature flag allows you to switch Asset DAG rendering to a tighter horizontal layout, which may be preferable in some scenarios
Documentation
  • Many public methods that were missing in the API docs are now documented. Updated classes include DagsterInstance, *MetadataValue, DagsterType, and others.
  • dagster-pandera now has an API docs page.
  • Deprecated methods in the API docs now are marked with a special badge.

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.


  • If you want to rebase/retry this PR, check this box

Signed-off-by: plural-renovate[bot] <plural-renovate[bot]@users.noreply.github.com>
@plural-renovate plural-renovate bot enabled auto-merge (squash) July 14, 2023 22:03
@@ -1,4 +1,4 @@
FROM docker.io/dagster/user-code-example:1.3.13
FROM docker.io/dagster/user-code-example:1.3.14

Check failure

Code scanning / Trivy

Image user should not be &#39;root&#39; High

Artifact: dagster/Dockerfile
Type: dockerfile
Vulnerability DS002
Severity: HIGH
Message: Specify at least 1 USER command in Dockerfile with non-root user as argument
Link: DS002
@@ -1,4 +1,4 @@
FROM docker.io/dagster/user-code-example:1.3.13
FROM docker.io/dagster/user-code-example:1.3.14

Check notice

Code scanning / Trivy

No HEALTHCHECK defined Low

Artifact: dagster/Dockerfile
Type: dockerfile
Vulnerability DS026
Severity: LOW
Message: Add HEALTHCHECK instruction in your Dockerfile
Link: DS026
@plural-renovate plural-renovate bot merged commit ced4a2e into main Jul 14, 2023
2 of 3 checks passed
@plural-renovate plural-renovate bot deleted the renovate/patch-dagster branch July 14, 2023 22:06
@plural-bot
Copy link
Collaborator

🎉 This PR is included in version 1.0.10 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant