CHANGELOG.rst

Changelog

Next

Version 0.3.4 - 2025-02-05

• The sync native command with the --split and/or --continue-on-error flags now show more information with --loglevel=DEBBUG (#342).
• The sync native command now always update the DB password when provided (#343).
• The --continue-on-error flag for the sync native command now also handle issues with asset dependencies (#344).

Version 0.3.3 - 2025-01-29

• It's now possible to specify the DB connection (using either its ID or display name) that should be used for the dbt Cloud sync (#339).
• It's now possible to pass DB passwords in the sync native/import-assets command using the --db-password argument (#340).

Version 0.3.2 - 2025-01-22

• The --asset-type selector is now supported with the --split / --continue-on-error flags (#337).
• Fixed a bug affecing the logic to skip assets based on the progress.log file (#337).

Version 0.3.1 - 2025-01-14

• Partial/aborted executions for non-atomic commands now also populate the progress.log file (#335).

Version 0.3.0 - 2025-01-07

BREAKING CHANGES:

• The logfile that stores progress for non-atomic commands was re-named from checkpoint.log to progress.log. Its structure is also different (it's now a YAML file including additional data for each asset depending on the command). Before upgrading the CLI, make sure you have completed any pending import that was logged in the old file.
• The --preserve-columns flag for the dbt sync commands has been removed, as it was marked for deprecation in favor of --preserve-metadata.

---

• Added support for importing a dashboard that contains a divider using the --split flag (#321).
• It's now possible to define default values for metric and columns configurations in the dbt sync (#325).
• The dbt Cloud sync now uses v3 endpoints and supports None on the custom_branch_only field (#330).
• Added a new --continue-on-error flag for the import-assets command so that a failure in one asset doesn't stop the import process (#328).
• Added a new --continue-on-error flag for the import-ownership command so that a failure in one asset doesn't stop the import process. This command now also skips assets that failed to import (#331).
• It's now possible to sync calculated columns from dbt (#332).

Version 0.2.21 - 2024-10-16

• It's now possible to set export commands to generate YAML files with Unix EOL (#316).
• Calls to the Preset Manager API are now paginated (#320).
• It's now possible to trigger an import that creates missing dependencies but don't overwrite them (#321).

Version 0.2.20 - 2024-08-20

• It's now possible to link dbt metrics with Preset metrics that don't have the same name (#311).
• The dbt sync continues in case syncing columns for a dataset fails (#313).
• dbt Cloud versionless jobs are now supported (#314).

Version 0.2.19 - 2024-07-04

• It's now possible to bypass MetricFlow when syncing metrics from dbt Core (#305).
• Migrated the GraphQL query to fetch metrics from dbt Cloud to support jobs with an ID exceeding 32 bits (#306).

Version 0.2.18 - 2024-06-21

• It's now possible to sync a dbt Cloud project that doesn't have a semantic layer (#301).
• The dbt Core sync now supports syncing legacy metrics from dialects that are not supported by MetricFlow (#302).
• The dbt Cloud sync API endpoints were updated to support custom base domains (#303).

Version 0.2.17 - 2024-06-03

• The dbt sync now supports creating physical datasets in other catalogs (requires catalog Support in Superset) (#295 and #297).
• The dbt ModelSchema now supports models with columns set to None (#298).
• It's now deprecated to trigger a dbt Core sync passing a dbt_project.yml file (support for passing the manifest.json file is no longer deprecated) (#299).

Version 0.2.16 - 2024-05-10

• Changed the dbt sync logic so that metrics that are not associated with the models being synced are ignored (#289).
• The dbt sync now indicates if the snowflake SQLAlchemy package is missing in the environment (#290).
• The dbt sync now removes Redshift-specific metatada from columns when refreshing a dataset (#291).
• Datasets used in dashboard filters are now included when importing assets individually with the --split flag (#292).

Version 0.2.15 - 2024-04-22

• The profiles.yml content is now rendered so that Jinja variables are handled properly (#280).
• Added an upper bound limit to the sqlglot version that gets installed to avoid compatibility issues (#283).
• The sync native command now tries to retrieve the DB connection uuid through the API first, to avoid exporting assets if not necessary (#284).
• Added support for syncing derived metrics that rely on other derived metrics containing Superset-specific Jinja syntax (#285).

Version 0.2.14 - 2024-04-10

• Fixed an issue when syncing columns for datasets powered by BigQuery (#278).
• Added support for syncing derived metrics that don't rely on other metrics, and also metrics including Superset-Jinja specific syntax (#277).

Version 0.2.13 - 2024-03-25

• Improved metric parsing with sqlglot (#273 and #274).
• Fixed the dataset creation flow for the dbt sync (#275).

Version 0.2.12 - 2024-03-19

• Support for including the account and project IDs with the dbt Cloud command (#264).
• Support MetricFlow/new dbt Semantic Layer for the dbt Core sync (#265).
• New --raise-failures flag added to the dbt sync commands to end the execution with an error in case any model failed to sync (#266).
• Syncing from dbt with the --preserve-metadata / --merge-metadata flags now sync the dataset columns (#268).
• Derived metrics for older dbt versions are now syncing properly (#270).

Version 0.2.11 - 2024-02-14

• Support for custom access URLs when connecting to dbt Cloud APIs (#262).

Version 0.2.10 - 2024-01-10

• Small improvements to the dbt sync logic (#258 and #259).

Version 0.2.9 - 2024-01-10

• Initial support for syncing metrics from dbt/MetricFlow (#256).

Version 0.2.8 - 2023-09-12

• The Jinja rendering/escaping logic for content migration was improved (#237).
• It's now possible to specify dbt models to be synced using the file name/path (#242).

Version 0.2.7 - 2023-09-08

• The CLI now has a re-try mechanism to address Session-related errors (#235).
• It's now possible to trigger a dbt sync and merge dbt metadata with Preset metadata (#238).

Version 0.2.6 - 2023-08-17

• The dbt sync now uses Superset updated endpoints to properly create a virtual dataset (#232).
• It's now possible to authenticate to Superset instances that require a CSRF token (#233).

Version 0.2.5 - 2023-07-26

• Further adjustments to dbt marshmallow schemas to avoid integration errors (#229).

Version 0.2.4 - 2023-07-20

• Further adjustments to dbt marshmallow schemas to avoid integration errors (#228).
• Export RLS rules is now compatible with Preset Cloud and older Superset installations (#227)

Version 0.2.3 - 2023-07-14

• Adjustments to dbt marshmallow schemas to avoid integration errors (#225).

Version 0.2.2 - 2023-07-05

• certification and additional extra information is now synced from dbt models (#213 and #215).
• Improved the exposures sync (#221).
• The --preserve-columns flag can now be used to preserve groupby and filterable values for existing columns during a dbt sync (#221).
• The search for roles during the sync roles command now uses Equals comparison, instead of Starts with (#222).

Version 0.2.1 - 2023-05-30

• Fix for apache/superset#24067 (#211).

Version 0.2.0 - 2023-05-23

• Column descriptions and labels are now synced from dbt models (#72, #111, #195 and #197).
• CLI can now provision users directly to the team (doesn't require accepting the invitation) with the import-users command. The export-users command can now also be used with Preset Workspaces. (#74, #100 and #148).
• It's possible to export roles information using the export-roles command (#75 and #161).
• Exported roles information can be imported via the import-roles command (#76, #167 and #179).
• Improved session object logic (#77).
• Improved export/import logic for owernship and role information (#79).
• CLI can now add users to imported roles (#81).
• A JWT token can now be passed for authentication (#82).
• Added debug logging to API requests (#83 <https://github.com/preset-io/backend-sdk/pull/83>_).
• CLI can now export specific asset types, using the --asset-type flag (#84).
• CLI can now export specific assets only, using the --$asset_type-ids (for example --dashboard-ids) flag (#85 and #88).
• CLI can now authenticate to Superset (On Premises) without CSRF token (#87).
• Workspace/Team prompt no longer happens in case --help was pased (#89).
• Team Roles, Workspace Roles and DARs can now be synced to a Preset team based on a YAML file (#90).
• Added --version command to display the installed version (#91).
• Fixed parent/child node selection in dbt Core for proper graph selection (#92).
• Improved logging for the dbt Client (#94).
• CLI now can create datasets for different databases (for DB Engines that supports multiple databases like Snowflake, BigQuery, etc) (#95).
• BQ connection can now successfully be created/updated from the profiles.yml information (#96).
• Redshift connectons now get created with the redshift+psycopg2 driver (#97).
• YAML files outside of asset folders aren't imported in the native sync (#99).
• Improved BQ DB detection (#102).
• Reduced the maximum amount of files included in an export file (#105).
• Workspaces can now be defined as environment variables (#106).
• CLI can now create Snowflake connections authenticated via private key pair (#108).
• Improved the --exclude filter for the dbt sync (#109).
• Improved database connection logic (#111).
• CLI can now create Snowflake connections authenticated with DUO MFA (#112).
• dbt target definition now defaults to the profile.yml if not specified (#114).
• The dbt sync can now be triggered using the dbt_project.yml file rather than the manifest.json (#115).
• CLI now supports None as column type (#116).
• Database connection is now tested before triggering the import (#118).
• Added support for companion YAML templates (#120).
• YAML rendering logic is now improved (#121 and #205).
• DB connection password is no longer logged in case the connection fails (#122).
• Import assets is now performed through the assets endpoint (#124).
• Large imports can be performed with the --split flag to prevent timeouts (#124). It also creates a checkpoint in case it fails so the retry would ignore already imported assets (#137 and #139).
• Preset Manager requests updated to use api.app.preset.io (#127).
• CLI now prompts user for job information if not specified when triggering a sync from dbt Cloud (#128).
• dbt exposures now includes assets that were created by manual datasets, based on the schema and table name (#132).
• Added support for Python 3.11 (#133).
• CLI now refreshes JWT token if needed (#134).
• Import failures due to connection errors are automatically retried (#135).
• Improved Get Resources logic (#136).
• CLI no longer prompts user to enter the DB password in case the connection already exists (#140).
• It's now possible to trigger a sync only for exposures back to dbt, using the --exposures-only flag (#142).
• CLI can be used to list SCIM groups and membership with the list-group-membership command (#143).
• The dbt profile name is now used to look for an existing DB connection in the Workspace, instead of the project name (#151).
• Added support for dbt derived metrics (#154, #160, #196, #198 and #199).
• Fixed column configuration issues after a dbt sync (#156 and #165).
• Added support for dbt 1.3 (#159).
• Improved the MetricSchema loading (#159).
• Added support for Secondary Contributor Workspace Role (#186).
• Use model table alias for dataset creation (#192).
• The dbt sync now only updates the DB connection in case --import-db is passed. It's also possible to trigger a sync without this flag (#193 and #200).
• Added support for specifying a certification payload for dbt syncs (#203).
• dbt models can now be filtered using config options (#204).
• It's now possible to disable Jinja syntax escaping during export, and Jinja syntax rendering during import (#205).

Version 0.1.1 - 2022-09-13

• File path is now passed to template as filepath in the sync native command.
• CLI can now invite users to Preset from a YAML file created by export-users.
• Fix database update in the dbt sync.

Version 0.1.0 - 2022-09-09

• Initial release.