Handle changes from Databricks Python SDK 0.37.0 #320
Merged
+33
−30
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Nov 15, 2024
|
❌ 33/36 passed, 3 failed, 4 skipped, 4m41s total ❌ test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import NotFound\nbackend = RuntimeBackend()\ntry:\n query_response = backend.fetch("SELECT * FROM TEST_SCHEMA.__RANDOM__")\n return "FAILED"\nexcept NotFound as e:\n return "PASSED"\n]: assert '{"ts": "2024...]}}\n"PASSED"' == 'PASSED' (20.956s)❌ test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import NotFound\nbackend = RuntimeBackend()\ntry:\n backend.execute("SELECT * FROM TEST_SCHEMA.__RANDOM__")\n return "FAILED"\nexcept NotFound as e:\n return "PASSED"\n]: assert '{"ts": "2024...]}}\n"PASSED"' == 'PASSED' (21.482s)❌ test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import NotFound\nbackend = RuntimeBackend()\ntry:\n query_response = backend.fetch("DESCRIBE __RANDOM__")\n return "FAILED"\nexcept NotFound as e:\n return "PASSED"\n]: assert '{"ts": "2024...]}}\n"PASSED"' == 'PASSED' (19.868s)Running from acceptance #450 |
github-merge-queue bot
pushed a commit
to databrickslabs/ucx
that referenced
this pull request
Nov 18, 2024
Pin Databricks SDK to minor version to circumvent the problems with the [0.37.0 release](https://github.com/databricks/databricks-sdk-py/releases/tag/v0.37.0). Will unpin later when we resolved that issues. Waiting on solution in: - [ ] [lsql](databrickslabs/lsql#320) - [ ] [Databricks python sdk](databricks/databricks-sdk-py#824)
github-merge-queue bot
pushed a commit
to databrickslabs/ucx
that referenced
this pull request
Nov 18, 2024
Pin Databricks SDK to minor version to circumvent the problems with the [0.37.0 release](https://github.com/databricks/databricks-sdk-py/releases/tag/v0.37.0). Will unpin later when we resolved that issues. Waiting on solution in: - [ ] [lsql](databrickslabs/lsql#320) - [ ] [Databricks python sdk](databricks/databricks-sdk-py#824)
|
@nfx, @JCZuurmond : Should this PR have included a change to |
|
Is within range already: "databricks-sdk>=0.29.0", |
This was referenced Nov 18, 2024
JCZuurmond
added a commit
that referenced
this pull request
Nov 19, 2024
* Changes to work with Databricks SDK `v0.38.0` ([#350](#350)). In this release, the open-source library has been updated to be compatible with Databricks SDK version 0.38.0, addressing issues [#349](#349) to [#332](#332). The changes include modifying the `create_dashboard` function, which now directly passes the `SDKDashboard` instance to the `ws.lakeview.create` and `ws.lakeview.update` methods, eliminating the need for dictionary conversions. A new SQL query, `NYC_TAXI_TRIPS_LIMITED`, has been introduced, and the `test_sql_execution` method has been updated to use this query. The `test_sql_execution_partial` and `test_sql_execution_as_iterator` methods have been removed, and the `test_fetch_one_works` method now includes an assertion to verify the returned row's `pickup_zip` value. These updates improve the library's compatibility with the latest Databricks SDK, refactor SQL query-based tests, and enhance test reliability. * Specify the minimum required version of `databricks-sdk` as 0.37.0 ([#331](#331)). In this release, we have updated the minimum required version of the `databricks-sdk` package to 0.37.0, as specified in the project's `pyproject.toml` file. This update is necessary due to the modifications made in pull request [#320](#320), which constrains the `databricks-sdk` version to 0.37.x for compatible updates within the same minor version. This change also resolves issue [#330](#330). It is important to note that no new methods have been added or existing functionality changed in the codebase as part of this commit. Therefore, the impact on the existing functionality should be minimal and confined to the interaction with the `databricks-sdk` package.
Closed
JCZuurmond
added a commit
that referenced
this pull request
Nov 19, 2024
* Changes to work with Databricks SDK `v0.38.0` ([#350](#350)). In this release, we have upgraded the Databricks SDK to version 0.38.0 from version 0.37.0 to ensure compatibility with the latest SDK and address several issues. The update includes changes to make the code compatible with the new SDK version, removing the need for `.as_dict()` method calls when creating or updating dashboards and utilizing a `sdk_dashboard` variable for interacting with the Databricks workspace. We also updated the dependencies to "databricks-labs-blueprint[yaml]" package version greater than or equal to 0.4.2 and `sqlglot` package version greater than or equal to 22.3.1. The `test_core.py` file has been updated to address multiple issues ([#349](#349) to [#332](#332)) related to the Databricks SDK and the `test_dashboards.py` file has been revised to work with the new SDK version. These changes improve integration with Databricks' lakeview dashboards, simplify the code, and ensure compatibility with the latest SDK version, resolving issues [#349](#349) to [#332](#332). * Specify the minimum required version of `databricks-sdk` as 0.37.0 ([#331](#331)). In this release, we have updated the minimum required version of the `databricks-sdk` package to 0.37.0 from 0.29.0 in the `pyproject.toml` file to ensure compatibility with the latest version. This change was made necessary due to updates made in issue [#320](#320). To accommodate any patch release of `databricks-sdk` with a major and minor version of 0.37, we have updated the dependency constraint to use the `~=` operator, resolving issue [#330](#330). These changes are intended to enhance the compatibility and stability of our software.
Merged
JCZuurmond
added a commit
that referenced
this pull request
Nov 19, 2024
* Changes to work with Databricks SDK `v0.38.0` ([#350](#350)). In this release, we have upgraded the Databricks SDK to version 0.38.0 from version 0.37.0 to ensure compatibility with the latest SDK and address several issues. The update includes changes to make the code compatible with the new SDK version, removing the need for `.as_dict()` method calls when creating or updating dashboards and utilizing a `sdk_dashboard` variable for interacting with the Databricks workspace. We also updated the dependencies to "databricks-labs-blueprint[yaml]" package version greater than or equal to 0.4.2 and `sqlglot` package version greater than or equal to 22.3.1. The `test_core.py` file has been updated to address multiple issues ([#349](#349) to [#332](#332)) related to the Databricks SDK and the `test_dashboards.py` file has been revised to work with the new SDK version. These changes improve integration with Databricks' lakeview dashboards, simplify the code, and ensure compatibility with the latest SDK version, resolving issues [#349](#349) to [#332](#332). * Specify the minimum required version of `databricks-sdk` as 0.37.0 ([#331](#331)). In this release, we have updated the minimum required version of the `databricks-sdk` package to 0.37.0 from 0.29.0 in the `pyproject.toml` file to ensure compatibility with the latest version. This change was made necessary due to updates made in issue [#320](#320). To accommodate any patch release of `databricks-sdk` with a major and minor version of 0.37, we have updated the dependency constraint to use the `~=` operator, resolving issue [#330](#330). These changes are intended to enhance the compatibility and stability of our software.
gueniai
added a commit
that referenced
this pull request
Nov 19, 2024
* Changes to work with Databricks SDK `v0.38.0` ([#350](#350)). In this release, we have updated the Databricks SDK from version 0.37 to 0.38 to ensure compatibility with the latest SDK version. This update includes modifications to the `test_dashboards.py` file, where the `create_dashboard` function has been updated to use the new `sdk_dashboard` object directly, eliminating the need for dictionary conversion using the `as_dict()` method. This change has been implemented in both the `ws.lakeview.create` and `ws.lakeview.update` methods. These updates address several issues, as listed in the commit message, which were related to compatibility and functionality with the previous SDK version. Additionally, new methods such as `test_sql_execution`, `test_sql_execution_as_iterator`, and an updated `test_fetch_one_works` method have been introduced to improve compatibility with the Databricks SDK version 0.38.0. These changes are essential for adopters of the project who wish to ensure compatibility with the latest version of the Databricks SDK. * Release v0.14.1 ([#352](#352)). 0.14.1 release brings updates for compatibility with Databricks SDK v0.38.0, addressing several issues and improving code compatibility with the new SDK version. It removes the need for `.as_dict()` method calls when creating or updating dashboards, introduces a `sdk_dashboard` variable for interacting with the Databricks workspace, and updates dependencies to include the "databricks-labs-blueprint[yaml]" package version 0.4.2 or greater and `sqlglot` package version 22.3.1 or greater. Test files have been revised to address multiple issues related to the Databricks SDK, and the minimum required version of `databricks-sdk` has been updated to 0.37.0 from 0.29.0. These changes improve integration with Databricks' lakeview dashboards, simplify the code, and enhance compatibility and stability of the software. * Specify the minimum required version of `databricks-sdk` as 0.37.0 ([#331](#331)). In this release, we have updated the minimum required version of the `databricks-sdk` library to a version compatible with 0.37.0. This update is necessary due to changes introduced in pull request [#320](#320). We now specify the `databricks-sdk` version using the "~=" operator, indicating that any version with a major version number of 0 and minor version number of 37 is acceptable. This allows for future updates while ensuring compatibility with the required features. This change enhances the reliability and compatibility of the project by ensuring that it utilizes a recent and supported version of the `databricks-sdk`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Handle changes from Databricks Python SDK 0.37.0: LakeviewAPI now works with a Dashboard object
Requires change in Python SDK--> Implemented workaround, see this issue about removing the workaround when the SDK resolved the LakeviewAPI deployment issues