-
Notifications
You must be signed in to change notification settings - Fork 22
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
## Changes What's new in the 2.5 --------- Co-authored-by: Julia Crawford (Databricks) <[email protected]>
- Loading branch information
1 parent
e904f71
commit 392bd0a
Showing
8 changed files
with
67 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
<img src="./1.2/databricks-logo.png" alt="databricks-logo" width="100"/> | ||
|
||
# Databricks Extension v2.5.0 | ||
|
||
## What's new? | ||
|
||
- [Pipeline validation and partial update actions](#validation) | ||
- [Pipeline events and diagnostics](#events) | ||
- [Pipeline datasets](#datasets) | ||
- [Quick run action](#quick-run) | ||
|
||
### <a id="validation"></a> Pipeline validation and partial update actions | ||
|
||
In previous versions, you could only trigger a [Refresh all](https://docs.databricks.com/en/delta-live-tables/updates.html#how-dlt-updates) pipeline update. Now you can do validation and partial updates: | ||
|
||
<img src="./2.5/validation.png" alt="validation action" width="600"/> | ||
|
||
The partial update action preloads datasets from past runs, and lets you enter table names or select a full refresh: | ||
|
||
<img src="./2.5/partial-update.png" alt="partial update action" width="600"/> | ||
|
||
### <a id="events"></a> Pipeline events and diagnostics | ||
|
||
During and after a pipeline update, the extension now shows events in the resource explorer UI: | ||
|
||
<img src="./2.5/events.png" alt="pipeline events" width="600"/> | ||
|
||
Any warnings or errors that have source locations are also displayed in the Visual Studio Code **PROBLEMS** panel and highlighted in the source files: | ||
|
||
<img src="./2.5/diagnostics.gif" alt="pipeline diagnostics" width="600"/> | ||
|
||
### <a id="datasets"></a> Pipeline datasets | ||
|
||
You can now see pipeline datasets and their schemas in the extension resource explorer: | ||
|
||
<img src="./2.5/datasets.png" alt="pipeline datasets" width="600"/> | ||
|
||
Datasets are gathered from the events of past runs, so make sure to update or validate your pipeline at least once. | ||
|
||
### <a id="quick-run"></a> Quick run action | ||
|
||
Previously, run and update actions were only available from the extension resource explorer UI. Now you can also use a new `Deploy the bundle and run a resource` command from the Command Palette: | ||
|
||
<img src="./2.5/run-command-search.png" alt="pipeline datasets" width="600"/> | ||
|
||
By default, you select the resource to execute: | ||
<img src="./2.5/run-command-ui.png" alt="pipeline datasets" width="600"/> | ||
|
||
You can also specify a shortcut to run the command with specific arguments. | ||
|
||
Open the keyboard shortcuts JSON using the command `Preferences: Open Keyboard Shortcuts (JSON)`. Add a new entry to the `keybindings` array: | ||
|
||
```json | ||
{ | ||
// Replace with your desired shortcut | ||
"key": "ctrl+shift+x", | ||
"command": "databricks.bundle.deployAndRunFromInput", | ||
"args": { | ||
// "jobs" or "pipelines" | ||
"resourceType": "pipelines", | ||
// The key of the pipeline or job you want to run, as defined in your asset bundle configuration | ||
"resourceKey": "my_pipeline_key", | ||
// Any flags that will be passed to the "databricks bundle run" command | ||
"args": "--validate-only" | ||
} | ||
} | ||
``` |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+94.6 KB
packages/databricks-vscode/resources/whats-new/2.5/partial-update.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+101 KB
packages/databricks-vscode/resources/whats-new/2.5/run-command-search.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+76.9 KB
packages/databricks-vscode/resources/whats-new/2.5/run-command-ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.