Skip to content

Commit

Permalink
Whats new 2.5 (#1481)
Browse files Browse the repository at this point in the history
## Changes
What's new in the 2.5

---------

Co-authored-by: Julia Crawford (Databricks) <[email protected]>
  • Loading branch information
ilia-db and juliacrawf-db authored Dec 12, 2024
1 parent e904f71 commit 392bd0a
Show file tree
Hide file tree
Showing 8 changed files with 67 additions and 0 deletions.
67 changes: 67 additions & 0 deletions packages/databricks-vscode/resources/whats-new/2.5.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
<img src="./1.2/databricks-logo.png" alt="databricks-logo" width="100"/>

# Databricks Extension v2.5.0

## What's new?

- [Pipeline validation and partial update actions](#validation)
- [Pipeline events and diagnostics](#events)
- [Pipeline datasets](#datasets)
- [Quick run action](#quick-run)

### <a id="validation"></a> Pipeline validation and partial update actions

In previous versions, you could only trigger a [Refresh all](https://docs.databricks.com/en/delta-live-tables/updates.html#how-dlt-updates) pipeline update. Now you can do validation and partial updates:

<img src="./2.5/validation.png" alt="validation action" width="600"/>

The partial update action preloads datasets from past runs, and lets you enter table names or select a full refresh:

<img src="./2.5/partial-update.png" alt="partial update action" width="600"/>

### <a id="events"></a> Pipeline events and diagnostics

During and after a pipeline update, the extension now shows events in the resource explorer UI:

<img src="./2.5/events.png" alt="pipeline events" width="600"/>

Any warnings or errors that have source locations are also displayed in the Visual Studio Code **PROBLEMS** panel and highlighted in the source files:

<img src="./2.5/diagnostics.gif" alt="pipeline diagnostics" width="600"/>

### <a id="datasets"></a> Pipeline datasets

You can now see pipeline datasets and their schemas in the extension resource explorer:

<img src="./2.5/datasets.png" alt="pipeline datasets" width="600"/>

Datasets are gathered from the events of past runs, so make sure to update or validate your pipeline at least once.

### <a id="quick-run"></a> Quick run action

Previously, run and update actions were only available from the extension resource explorer UI. Now you can also use a new `Deploy the bundle and run a resource` command from the Command Palette:

<img src="./2.5/run-command-search.png" alt="pipeline datasets" width="600"/>

By default, you select the resource to execute:
<img src="./2.5/run-command-ui.png" alt="pipeline datasets" width="600"/>

You can also specify a shortcut to run the command with specific arguments.

Open the keyboard shortcuts JSON using the command `Preferences: Open Keyboard Shortcuts (JSON)`. Add a new entry to the `keybindings` array:

```json
{
// Replace with your desired shortcut
"key": "ctrl+shift+x",
"command": "databricks.bundle.deployAndRunFromInput",
"args": {
// "jobs" or "pipelines"
"resourceType": "pipelines",
// The key of the pipeline or job you want to run, as defined in your asset bundle configuration
"resourceKey": "my_pipeline_key",
// Any flags that will be passed to the "databricks bundle run" command
"args": "--validate-only"
}
}
```
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 392bd0a

Please sign in to comment.