Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Datasets section with dataset schemas to the pipelines explorer #1479

Merged
merged 5 commits into from
Dec 12, 2024

Conversation

ilia-db
Copy link
Contributor

@ilia-db ilia-db commented Dec 5, 2024

Changes

Initially Datasets aren't loaded, but you can expand them to trigger loading
Screenshot 2024-12-05 at 10 14 41

If we can't find dataset definitions in the latests runs we show "please run or validate" item:
Screenshot 2024-12-05 at 10 14 58

Screenshot 2024-12-05 at 10 15 14

Running or validating the pipeline will also show 'loading' state for the datasets section:

Screenshot 2024-12-05 at 10 28 09

Tests

Unit and e2e tests

@ilia-db ilia-db requested a review from fjakobs December 5, 2024 09:29
@ilia-db ilia-db temporarily deployed to test-trigger-is December 5, 2024 09:29 — with GitHub Actions Inactive
@ilia-db ilia-db temporarily deployed to test-trigger-is December 5, 2024 09:29 — with GitHub Actions Inactive
Don't load all events every poll intervall, but only the new ones.
Copy link
Contributor

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/vscode

Inputs:

  • PR number: 1479
  • Commit SHA: 259e504ca680c123b86511365ce1b82edf751798

Checks will be approved automatically on success.

@eng-dev-ecosystem-bot
Copy link
Collaborator

Test Details: go/deco-tests/12275737839

Copy link
Contributor

@fjakobs fjakobs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really nice!

@ilia-db ilia-db merged commit e904f71 into main Dec 12, 2024
6 checks passed
@ilia-db ilia-db deleted the dlt-schemas branch December 12, 2024 09:24
@github-actions github-actions bot mentioned this pull request Dec 12, 2024
ilia-db added a commit that referenced this pull request Dec 13, 2024
## packages/databricks-vscode
##  (2024-12-12)

* Add `Datasets` section with dataset schemas to the pipelines explorer
(#1479)
([e904f71](e904f71)),
closes
[#1479](#1479)
* Add Event Log section to the pipeline subtree (#1444)
([3c3fc9c](3c3fc9c)),
closes
[#1444](#1444)
* Add partial update to pipelines (#1453)
([b7cbe4c](b7cbe4c)),
closes
[#1453](#1453)
* Add validate runs for pipelines (#1436)
([bc0d488](bc0d488)),
closes
[#1436](#1436)
* Bump JS SDK (#1447)
([aa8eae8](aa8eae8)),
closes
[#1447](#1447)
* Context menus everywhere (#1461)
([9db90f8](9db90f8)),
closes
[#1461](#1461)
* DLT diagnostics (#1471)
([7b962ff](7b962ff)),
closes
[#1471](#1471)
* Do not load too many events (#1484)
([79ad92c](79ad92c)),
closes
[#1484](#1484)
* Improve bundle logs (#1475)
([ba9518d](ba9518d)),
closes
[#1475](#1475)
* Improve DLT UX (#1474)
([27f8a9b](27f8a9b)),
closes
[#1474](#1474)
* Increase the polling time for job run status (#1451)
([d410b62](d410b62)),
closes
[#1451](#1451)
* Preload datasets (#1458)
([910dccb](910dccb)),
closes
[#1458](#1458)
* Python launch config is deprecated (#1465)
([1f1df4b](1f1df4b)),
closes
[#1465](#1465)
* Update Databricks CLI to v0.236.0 (#1480)
([8a19a41](8a19a41)),
closes
[#1480](#1480)
* Use cluster policy in pipeline tests (#1440)
([5ba42f7](5ba42f7)),
closes
[#1440](#1440)
* Validate icon (#1441)
([54f7376](54f7376)),
closes
[#1441](#1441)
* Whats new 2.5 (#1481)
([392bd0a](392bd0a)),
closes
[#1481](#1481)



## packages/databricks-vscode-types
##  (2024-12-12)

---------

Co-authored-by: releasebot <[email protected]>
Co-authored-by: Ilia Babanov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants