Code

New "refresh" mode and "dev_mode"#1063

Code

Merged

rudolfix merged 51 commits intodeveldlt-hub/dlt:develfrom sthor/full-refresh-adjustmentdlt-hub/dlt:sthor/full-refresh-adjustmentCopy head branch name to clipboardJun 3, 2024

+1,263-380

Commits

Commits on May 29, 2024

• Rename full_refresh -> dev_mode, add deprecation warning

  steinitzu

  committed
  dba576b

• Replace some full_refresh usage in code and docs

  steinitzu

  committed
  432d1d7

• Replace full_refresh usage in tests

  steinitzu

  committed
  c7e86ef

• Init experimental refresh = full with drop command

  steinitzu

  committed
  8c13683

• Refresh modes with dropped_tables file

  steinitzu

  committed
  613257e

• Init separate local/write drop command

  steinitzu

  committed
  2d46305

• Use load_package_state instead of drop tables file

  steinitzu

  committed
  9b63cff

• Use drop schema in init_client (TODO: error)

  steinitzu

  committed
  3b6d809

• Separate cli drop command instructions/execute

  steinitzu

  committed
  4334553

• drop tables in init_client

  steinitzu

  committed
  b6a21aa

• dropped_tables field on load package state

  steinitzu

  committed
  e9543e0

• Fix import

  steinitzu

  committed
  7f95d9e

• test/fix truncate mode

  steinitzu

  committed
  3e0af96

• Save truncated tables in load package state

  steinitzu

  committed
  e9528b1

• Remove load package state copying

  steinitzu

  committed
  10fe6f3

• cleanup

  steinitzu

  committed
  44cc1cc

• Drop cmd use package state, refactoring

  steinitzu

  committed
  54de570

• Don't drop tables without data

  steinitzu

  committed
  6aae866

• Validate literals in configspec

  steinitzu

  committed
  29b6c96

• Match stored schema by version+version_hash

  Show description for 8a2276b
  steinitzu

  committed
  8a2276b

• Cleanup

  steinitzu

  committed
  2546021

• Fix dlt version test

  steinitzu

  committed
  c749eae

• Cleanup

  steinitzu

  committed
  7802e85

• Remove dropped_tables_filename

  steinitzu

  committed
  ce41fb0

• Fix snippet

  steinitzu

  committed
  0f9358d

• Pipeline refresh docs

  steinitzu

  committed
  7a161a8

• refresh argument docstring

  steinitzu

  committed
  113d577

• Restore and update commented drop cmd tests

  steinitzu

  committed
  73aab0e

• Cleanup refresh tests and test whether table dropped vs truncated

  steinitzu

  committed
  f18509e

• Test existing schema hash

  steinitzu

  committed
  b662381

• Revert "Match stored schema by version+version_hash"

  Show description for ed43445
  steinitzu

  committed
  ed43445

• Use replace_schema flag

  steinitzu

  committed
  fbfb97c

• Change drop_tables replace_schema to delete_schema

  steinitzu

  committed
  2ed7d39

• Refresh drop only selected sources

  steinitzu

  committed
  b8d98f6

• Rename refresh modes, update docs

  steinitzu

  committed
  1a28168

• pipeline.run/extract refresh argument

  steinitzu

  committed
  360e2c4

• Don't modify schema when refresh='drop_data'

  steinitzu

  committed
  6f5a853

• Move refresh tests to load, add filesystem truncate test

  steinitzu

  committed
  a122834

• Fix duck import

  steinitzu

  committed
  bbf56a0

• Remove generated doc sections

  steinitzu

  committed
  d9bfe90

• Default full_refresh=None

  steinitzu

  committed
  0abffa5

• Cleanup unused imports

  steinitzu

  committed
  f096596

• Close caution blocks

  steinitzu

  committed
  be7e0d0

• Update config field docstring

  steinitzu

  committed
  8b5aa2d

• Add filesystem drop_tables method

  steinitzu

  committed
  aaaf519

• Run all refresh tests on local filesystem destination

  steinitzu

  committed
  e3b5b35

• Fix test drop

  steinitzu

  committed
  3dcd1ab

• Fix iter filesystem schemas

  steinitzu

  committed
  8580ea4

• Fix drop_resources

  steinitzu

  committed
  6a06449

• Default config.full_refresh also None

  steinitzu

  committed
  87cfd16

• Fix filesystem + test

  steinitzu

  committed
  879e4e5