Skip to content

Commit

Permalink
add more tests
Browse files Browse the repository at this point in the history
  • Loading branch information
AstrakhantsevaAA committed Oct 30, 2023
1 parent 21c837b commit 5f7d89b
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -146,9 +146,11 @@ def load_data_with_retry(pipeline, data):
# we reuse the pipeline instance below and load to the same dataset as data
logger.info("Saving the load info in the destination")
pipeline.run([load_info], table_name="_load_info")
assert "_load_info" in pipeline.last_trace.last_normalize_info.row_counts # @@@DLT_REMOVE
# save trace to destination, sensitive data will be removed
logger.info("Saving the trace in the destination")
pipeline.run([pipeline.last_trace], table_name="_trace")
assert "_trace" in pipeline.last_trace.last_normalize_info.row_counts # @@@DLT_REMOVE

# print all the new tables/columns in
for package in load_info.load_packages:
Expand All @@ -160,6 +162,7 @@ def load_data_with_retry(pipeline, data):
# save the new tables and column schemas to the destination:
table_updates = [p.asdict()["tables"] for p in load_info.load_packages]
pipeline.run(table_updates, table_name="_new_tables")
assert "_new_tables" in pipeline.last_trace.last_normalize_info.row_counts # @@@DLT_REMOVE

return load_info

Expand Down
4 changes: 2 additions & 2 deletions docs/website/docs/examples/chess_production/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ We'll learn how to:
- Loading back load information, schema updates, and traces.
- Triggering notifications in case of schema evolution.
- Using context managers to independently retry pipeline stages.
- Run basic tests utilizing sql_client and normalize_info.
- Run basic tests utilizing `sql_client` and `normalize_info`.

### Init pipeline
### Init chess source

<!--@@@DLT_SNIPPET_START ./code/chess-snippets.py::markdown_source-->
```py
Expand Down

0 comments on commit 5f7d89b

Please sign in to comment.