Skip to content

Commit

Permalink
finalizes the docs
Browse files Browse the repository at this point in the history
  • Loading branch information
rudolfix committed Apr 7, 2024
1 parent 67f2fb5 commit 5039b2f
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 1 deletion.
11 changes: 11 additions & 0 deletions docs/website/docs/dlt-ecosystem/destinations/duckdb.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,17 @@ You can configure the following file formats to load data to duckdb:
:::
* [jsonl](../file-formats/jsonl.md) **is supported but does not work if JSON fields are optional. The missing keys fail the COPY instead of being interpreted as NULL.**

:::tip
`duckdb` has [timestamp types](https://duckdb.org/docs/sql/data_types/timestamp.html) with resolutions from milliseconds to nanoseconds. However
only microseconds resolution (the most common used) is time zone aware. `dlt` generates timestamps with timezones by default so loading parquet files
with default settings will fail (`duckdb` does not coerce tz-aware timestamps to naive timestamps).
Disable the timezones by changing `dlt` [parquet writer settings](../file-formats/parquet.md#writer-settings) as follows:
```sh
DATA_WRITER__TIMESTAMP_TIMEZONE=""
```
to disable tz adjustments.
:::

## Supported column hints
`duckdb` may create unique indexes for all columns with `unique` hints, but this behavior **is disabled by default** because it slows the loading down significantly.

Expand Down
2 changes: 1 addition & 1 deletion docs/website/docs/dlt-ecosystem/file-formats/parquet.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ NORMALIZE__DATA_WRITER__TIMESTAMP_TIMEZONE

### Timestamps and timezones
`dlt` adds timezone (UTC adjustment) to all timestamps regardless of a precision (from seconds to nanoseconds). `dlt` will also create TZ aware timestamp columns in
the destinations. If the latter is impossible, there are workaround
the destinations. [duckdb is an exception here](../destinations/duckdb.md#supported-file-formats)

### Disable timezones / utc adjustment flags
You can generate parquet files without timezone adjustment information in two ways:
Expand Down

0 comments on commit 5039b2f

Please sign in to comment.