Skip to content

Commit

Permalink
delete manual duckdb extension installation
Browse files Browse the repository at this point in the history
  • Loading branch information
AstrakhantsevaAA committed Jun 5, 2024
1 parent f9e51d8 commit 88aff7c
Show file tree
Hide file tree
Showing 3 changed files with 5 additions and 14 deletions.
4 changes: 0 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,6 @@ lint-and-test-examples:
poetry run mypy --config-file mypy.ini docs/examples
poetry run flake8 --max-line-length=200 docs/examples
cd docs/tools && poetry run python prepare_examples_tests.py
# install duckdb extension for postgres_to_postgres example
cd docs/examples/postgres_to_postgres && wget http://extensions.duckdb.org/v0.10.3/linux_amd64_gcc4/postgres_scanner.duckdb_extension.gz && gunzip ./postgres_scanner.duckdb_extension.gz
cd docs/examples/postgres_to_postgres && poetry run python ./install_duckdb_extensions.py
# run tests
cd docs/examples && poetry run pytest


Expand Down

This file was deleted.

8 changes: 5 additions & 3 deletions docs/examples/postgres_to_postgres/postgres_to_postgres.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@
keywords: [connector x, pyarrow, zero copy, duckdb, postgres, initial load]
---
:::info
Huge shout out to [Simon Späti](https://github.com/sspaeti) for this example!
:::
This examples shows you how to export and import data from Postgres to Postgres in a fast way with ConnectorX and DuckDB
since the default export will generate `Insert_statement` during the normalization phase, which is super slow for large tables.
Expand Down Expand Up @@ -218,9 +222,7 @@ def table_desc(table_name, pk, schema_name, order_date, columns="*"):
print("##################################### START DUCKDB LOAD ########")
# connect to local duckdb dump
conn = duckdb.connect(f"{load_info.destination_displayable_credentials}".split(":///")[1])
conn.install_extension(
"./postgres_scanner.duckdb_extension"
) # duckdb_extension is downloaded/installed manually here, only needed if `LOAD/INSTALL postgres` throws an error
conn.sql("INSTALL postgres;")
conn.sql("LOAD postgres;")
# select generated timestamp schema
timestamped_schema = conn.sql(
Expand Down

0 comments on commit 88aff7c

Please sign in to comment.