Skip to content

Commit

Permalink
Merge branch 'master' into blog-pyairbyte
Browse files Browse the repository at this point in the history
  • Loading branch information
adrianbr authored Feb 28, 2024
2 parents 1b73d83 + 52e3860 commit 0bc5c29
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 1 deletion.
2 changes: 2 additions & 0 deletions docs/website/blog/2024-02-28-what-is-pyairbyte.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,10 @@ Here at dltHub, we work on the python library for data ingestion. So when I hear

PyAirbyte is an interesting Airbyte’s initiative - similar to the one that Meltano had undertook 3 years ago. It provides a convenient way to download and install Airbyte sources and run them locally storing the data in a cache dataset. Users are allowed to then read the data from this cache.


A Python wrapper on the Airbyte source is quite nice and has a feeling close to [Alto](https://github.com/z3z1ma/alto). The whole process of cloning/pip installing the repository, spawning a separate process to run Airbyte connector and read the data via UNIX pipe is hidden behind Pythonic interface.


Note that this library is not an Airbyte replacement - the loaders of Airbyte and the library are very different. The library loader uses pandas.to_sql and sql alchemy and is not a replacement for Airbyte destinations that are available in Open Source Airbyte

# Questions I had, answered
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,4 +140,4 @@ def get_pages(

# check that stuff was loaded
row_counts = pipeline.last_trace.last_normalize_info.row_counts
assert row_counts["ticket_events"] == 16
assert row_counts["ticket_events"] >= 17

0 comments on commit 0bc5c29

Please sign in to comment.