Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ETL: Add Apache Hop and Apache NiFi #83

Merged
merged 1 commit into from
May 29, 2024
Merged

ETL: Add Apache Hop and Apache NiFi #83

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions docs/integrate/etl/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,25 @@ Tutorials and resources about configuring the managed variants, Astro and CrateD
- [Executable stack: Apache Kafka, Apache Flink, and CrateDB]


## Apache Hop

- [Using Apache Hop with CrateDB]
- [CrateDB dialect for Apache Hop]
- [CrateDB Apache Hop Bulk Loader transform]


## Apache Kafka

- [Data Ingestion using Kafka and Kafka Connect]
- [Executable stack: Apache Kafka, Apache Flink, and CrateDB]
- [Tutorial: Replicating data to CrateDB with Debezium and Kafka]


## Apache NiFi

- [Connecting to CrateDB from Apache NiFi]


## Azure Functions

- {ref}`azure-functions`
Expand Down Expand Up @@ -113,7 +125,10 @@ A demo project which uses SSIS and ODBC to read and write data from CrateDB:
[Build a data ingestion pipeline using Kafka, Flink, and CrateDB]: https://dev.to/crate/build-a-data-ingestion-pipeline-using-kafka-flink-and-cratedb-1h5o
[Building a hot and cold storage data retention policy in CrateDB with Apache Airflow]: https://community.cratedb.com/t/cratedb-and-apache-airflow-building-a-hot-cold-storage-data-retention-policy/934
[Community Day: Stream processing with Apache Flink and CrateDB]: https://cratedb.com/blog/cratedb-community-day-2nd-edition-summary-and-highlights
[Connecting to CrateDB from Apache NiFi]: https://community.cratedb.com/t/connecting-to-cratedb-from-apache-nifi/647
[CrateDB and Apache Airflow: Building a data ingestion pipeline]: https://community.cratedb.com/t/cratedb-and-apache-airflow-building-a-data-ingestion-pipeline/926
[CrateDB Apache Hop Bulk Loader transform]: https://hop.apache.org/manual/latest/pipeline/transforms/cratedb-bulkloader.html
[CrateDB dialect for Apache Hop]: https://hop.apache.org/manual/latest/database/databases/cratedb.html
[CrateDB's PostgreSQL interface]: inv:crate-reference#interface-postgresql
[Data Ingestion using Kafka and Kafka Connect]: https://cratedb.com/docs/crate/howtos/en/latest/integrations/kafka-connect.html
[ETL pipeline using Apache Airflow with CrateDB (Source)]: https://github.com/astronomer/astro-cratedb-blogpost
Expand All @@ -128,6 +143,7 @@ A demo project which uses SSIS and ODBC to read and write data from CrateDB:
[Setting up data pipelines with CrateDB and Kestra]: https://community.cratedb.com/t/setting-up-data-pipelines-with-cratedb-and-kestra-io/1400
[Tutorial: Replicating data to CrateDB with Debezium and Kafka]: https://community.cratedb.com/t/replicating-data-to-cratedb-with-debezium-and-kafka/1388
[Updating stock market data automatically with CrateDB and Apache Airflow]: https://community.cratedb.com/t/updating-stock-market-data-automatically-with-cratedb-and-apache-airflow/1304
[Using Apache Hop with CrateDB]: https://community.cratedb.com/t/using-apache-hop-with-cratedb/1754
[Using dbt with CrateDB]: https://community.cratedb.com/t/using-dbt-with-cratedb/1566
[Using SQL Server Integration Services with CrateDB]: https://github.com/crate/cratedb-examples/tree/main/application/microsoft-ssis
[Webinar: How to replicate data from other databases to CrateDB with Debezium and Kafka]: https://cratedb.com/resources/webinars/lp-wb-debezium-kafka
Expand Down