Skip to content

Latest commit

 

History

History
56 lines (43 loc) · 5.08 KB

connectors.md

File metadata and controls

56 lines (43 loc) · 5.08 KB

Conduit Connectors

Connectors are an integral part of Conduit. Conduit ships with a couple of connectors that are built into the service to help developers bootstrap pipelines much more quickly. The built-in connectors include Postgres, File, Random Data Generator, Kafka and Amazon S3 connectors.

The difference between Conduit connectors and those you might find from other services is that Conduit connectors are Change Data Capture-first (CDC). CDC allows your pipeline to only get the changes that have happened over time instead of pulling down an entire upstream data store and then tracking diffs between some period of time. This is critical for building real-time event-driven pipelines and applications. But, we'll note where connectors don't have CDC capabilities.

Roadmap & Feedback

If you need support for a particular data store that doesn't exist on the connector list, check out the list of requested source connectors and the list of requested destination connectors. Give the issue a +1 if you really need that connector. The upvote will help the team understand demand for any particular connector. If you find that an issue hasn't been created for your data store, please create a new issue in the Conduit repo.

Connectors

Support Types

  • Conduit - These are connectors that are built by the Conduit. Any issues or problems filed on those repos will be respond to by the Conduit team.
  • Community - A community connector is one where a developer created a connector and they're the ones supporting it not the Conduit team.
  • Legacy - Some connectors are built using non-preferred methods. For example, Kafka Connect connectors can be used on Conduit. This is considered a stop gap measure until conduit or community connectors are built.

At this time, Conduit does not have any commercially supported connectors.

Source & Destination

Source means the connector has the ability to get data from an upstream data store. Destination means the connector can to write to a downstream data store.

The List

Connector Source Destination Support Conduit version support
Airtable WIP Community WIP
Algolia Conduit v0.3.0
Azure Storage Community v0.3.0
BigQuery Community v0.3.0
DB2 WIP WIP Community WIP
Elasticsearch Community v0.3.0
File Conduit v0.3.0
Firebolt WIP WIP Community WIP
GCP PubSub Community WIP
Google Cloud Storage Community v0.3.0
Google Sheets Community v0.3.0
Kafka Conduit v0.3.0
Kafka Connect Wrapper Legacy v0.3.0
Marketo Community v0.3.0
Materialize Community v0.3.0
Nats Jetstream Community v0.3.0
Nats PubSub Community v0.3.0
Oracle DB WIP WIP Community WIP
Postgres Conduit v0.3.0
Random Generator Conduit v0.3.0
Redis Community v0.3.0
S3 Conduit v0.3.0
Salesforce Community v0.3.0
Snowflake Community v0.3.0
Stripe Community v0.3.0
Vitess WIP WIP Community WIP
Zendesk Community v0.3.0