Fixes for Postgres CDC flags

Merge pull request #103 from batchcorp/blinktag/postgres_cdc_flag_fixes

Flag fixes for CDC Postgres

Improvements to Postgres and Mongo CDC

Improvements:

• A new record is shipped for every change for cdc-postgres, instead of grouping by commit
• Reduced cdc-postgres code deduplication

Fixes:

• Adding missing stats.Incr() calls for cdc-postgres and cdc-mongo
• You may now listen for changes to your entire mongo deployment by omitting the database and collection flags

Minor fix for cdc-postgres flags

v0.22.1

Requiring posgres hostname, username, and database name

PostgreSQL and MongoDB Change Data Capture

This release adds two new relay commands: cdc-postgres and cdc-mongo to allow relaying of CDC data from Postgres' logical replication streams and MongoDB's change stream events.

See https://docs.batch.sh/event-ingestion/change-data-capture/mongodb and https://docs.batch.sh/event-ingestion/change-data-capture/postgresql for documentation on how to stream these events into a Batch.sh collection

v0.21.2

Fixing APK command

Additional flags for nats-streaming

--all - Delivers all available messages
--since - Delivers messages from an interval ago. Ex: 1m, 24h
--last - Delivers last received message
--seq - Delivers message from a given sequence ID

Azure Event Hub,Nats streaming support and SQS FIFO fix

• plumber read azure-eventhub and plumber write azure-eventhub to publish and consume from an azure event hub
• plumber read nats-streaming and plumber write nats-streaming to publish and consume from Nats jetstream
• Added --message-group-id' and --message-deduplication-idflags toplumber write aws-sqs` to support FIFO queus

v0.20.0: Batch account access!

Now you can manage common Batch.sh account tasks directly from Plumber!

Command	Description
plumber batch login	Login and save credentials to ~/.batchsh/config.json
plumber batch logout	Logout and clear saved credentials
plumber batch list collection	List collections
plumber batch create collection --name "My events" --schema-id "0e31f330-aca0-468c-a186-a26e889a19e9"	Create collection
plumber batch search --collection-id="..." --query="*"	Search a collection
plumber batch list schema	List schemas
plumber batch list replay	List replays
plumber batch list destination	List destinations
plumber batch create destination kafka	Create kafka destination
plumber batch create destination aws-sqs	Create SQS destination
plumber batch create destination rabbit	Create RabbitMQ destination
plumber batch create destination http	Create HTTP destination
plumber batch create replay	Create and start a new replay

v0.19.0

Minor relay fixes:

• Fix redis-streams stats display (wasn't increasing stats or muting on error)
• Fix bug where redis-streams relay incorrectly exits on error (should run forever and just log error)
• Kafka relay stats now get muted when read results in an error (+ retry in 5s)
• Add RetryReadInterval on error for gcp pubsub relay
• Add RetryReadInterval on error for aws-sqs relay

v0.18.0

• Added read, write and relay support for redis streams
• Renamed redis to redis-pubsub (to follow existing conventions such as aws-sqs and aws-sns)
• write redis-pubsub now writes to all specified channels (originally was just the first)
• use semantic versioning in --version .. turns out, having a short sha is not terribly intuitive 😄

Install

Download the appropriate release binary or install via brew:

brew tap batchcorp/public
brew install plumber