Redis relay

This release adds support for relaying Redis pubsub channels to Batch.

To run plumber relay with redis in Docker:

docker run -d --name plumber-redis -p 8080:8080 \
    -e PLUMBER_RELAY_REDIS_ADDRESS=localhost:6379 \
    -e PLUMBER_RELAY_REDIS_CHANNELS=channel1,channel2 \
    -e PLUMBER_RELAY_TYPE=redis \
    -e PLUMBER_RELAY_TOKEN=$YOUR-BATCHSH-TOKEN-HERE \
    batchcorp/plumber

Dockerhub: docker pull batchcorp/plumber:a934947

Relay mode fixes and cli flag improvements

Breaking changes:

• remove --output-type for reading; remove --input-type for writing. Both are unnecessary and the intent can be gathered from other flags.

Features:

• Kafka: Ability to specify an offset to begin reading from
• GCP: Environment variable PUBSUB_EMULATOR_HOST can now be specified to connect to a pubsub emulator instead of GCP
• Retries for GRPC calls in relay mode

Bug fixes:

• --queue-declare now actually declares the queue for rabbit
• Increasing max record size for relay payloads

Batching for all relays, stats, improved kafka

This is a fairly large (and awesome) release.

In this release:

• New --stats flag (and PLUMBER_STATS env var) which will cause plumber to periodically dump consume/produce speed stats.
• Lots of kafka improvements to improve read and relay throughput
  • Several new env vars that can be tweaked to further improve throughput and/or reliability
• Batching support for relay
• Pretty config printing - all relay modes will now print out the config they're using on console
  • plumber now has a lot of options - some messaging platforms (such as kafka) have 20+ flags/options - it is nice to be able to see what options are actually being used at runtime.

Enjoy!

Fixing quiet flag

v0.14.1

Fixing quiet flag

Support for relaying GCP PubSub messages to Batch collections

v0.14.0

Support for GCP PubSub relaying

Support for Azure relay, improved Kafka read and relay

1. Support for relaying Azure Service Bus to Batch.sh
2. Improved Kafka read and relay speeds; works with Confluent cloud!
   1. Kafka read and relay is highly configurable for improved throughput
3. Improved docs (including env var docs!)

Docker: https://hub.docker.com/r/batchcorp/plumber/

Updating GRPC token name

v0.12.1

Modifying grpc token name

Support for NATS and Redis PubSub

Merge pull request #65 from batchcorp/blinktag/redis_pubsub

Support for Redis PubSub

Support for Azure Service Bus and AWS SNS

Merge pull request #61 from batchcorp/blinktag/aws-sns

Write support for AWS-SNS

Kafka relay support

v0.10.0

Kafka relay support