Redis Streams adapter may cause event drops?

[RomainFranceschini]

Hi, I use the Redis stream adapter but I'm concerned with event dropping.

AFAIU, since this adapter evicts old entries when adding new events, a greater insertion rate than the read rate would cause event drops.

I guess this is acceptable since the maxLen configuration can be tweaked, but I'm struggling to find a good way to monitor the stream in production and gather valuable data that would help me have a comfortable maxLen value.

Am I overthinking this?

[darrachequesne]

Hi!

That's a valid concern. The maxLen values should be set in relation with:

• the number of connected clients
• the number of messages per second
• the total bandwidth

If the clients are not able to consume the events fast enough, then even setting maxLen to <a very big value> would be useless.

Regarding the monitoring, the adapter has an internal offset (here), which could possibly be made public in order to ease the monitoring. Or switch to a consumer group, though that complicates the setup a bit.