Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jedis - primary node failover causes issues with in-memory topology Map #4041

Open
Diocasach opened this issue Dec 19, 2024 · 0 comments
Open

Comments

@Diocasach
Copy link

We are using Jedis (5.2.0) to, amongst other things, ping Redis cluster nodes, both primary and replicas. We have configured Jedis to use a topologyRefreshPeriod (introduced in Jedis v5.1.0) to refresh the in-memory topology automatically every few minutes. All works fine until there is a primary node failover in which case the in-memory Map of Redis Node - > Connection Pool held in the Jedis client suddenly loses data: the Map loses data for one or more of the nodes in the same shard in which the failover occurred. Once the app sees that there is missing data in the Map, it calls ClusterConnectionProvider.renewSlotCache to update the Jedis in-memory topology, but the issue remains nonetheless for up to 8 minutes.
Due to using topologyRefreshPeriod and ClusterConnectionProvider.renewSlotCach, the issue eventually self-resolves - the in-memory Map once again is eventually populated with all the node entries for all the shards. As mentioned above, that happens after 8 minutes approx. (Note: before using topologyRefreshPeriod,ClusterConnectionProvider.renewSlotCach, the issue never self-resolved, we had to cycle the EC2 instances to get rid of the issue ).
Have ye seen anything like this before? Why does Jedis take 8 minutes approx to repopulate the in-memory Map with all the nodes?
Note: We are using Amazon ElastiCache and Java 17.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant