Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis Clustered Pubsub Delayed SSubscribe triggers MOVE error. #3099

Open
CR7273868 opened this issue Sep 1, 2024 · 0 comments
Open

Redis Clustered Pubsub Delayed SSubscribe triggers MOVE error. #3099

CR7273868 opened this issue Sep 1, 2024 · 0 comments

Comments

@CR7273868
Copy link

Redis or Redis GO does not go well with Clustered PUBSUB of Redis

Expected Behavior

Redis GO works even when using the same PUBSUB pointer to subscribe to other channels later on.

Current Behavior

Redis GO does SSubscribe, but then if u do one later one let's say in a goroutine. It says MOVED to another IP or PORT. And doesn't receive the messages at all, afterwards.

Possible Solution

Keep track of shards where what comes from. Fred from Redis RS provides a way to see what shard channel a server was bound to. Per message u'd receive, you could see which server it was hooked too, if I am not wrong.

FRED RS

Steps to Reproduce

package main

import (
	"context"
	"log"
	"time"

	"github.com/redis/go-redis/v9"
)

func main() {
	client := redis.NewClusterClient(&redis.ClusterOptions{
		Addrs: []string{"masked:7000"},
	})

	client_2 := redis.NewClusterClient(&redis.ClusterOptions{
		Addrs: []string{"masked:7000"},
	})

	pubsub := client.SSubscribe(context.Background(), "mychannel")
	defer pubsub.Close()

	go func() {
		// Subscribe to a channel
		time.Sleep(2 * time.Second)
		if err := pubsub.SSubscribe(context.Background(), "mychannel2"); err != nil {
			panic(err)
		}
		log.Println("Subscribed to mychannel2")
	}()
	go func() {
		ticker := time.NewTicker(1 * time.Second)
		for range ticker.C {
			// Channel 2 does not work when ssubscribing later
			client_2.SPublish(context.Background(), "mychannel2", "hello")
			// Does work since we subscribed when we started the program
			client.SPublish(context.Background(), "mychannel", "hello")
		}
	}()
	for {
		msg, err := pubsub.ReceiveMessage(context.Background())
		if err != nil {
			log.Println("ERROR", err)
			continue
		}
		log.Println(msg.Channel, msg.Payload)
	}

}

Output log:

2024/09/01 07:31:00 mychannel hello
2024/09/01 07:31:01 Subscribed to mychannel2
2024/09/01 07:31:01 ERROR MOVED 4312 masked:7002
2024/09/01 07:31:01 mychannel hello
2024/09/01 07:31:02 mychannel hello
2024/09/01 07:31:03 mychannel hello
2024/09/01 07:31:04 mychannel hello
2024/09/01 07:31:05 mychannel hello
2024/09/01 07:31:06 mychannel hello

All you need is a Redis Cluster and use SSUBSCRIBE and have it move across redis nodes to replicate this issue. I have tried several cluster setups correctly, but keep running into this issue. I assume IOREDIS from Node.JS has this exact same issue. The only that works is Fred from Redis which I actively use.

Context (Environment)

4 servers: Each has 3 master coupled with slave to each master. Granting 12 masters coupled with 12 slaves.
Each server is compiled with Redis:

Redis server v=7.2.5 sha=00000000:0 malloc=jemalloc-5.3.0 bits=64 build=b5c2f37750637637

Making Redis PUBSUB work correctly, without having to deal with weird issues like moving and it goes out of scope

Redis Clustered PUBSUB doesn't work after subscribing later on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant