Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis with Sentinel enabled does not start properly #30578

Open
augustas1 opened this issue Nov 22, 2024 · 0 comments
Open

Redis with Sentinel enabled does not start properly #30578

augustas1 opened this issue Nov 22, 2024 · 0 comments
Assignees
Labels
in-progress redis tech-issues The user has a technical issue about an application

Comments

@augustas1
Copy link

Name and Version

bitnami/redis 20.3.0

What architecture are you using?

None

What steps will reproduce the bug?

  1. Create k8s cluster using kind
kind create cluster
  1. Deploy redis helm chart with Sentinel enabled
helm install redis oci://registry-1.docker.io/bitnamicharts/redis --set sentinel.enabled=true
  1. Pods fail to start. See the logs of e.g. redis-node-0 sentinel pod. Includes errors like:
Could not connect to Redis at redis.default.svc.cluster.local:26379: Connection refused
Failed to resolve hostname 'redis-node-1.redis-headless.default.svc.cluster.local'  

Full redis-node-0 sentinel pod logs:

08:00:14.23 INFO  ==> about to run the command: REDISCLI_AUTH=$REDIS_PASSWORD timeout 90 redis-cli -h redis.default.svc.cluster.local -p 26379 sentinel get-master-addr-by-name mymaster                                       │    ││
│ Could not connect to Redis at redis.default.svc.cluster.local:26379: Connection refused                                                                                                                                         │    ││
│ Could not connect to Redis at redis.default.svc.cluster.local:26379: Connection refused                                                                                                                                         │    ││
│ 1:X 22 Nov 2024 08:00:24.435 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo                                                                                                                                                    │    ││
│ 1:X 22 Nov 2024 08:00:24.435 * Redis version=7.4.1, bits=64, commit=00000000, modified=1, pid=1, just started                                                                                                                   │    ││
│ 1:X 22 Nov 2024 08:00:24.435 * Configuration loaded                                                                                                                                                                             │    ││
│ 1:X 22 Nov 2024 08:00:24.435 * monotonic clock: POSIX clock_gettime                                                                                                                                                             │    ││
│ 1:X 22 Nov 2024 08:00:24.437 * Running mode=sentinel, port=26379.                                                                                                                                                               │    ││
│ 1:X 22 Nov 2024 08:00:24.437 * Sentinel ID is 2a09ba7abbb41ee71e79087310d75f9809c3c815                                                                                                                                          │    ││
│ 1:X 22 Nov 2024 08:00:24.437 # +monitor master mymaster redis-node-0.redis-headless.default.svc.cluster.local 6379 quorum 2                                                                                                     │    ││
│ 1:X 22 Nov 2024 08:00:51.518 # Failed to resolve hostname 'redis-node-1.redis-headless.default.svc.cluster.local'                                                                                                               │    ││
│ 1:X 22 Nov 2024 08:00:54.591 * +sentinel sentinel 33535e4e17bf8f9f9ff9ce8f9ddf609e558ff4f2 redis-node-1.redis-headless.default.svc.cluster.local 26379 @ mymaster redis-node-0.redis-headless.default.svc.cluster.local 6379    │    ││
│ 1:X 22 Nov 2024 08:00:54.593 * Sentinel new configuration saved on disk                                                                                                                                                         │    ││
│ 1:X 22 Nov 2024 08:00:54.594 # +tilt #tilt mode entered                                                                                                                                                                         │    ││
│ 1:X 22 Nov 2024 08:00:54.594 # waitpid() returned a pid (98) we can't find in our scripts execution queue!                                                                                                                      │    ││
│ 1:X 22 Nov 2024 08:00:54.594 # waitpid() returned a pid (114) we can't find in our scripts execution queue!                                                                                                                     │    ││
│ 1:X 22 Nov 2024 08:01:00.735 # Failed to resolve hostname 'redis-node-1.redis-headless.default.svc.cluster.local'                                                                                                               │    ││
│ 1:X 22 Nov 2024 08:01:13.022 # Failed to resolve hostname 'redis-node-1.redis-headless.default.svc.cluster.local'                                                                                                               │    ││
│ 1:X 22 Nov 2024 08:01:34.525 # Failed to resolve hostname 'redis-node-1.redis-headless.default.svc.cluster.local'

Considerations
I've also tried deploying it to AWS EKS cluster, but got the same result.

Are you using any custom parameters or values?

--set sentinel.enabled=true

What is the expected behavior?

Redis Sentinel starts without failure.

What do you see instead?

Erroring and restarting Redis Sentinel pods.

Additional information

No response

@augustas1 augustas1 added the tech-issues The user has a technical issue about an application label Nov 22, 2024
@github-actions github-actions bot added the triage Triage is needed label Nov 22, 2024
@github-actions github-actions bot removed the triage Triage is needed label Nov 25, 2024
@github-actions github-actions bot assigned migruiz4 and unassigned carrodher Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in-progress redis tech-issues The user has a technical issue about an application
Projects
None yet
Development

No branches or pull requests

3 participants