Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Just want the final solution for some previous similar issues(*nodes marked as dead) #1449

Closed
yudidi opened this issue Dec 21, 2020 · 1 comment

Comments

@yudidi
Copy link

yudidi commented Dec 21, 2020

Please use the following questions as a guideline to help me answer
your issue/question without further inquiry. Thank you.

Which version of Elastic are you using?

elastic.v5 (for Elasticsearch 5.x)

Please describe the expected behavior

Just want the final solution for some previous similar issues:

  1. When the context is cancelled the node is marked dead #484
  2. AWS endpoint with varying IP address #1091

Please describe the actual behavior

I saw this log sometimes in my app. But I don't know what causes this and I don't find the final solution after I view some similar issues. Could you please tell me the final solution?

ELASTIC 2020/12/14 10:53:38 elastic: http://xxxdomainxxx.elasticsearch.aliyuncs.com:9200 is dead
ELASTIC 2020/12/14 10:53:38 elastic: all 1 nodes marked as dead; resurrecting them to prevent deadlock

Any steps to reproduce the behavior?

It's hard to describe. The information I can provide is as following, are they enough?

My current code is:

elastic.SetSniff(false)

Should I solve it by closing the health check?

elastic.SetSniff(false),
elastic.SetHealthcheck(false),
@yudidi
Copy link
Author

yudidi commented Dec 25, 2020

I found the source code:

elastic/client.go

Lines 1164 to 1173 in 423089d

// We have a deadlock here: All nodes are marked as dead.
// If sniffing is disabled, connections will never be marked alive again.
// So we are marking them as alive--if sniffing is disabled.
// They'll then be picked up in the next call to PerformRequest.
if !c.snifferEnabled {
c.errorf("elastic: all %d nodes marked as dead; resurrecting them to prevent deadlock", len(c.conns))
for _, conn := range c.conns {
conn.MarkAsAlive()
}
}

If you want to know what is the purpose of the code, click the red rectangle for more detail.
image

@yudidi yudidi closed this as completed Dec 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant