Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to update server instance heartbeat to the database #125

Merged
merged 1 commit into from
Jun 12, 2024

Conversation

yanmxa
Copy link
Contributor

@yanmxa yanmxa commented Jun 12, 2024

When the maestro server is down for a while, it's up again. It might not sync its heartbeat to the database causing the following reasons:

E0612 09:28:05.379536   72292 logger.go:129]   failed to mark transaction for rollback: could not retrieve transaction from context 
E0612 09:28:05.380320   72292 logger.go:129]   Unable to upsert maestro instance: pq: duplicate key value violates unique constraint "server_instances_pkey" 

So the behavior will be:

  1. The server is up, update the heartbeat
  2. The server is down, then the liveness goroutine will mark the server as deleted if it reaches a specific duration
  3. The server is up again, it will update the the heartbeat to the database and then mark the deleted_at as null

Reference: #109
Signed-off-by: myan [email protected]

@yanmxa
Copy link
Contributor Author

yanmxa commented Jun 12, 2024

/assign @morvencao

@clyang82
Copy link
Contributor

/ok-to-test

Copy link
Contributor

@clyang82 clyang82 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

Copy link
Contributor

@morvencao morvencao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@clyang82 clyang82 merged commit 6c5f14c into openshift-online:main Jun 12, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants