Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update last reindex bug #79

Open
spacemansteve opened this issue Jan 30, 2019 · 0 comments
Open

Update last reindex bug #79

spacemansteve opened this issue Jan 30, 2019 · 0 comments
Assignees
@spacemansteve

Comments

@spacemansteve
Copy link
Contributor

spacemansteve commented Jan 30, 2019 •
edited
Loading

During its periodic, cron-initiated reindex, master pipeline wants to process new records. So, retrieves the timestamp of when it last reindexed (which is stored in its postgres key/value table). As expected, this timestamp is used in a filter to generate a list of new bibcodes. However, the timestamp is also immediately updated. So if we fail to queue the bibcodes and then process them, they will be skipped during the next reindex. This happened when all the servers in SDC crashed on Jan 24 2019.
@spacemansteve spacemansteve self-assigned this Jan 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spacemansteve spacemansteve

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@spacemansteve