Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add thread ID to ingest table #58

Open
gregorydlogan opened this issue Mar 19, 2021 · 0 comments
Open

Add thread ID to ingest table #58

gregorydlogan opened this issue Mar 19, 2021 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@gregorydlogan
Copy link
Member

As of 942df1a when running multiple ingest workers we end up with a race condition where ingests get uploaded multiple times. This is likely caused by a misunderstanding of how SQLAlchemy works.

The logic, as it stands, looks like this:
for each recording that's not in FINISHED:
set its state to in-progress
process it
...

I suspect that multiple threads are setting the in-progress value at the same time and sqlalchemy is allowing all of them to commit at the same time because they're all committing the same value. We need to add a thread id as well to prevent this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant