You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As of 942df1a when running multiple ingest workers we end up with a race condition where ingests get uploaded multiple times. This is likely caused by a misunderstanding of how SQLAlchemy works.
The logic, as it stands, looks like this:
for each recording that's not in FINISHED:
set its state to in-progress
process it
...
I suspect that multiple threads are setting the in-progress value at the same time and sqlalchemy is allowing all of them to commit at the same time because they're all committing the same value. We need to add a thread id as well to prevent this.
The text was updated successfully, but these errors were encountered:
As of 942df1a when running multiple ingest workers we end up with a race condition where ingests get uploaded multiple times. This is likely caused by a misunderstanding of how SQLAlchemy works.
The logic, as it stands, looks like this:
for each recording that's not in FINISHED:
set its state to in-progress
process it
...
I suspect that multiple threads are setting the in-progress value at the same time and sqlalchemy is allowing all of them to commit at the same time because they're all committing the same value. We need to add a thread id as well to prevent this.
The text was updated successfully, but these errors were encountered: