Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Non-unique IDs for tasks in different jobs cause tasks to be overwritten in local DB #2484

Open
gino-m opened this issue May 29, 2024 · 7 comments
Labels
type: bug Something isn't working
Milestone

Comments

@gino-m
Copy link
Collaborator

gino-m commented May 29, 2024

This leads to various odd behaviors, like jobs not showing up as "free form" data collection, or jobs having no tasks.

Instead, we should avoid using UUID as PKs in the local db. Instead, we can generate unique numeric IDs, or prevent non-unique IDs in the local DB.

This problem will go away if we serialize the survey as a binary proto in the local db.

@gino-m gino-m added the type: bug Something isn't working label May 29, 2024
@scolsen
Copy link
Contributor

scolsen commented Aug 2, 2024

Note from Gino: we should work toward using protos in the local db as well. It would be more valuable to fix this temporarily via destructive changes and to work toward leveraging protos instead.

@gino-m gino-m moved this to Todo in Ground Aug 9, 2024
@gino-m
Copy link
Collaborator Author

gino-m commented Aug 13, 2024

I propose the following solution long-term:

  • Serialize survey and jobs as protos in two tables in the local db, eliminating the Tasks table entirely.
  • Use synthetic IDs as PKs in the tables, make UUIDs indexed string fields (e.g., Jobs will have survey_id, job_id, and data columns, the latter storing the serialized Job proto).

@gino-m
Copy link
Collaborator Author

gino-m commented Aug 22, 2024

@scolsen Using protos everywhere is a much larger project. A patch is low priority here since this is a degenerate case which would only happen if someone accidentally manually adds to jobs with the same IDs. Moving to Icebox.

@gino-m gino-m modified the milestones: GA release, Icebox Aug 22, 2024
@kenstershiro
Copy link
Collaborator

Unassigning @scolsen and removing from the current iteration, based on latest comments

@gino-m
Copy link
Collaborator Author

gino-m commented Nov 4, 2024

@scolsen @anandwana001 Can you confirm that tasks with the same ID in different jobs will no longer break the local db?

@gino-m
Copy link
Collaborator Author

gino-m commented Nov 20, 2024

Simple solution: use both survey and job UUID when syncing in local db.

@gino-m
Copy link
Collaborator Author

gino-m commented Nov 20, 2024

Demoting to P2 since this won't happen unless we add a duplicate job ID by hand server side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
Status: Todo
Development

No branches or pull requests

4 participants