sync mechanism for modifications - baseline #1028
Conversation
eaypek-tfh
changed the title
eaypek-tfh
force-pushed
the
POP-2058/sync-mechanism-for-modifications
branch
from
c4aaa8a to
abc8ae7
Compare
| let ids: Vec<i64> = modifications.iter().map(|m| m.id).collect(); | ||
| let statuses: Vec<String> = modifications.iter().map(|m| m.status.clone()).collect(); | ||
| let persisteds: Vec<bool> = modifications.iter().map(|m| m.persisted).collect(); | ||
|
|
||
| sqlx::query( | ||
| r#" | ||
| UPDATE modifications | ||
| SET status = data.status, | ||
| persisted = data.persisted | ||
| FROM ( | ||
| SELECT | ||
| unnest($1::bigint[]) as id, | ||
| unnest($2::text[]) as status, | ||
| unnest($3::bool[]) as persisted | ||
| ) as data | ||
| WHERE modifications.id = data.id | ||
| "#, | ||
| ) |
There was a problem hiding this comment.
needed to use unnest to get the tests working. strangely, query builder did not work. i get this error before this commit
let me know if you know of a way to avoid the unnest if you think it looks ugly 🙏
There was a problem hiding this comment.
This seems to be the recommended way in sqlx doc. How can I bind an array…
eaypek-tfh
force-pushed
the
POP-2058/sync-mechanism-for-modifications
branch
from
9f86303 to
7732134
Compare
| }) = rx.recv().await | ||
| { | ||
| let _modifications = modifications; |
There was a problem hiding this comment.
This file should stay in-sync and do the same as server.rs.
(For sure this is a drawback of the current state, need refactoring.)
There was a problem hiding this comment.
I intentionally skipped any changes in hawk server because it's just duplicate of gpu server. i think this can be handled together with refactoring. i think any effort i put there now would probably be deleted during refactoring
| let ids: Vec<i64> = modifications.iter().map(|m| m.id).collect(); | ||
| let statuses: Vec<String> = modifications.iter().map(|m| m.status.clone()).collect(); | ||
| let persisteds: Vec<bool> = modifications.iter().map(|m| m.persisted).collect(); | ||
|
|
||
| sqlx::query( | ||
| r#" | ||
| UPDATE modifications | ||
| SET status = data.status, | ||
| persisted = data.persisted | ||
| FROM ( | ||
| SELECT | ||
| unnest($1::bigint[]) as id, | ||
| unnest($2::text[]) as status, | ||
| unnest($3::bool[]) as persisted | ||
| ) as data | ||
| WHERE modifications.id = data.id | ||
| "#, | ||
| ) |
There was a problem hiding this comment.
This seems to be the recommended way in sqlx doc. How can I bind an array…
Change
status=IN_PROGRESSandpersisted=false.status=COMPLETEDandpersisted=true.status=COMPLETEDandpersisted={success_result}.Side Changes
max_deletions_per_batchconfig so that we can safely define a lookback size for modifications sync.max_deletions_per_batchper cron execution.Links