Identify the amount of data lost and restore, and fix the rootcause

[mahalakshme]

People in 7.0.3 versions are facing data loss issues:

https://avni.freshdesk.com/a/tickets/3658
https://avni.freshdesk.com/a/tickets/3656

• In the meantime we try to reproduce and fix the rootcause of the issue, people on ground should not get affected by the lost data. So restore the data side by side.
• Identify the rootcause and fix the issue.

[petmongrels]

issue seems isolated to certain users. one user was test user since 7.0.0 and has new app with old db.

JNPCT issue is happening from before. Query used.

select users.username, program_enrolment.last_modified_date_time::date, count(*) from program_enrolment join program on program_enrolment.program_id = program.id and program.name = 'Eligible couple' join individual on program_enrolment.individual_id = individual.id join organisation on individual.organisation_id = organisation.id join users on users.id = program_enrolment.last_modified_by_id where program_enrolment.observations::text = '{}' and program_enrolment.program_exit_date_time is null and program_enrolment.is_voided = false and program.is_voided = false and program_enrolment.last_modified_date_time >= '2023-11-01' group by 1,2 order by 2 desc,1;

Other info.
-- no issue in jss and in registration. jnpct & ihmp in enrolment for some users only

[mahalakshme]

@petmongrels since Identifying the lost data is linked to identifying the rootcause - I added that also to this card, since u are already working on both.

[mahalakshme]

@petmongrels adding the updates here, kindly edit if I missed something:

• Data of all orgs restored
• Issue happening from around 3 months back - around 6.0 itself
• Fix done on server side to prevent saving of empty observations. During next time sync users will get back the actual value to mobile. But @petmongrels I think this will not happen since last_modified_date_time in the server db will not be updated.
• Inspite of the fix above, user will permanently lose any edited value.

[petmongrels]

These commits should be included in the next 7.0.x release.

• avniproject/avni-client@4073aec
• avniproject/avni-client@2e8756b