add notes and gender to patient bulk upload #6778

fzhao99 · 2023-10-20T17:22:13Z

BACKEND PULL REQUEST

Related Issue

resolves Add notes and gender to patient bulk upload #6649 by adding gender identity and an address notes column to the patient bulk upload. This will align our individual update with the bulk upload
Corresponding static site update here
- Will hold off on merging this until that PR gets ok'ed (pending a review from Laura)

Changes Proposed

Adds the new fields to the PatientUploadRow creation and validation process
Refactors the Person model slightly to work better with the two fields

Testing

deployed on dev2

Happy Path

Example csv below. Note the

Alternate casing for gender identity
>500 char string for address notes
- We have a 500 char cap for address, but guessing this is a smarty limitation since first / last name don't have limitations on char length
- Happy to cap this at a certain value though in case we want a certain limitation!

last_name,first_name,middle_name,suffix,race,date_of_birth,biological_sex,ethnicity,street,street_2,city,county,state,zip_code,phone_number,phone_number_type,employed_in_healthcare,resident_congregate_setting,role,email,gender_identity,address_notes
Kuphal,Lucienne,Horton,,black or african american,3/12/86,female,hispanic or latino,5945 Rath Manors,,West Murielstead,,AK,99501,410-675-4559,Mobile,No,No,Staff,[email protected],transman,
Lebsack,Devyn,Kyle ,,asian,11/5/69,male,hispanic or latino,263 Windler Canyon,,Marquardtburgh,,AK,99501,410-646-4949,Mobile,No,No,Staff,[email protected],nonbinary,Test note
West,Theodora,Stacy,,asku,3/4/66,female,not hispanic or latino,40405 Lionel Lakes,,New Savannah,,AK,99501,410-754-2890,Landline,No,No,Staff,[email protected],,
MacGyver,Retta,Moreno,,asian,10/7/57,male,hispanic or latino,089 Dayton Rue,,Vacaville,,AK,99501,410-483-5354,Mobile,No,No,Staff,[email protected],Female,
Durgan,Cristobal,,,other,1/20/87,female,hispanic or latino,60556 Jacobs Lights,,Plantation,,AK,99501,410-115-7652,Landline,No,No,Staff,[email protected],,
Gutmann,Garth,Mendez,,asku,10/12/66,male,not hispanic or latino,90074 Timothy Club,,Fort Daisyport,,AK,99501,410-895-2894,Mobile,No,No,Staff,[email protected],,
Hettinger,Albert,,,other,6/13/85,female,hispanic or latino,333 Schinner Highway,,West Zora,,AK,99501,410-392-4738,Mobile,No,No,Staff,[email protected],,"Test note with long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long string longer than 500 char
"
Morissette,Madilyn,Ethel ,,native hawaiian or other pacific islander,2/28/03,male,hispanic or latino,17602 Armand Lock,,Madisenborough,,AK,99501,410-745-5120,Mobile,No,No,Staff,[email protected],Male,Test note again with @ a special ^ char
McDermott,Jovan,Norton,,black or african american,5/18/62,female,not hispanic or latino,214 Lilla Village,,Gislasonland,,AK,99501,410-759-7317,Mobile,No,No,Staff,[email protected],,
Sporer,Rahsaan,,,asku,6/12/40,male,hispanic or latino,838 Waelchi Extensions,,Palm Bay,,AK,99501,410-881-4268,Landline,No,No,Staff,[email protected],,

Screen.Recording.2023-10-23.at.11.27.48.AM.mov

Bad string value for gender identity

Upload without extra columns still works

Screen.Recording.2023-10-23.at.11.51.37.AM.mov

backend/src/main/java/gov/cdc/usds/simplereport/db/model/Person.java

@@ -163,6 +164,56 @@
    this.notes = notes;
  }

+  @Builder


fzhao99 · 2023-10-20T17:40:04Z

backend/src/main/java/gov/cdc/usds/simplereport/service/PatientBulkUploadServiceAsync.java

-                null, // preferredLanguage
-                null // testResultDeliveryPreference
-                );
+            Person.builder()


it was super annoying to deal with the multi-param constructor off the person object so I added a new one with the Builder annotation to make it more manageable. Do recognize that we're adding an entirely new constructor to enable this though, so if folks have a better way to work with this large object lmk.

fzhao99 · 2023-10-23T16:00:09Z

backend/src/main/java/gov/cdc/usds/simplereport/service/PatientBulkUploadServiceAsync.java

+                .middleName(extractedData.getMiddleName().getValue())
+                .lastName(extractedData.getLastName().getValue())
+                .suffix(extractedData.getSuffix().getValue())
+                .notes(extractedData.getNotes().getValue())


Right now since we're not doing any string interpolation this column can either get filled with null (if the columns aren't specified) or the empty string (if the column is specified but there isn't any value for that row.) I left it this way since we're doing the same thing in other places.

lmk if anyone prefers null or "" explicitly though I can go back and clean it up here / in other settings

fzhao99 · 2023-10-23T16:30:08Z

backend/src/main/java/gov/cdc/usds/simplereport/service/PatientBulkUploadServiceAsync.java

-                parseYesNoUnk(extractedData.getResidentCongregateSetting().getValue()),
-                parseYesNoUnk(extractedData.getEmployedInHealthcare().getValue()),
-                null, // preferredLanguage
-                null // testResultDeliveryPreference


Removed these explicit nulls by not specifying them in the builder, but lmk if we want to add them back in.

backend/src/main/java/gov/cdc/usds/simplereport/service/PatientBulkUploadServiceAsync.java

backend/src/main/java/gov/cdc/usds/simplereport/validators/CsvValidatorUtils.java

sonarqubecloud · 2023-10-25T17:30:54Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
2 Code Smells

95.2% Coverage
0.0% Duplication

BobanL

Nice work! 👍

emyl3

Thank you for your work on this!

github-advanced-security bot found potential problems Oct 20, 2023

View reviewed changes

fzhao99 added 3 commits October 20, 2023 13:36

refactor person creation to use builder in bulk upload

8145b40

get notes and gender identity fields working

2e5e814

lint

38a4807

fzhao99 force-pushed the bob/6649-add-notes-and-gender-to-patient-bulk-upload branch from 7f3176c to 38a4807 Compare October 20, 2023 17:36

fzhao99 commented Oct 20, 2023

View reviewed changes

fzhao99 changed the title ~~Bob/6649 add notes and gender to patient bulk upload~~ add notes and gender to patient bulk upload Oct 20, 2023

fzhao99 added 5 commits October 20, 2023 14:59

add tests

c50418f

replace with empty

a6737b1

fix merge conflict

9f696c4

lint

2284a0a

fix export

2dc1268

fzhao99 temporarily deployed to dev2 October 23, 2023 15:08 — with GitHub Actions Inactive

fzhao99 temporarily deployed to dev2 October 23, 2023 15:16 — with GitHub Actions Inactive

fzhao99 marked this pull request as ready for review October 23, 2023 15:32

fzhao99 marked this pull request as draft October 23, 2023 15:32

fzhao99 mentioned this pull request Oct 23, 2023

update guide and sample csv CDCgov/prime-simplereport-site#613

Merged

4 tasks

fzhao99 commented Oct 23, 2023

View reviewed changes

fzhao99 marked this pull request as ready for review October 23, 2023 16:05

fzhao99 requested review from mehansen, BobanL, emyl3 and johanna-skylight October 23, 2023 16:06

remove non-necessary nulls

24760e5

fzhao99 commented Oct 23, 2023

View reviewed changes

fzhao99 temporarily deployed to dev2 October 23, 2023 16:32 — with GitHub Actions Inactive

fzhao99 temporarily deployed to dev2 October 23, 2023 16:40 — with GitHub Actions Inactive

emyl3 requested changes Oct 23, 2023

View reviewed changes

backend/src/main/java/gov/cdc/usds/simplereport/service/PatientBulkUploadServiceAsync.java Outdated Show resolved Hide resolved

handle wrongly-cased gender identity strings in upload

cc36960

fzhao99 requested a review from emyl3 October 23, 2023 18:57

fzhao99 removed the request for review from johanna-skylight October 23, 2023 19:51

fzhao99 marked this pull request as draft October 23, 2023 19:52

fix null pointer

5c5762a

fzhao99 marked this pull request as ready for review October 23, 2023 20:49

fzhao99 requested review from emyl3, BobanL, mehansen and johanna-skylight October 23, 2023 20:49

fzhao99 temporarily deployed to dev2 October 23, 2023 20:56 — with GitHub Actions Inactive

fzhao99 had a problem deploying to dev2 October 23, 2023 21:06 — with GitHub Actions Failure

fzhao99 temporarily deployed to dev2 October 23, 2023 21:12 — with GitHub Actions Inactive

fzhao99 temporarily deployed to dev2 October 23, 2023 21:30 — with GitHub Actions Inactive

fzhao99 temporarily deployed to dev2 October 23, 2023 21:41 — with GitHub Actions Inactive

emyl3 reviewed Oct 25, 2023

View reviewed changes

backend/src/main/java/gov/cdc/usds/simplereport/validators/CsvValidatorUtils.java Outdated Show resolved Hide resolved

fzhao99 added 2 commits October 25, 2023 10:28

move format function to the bulk upload service

7729d0b

be better at software

7c2feb5

fzhao99 force-pushed the bob/6649-add-notes-and-gender-to-patient-bulk-upload branch from fb369e0 to 7c2feb5 Compare October 25, 2023 17:16

fzhao99 requested a review from emyl3 October 25, 2023 17:48

BobanL approved these changes Oct 25, 2023

View reviewed changes

emyl3 approved these changes Oct 26, 2023

View reviewed changes

fzhao99 added this pull request to the merge queue Oct 26, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 26, 2023

fzhao99 added this pull request to the merge queue Oct 26, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 26, 2023

fzhao99 added this pull request to the merge queue Oct 26, 2023

Merged via the queue into main with commit 086a418 Oct 26, 2023

fzhao99 deleted the bob/6649-add-notes-and-gender-to-patient-bulk-upload branch October 26, 2023 15:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add notes and gender to patient bulk upload #6778

add notes and gender to patient bulk upload #6778

fzhao99 commented Oct 20, 2023 •

edited

Loading

fzhao99 Oct 20, 2023 •

edited

Loading

fzhao99 Oct 23, 2023 •

edited

Loading

fzhao99 Oct 23, 2023

sonarqubecloud bot commented Oct 25, 2023

BobanL left a comment

emyl3 left a comment

add notes and gender to patient bulk upload #6778

add notes and gender to patient bulk upload #6778

Conversation

fzhao99 commented Oct 20, 2023 • edited Loading

BACKEND PULL REQUEST

Related Issue

Changes Proposed

Testing

Happy Path

Bad string value for gender identity

Upload without extra columns still works

fzhao99 Oct 20, 2023 • edited Loading

Choose a reason for hiding this comment

fzhao99 Oct 23, 2023 • edited Loading

Choose a reason for hiding this comment

fzhao99 Oct 23, 2023

Choose a reason for hiding this comment

sonarqubecloud bot commented Oct 25, 2023

BobanL left a comment

Choose a reason for hiding this comment

emyl3 left a comment

Choose a reason for hiding this comment

fzhao99 commented Oct 20, 2023 •

edited

Loading

fzhao99 Oct 20, 2023 •

edited

Loading

fzhao99 Oct 23, 2023 •

edited

Loading