test-plan/git-lite/submitter

[bwalsh]

Epic

As a release manager , I want a test script to ensure comprehensive and repeatable testing of the new feature(s).

Use case

As a testing engineer, I want a data submission, validation, and upload script to ensure accurate and secure processing of user-submitted data.

Definition of Done:

• Test script is created and documented.
• The script is reviewed and approved by the testing team.
• optional The script is integrated into the testing process and automated frameworks.
• script is executed and all acceptance criteria are met.

Considerations

• The system should generate clear error messages for users in case of invalid data submissions, guiding them on how to correct the issues.
• It should perform thorough validation of data integrity to prevent corruption or loss during the upload process.
• Security measures should be implemented to protect against potential data breaches or unauthorized access during the submission and upload process.
• It must log relevant information, including successful uploads and any errors encountered, for auditing and debugging purposes.
• The system should be version-controlled to track changes and updates over time.
• The validation and upload process should be easily integrable into automated testing frameworks for continuous integration.

Script

submitter test script

# Use case: As a data submitter, I will need to create a project.
## test should work with or without environment variables
#export G3T_PROFILE=local
#export G3T_PROJECT_ID=ohsu-test002b
#g3t init
unset G3T_PROJECT_ID
unset G3T_PROFILE
g3t --profile local init ohsu-test001b

# Use case: As a institution data steward, I need to approve the project before it can be shared.
g3t utilities access sign

# Use case: As a ACED administrator, I need to create projects in sheepdog so that submissions can take place.
g3t utilities projects ls
## test: the project should be listed as incomplete
g3t utilities projects create
## test: the project should be listed as complete

# Use case: As a data submitter, I will need to add files to the project and associate them with a subject(patient).
g3t add tests/fixtures/dir_to_study/file-1.txt  --patient P1
g3t utilities meta create
## test meta generation:  META should have 4 files
g3t commit  -m "commit-1"
## test that the commit: g3t status should return commit info - was message added?
#  resource_counts:
#      DocumentReference: 1
#      Patient: 1
#      ResearchStudy: 1
#     ResearchSubject: 1

# Use case: when subjects are added to study I need to add them to the project.
g3t add tests/fixtures/dir_to_study/file-2.csv  --patient P2
g3t status
## test add: should return one entry in "uncommitted_manifest:"
g3t utilities meta create
## test meta generation:  META should have 4 files Patient ResearchSubject DocumentReference should have 1 new record each
g3t commit -m "commit-2"
## test the commit: g3t status should return commit info - was message added? there should only be the three new records
#    resource_counts:
#      DocumentReference: 1
#      Patient: 1
#      ResearchSubject: 1
#    manifest_files:
#    - tests/fixtures/dir_to_study/file-2.csv

# Use case: some subjects have specimens, I need to add them to the project.
g3t add tests/fixtures/dir_to_study/sub-dir/file-3.pdf --patient P3 --specimen S3
g3t utilities meta create
## test should create a Specimen.ndjson file in META
# Created 4 new records.
wc -l META/Specimen.ndjson
#       1 META/Specimen.ndjson
g3t diff
## test diff: should show new records
g3t commit -m "commit-3"
## test the commit: g3t status should return commit info - was message added? 4 new records
#    message: commit-3
#    resource_counts:
#      DocumentReference: 1
#      Patient: 1
#      ResearchSubject: 1
#      Specimen: 1
#    manifest_files:
#    - tests/fixtures/dir_to_study/sub-dir/file-3.pdf

# Use case: I'm ready to share my data
## push to remote
g3t push
## test:  the system should respond with reasonable, informative messages without too much verbosity
## I need to know the status of my project. During job execution, I should be able to query the status.
g3t status
## test: After job execution, I should have detailed information about the results.
#  pushed_commits:
#  - published_timestamp: 2024-01-19T09:45:47.018426
#    published_job:
#      output:
#        uid: 82322961-8d2a-47e4-8833-af0e299aa393
#        name: fhir-import-export-ohiwi
#        status: Completed
#    commits:
#    - d050c8f931bab152279ff18e0a21434f commit-1
#    - 2f77cf6017ec3b0485b7493ebe459f53 commit-2
#    - a550281b43713937ce684e3cab13639f commit-3

## test: Once complete, the remote counts should reconcile with my activity
#remote:
#  resource_counts:
#    DocumentReference: 3
#    Patient: 3
#    ResearchStudy: 1
#    ResearchSubject: 3
#    Specimen: 1
wc -l META/*.ndjson
#       3 META/DocumentReference.ndjson
#       3 META/Patient.ndjson
#       1 META/ResearchStudy.ndjson
#       3 META/ResearchSubject.ndjson
#       1 META/Specimen.ndjson

## If I want more detailed information, I should be able to query it
## get UID from status -> local.pushed_commits.published_job.output.uid
g3t utilities jobs get UID
# ....


# Use case: As a data submitter, when I know more about meta, I should be able to add it.
# e.g. alter a patient record
sed -i.bak 's/"P1"}]}/"P1"}], "gender": "male"}/' META/Patient.ndjson
# see https://stackoverflow.com/a/22084103
rm META/Patient.ndjson.bak
g3t diff
## test diff: should show changed records
g3t commit -m "commit-4"
## test: the commit should process only one patient record
#resource_counts:
#  Patient: 1

## Use case: I should be able to publish a 'meta only' change
g3t push

## Use case: As a human being, I make mistakes, the system should prevent me from committing `no changes`
g3t commit -m "commit-5 has no changes"
## test: the system should reject the commit
# msg: No resources changed in META

## Use case: As a human being, I make mistakes, the system should prevent me from committing `invalid fhir`
sed -i.bak 's/"gender"/"foobar"/' META/Patient.ndjson
# see https://stackoverflow.com/a/22084103
rm META/Patient.ndjson.bak
g3t commit -m "commit-6 has invalid fhir"
## test: should fail validation, the response should be informative and give me enough information to fix the problem