Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(grants-collaboration): expose notes retrieval #3395

Merged
merged 9 commits into from
Aug 22, 2024

Conversation

greg-adams
Copy link
Contributor

Ticket #3205

Description

Created route for notes by grant Id.

Added tests for exposed api route. After discussion with @TylerHendrickson and in an effort to increase data isolation for tests, I updated grants tests to re-seed db between tests. This approach would maintain some seed data for convenience (E.g. users, grants, agencies), but create and rollback records for a particular test. This adds some time to run the tests, but seems OK at least on my machine. I'd like to just rollback records for an individual test, but rollback/transaction approaches don't seem useful here, I'm not sure if a db template would be a faster approach.

Screenshots / Demo Video

Testing

Automated and Unit Tests

  • Added Unit tests

Manual tests for Reviewer

  • Added steps to test feature/functionality manually

Checklist

  • Provided ticket and description
  • Provided screenshots/demo
  • Provided testing information
  • Provided adequate test coverage for all new code
  • Added PR reviewers

@github-actions github-actions bot added enhancement New feature or request javascript Pull requests that update Javascript code labels Aug 15, 2024
Copy link

github-actions bot commented Aug 15, 2024

QA Summary

QA Check Result
🌐 Client Tests
🔗 Server Tests
🤝 E2E Tests
📏 ESLint
🧹 TFLint

Test Coverage

Coverage report for `packages/client`
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🔴 All files 32.6 31.01 29.29 34.23
🔴  src 0 100 100 0
🔴   App.vue 0 100 100 0 2-9
🔴  src/arpa_reporter 0 100 100 0
🔴   App.vue 0 100 100 0 2-13
🟡  ...ter/components 57.57 40 46.8 59.78
🟡   AlertBox.vue 80 75 50 80 13
🔴   ...oadButton.vue 20 0 0 20 2-7,38-67
🟢   ...ileButton.vue 100 50 100 100 5
🟢   ...ttonSmall.vue 100 100 100 100
🟢   ...mplateBtn.vue 100 100 100 100
🟡   ...avigation.vue 67.74 63.63 52.63 67.74 ...13-219,228-235
🔴   StandardForm.vue 42.5 37.5 41.66 45.45 ...24-128,135-157
🟢  ...porter/helpers 84.61 79.48 87.5 84.61
🟢   form-helpers.js 84.21 79.48 85.71 84.21 7,16,25,81-83
🟢   short-uuid.js 100 100 100 100
🔴  ...eporter/router 0 0 0 0
🔴   index.js 0 0 0 0 20-131
🔴  ...reporter/store 3.92 0 2.17 4.12
🔴   index.js 3.92 0 2.17 4.12 11-14,32-261
🔴  ...reporter/views 40.98 25.13 41.37 42.91
🟢   AgenciesView.vue 100 0 100 100 16
🔴   AgencyView.vue 40.74 36.36 50 45.83 53-62,70-96
🔴   HomeView.vue 26.66 34.14 36.36 26.66 36-66,113,137-207
🔴   LoginView.vue 29.62 36.36 20 30.76 1,4,19-34,72-100
🔴   ...plateView.vue 27.02 35.71 45.45 25 ...2,30-37,69-113
🔴   ...ploadView.vue 24.24 16.66 33.33 26.66 1,30-31,116-144
🔴   ...eriodView.vue 48 18.18 75 52.17 64-90
🟡   ...riodsView.vue 57.57 28.57 60 59.37 101,132,149-171
🔴   ...pientView.vue 40 10.52 30.76 41.02 73-93,110-152
🔴   ...ientsView.vue 48.48 8.33 46.66 53.33 ...46,165-190,203
🟡   UploadView.vue 61.53 43.75 62.5 61.36 ...41-442,448-449
🔴   UploadsView.vue 16.66 0 0 18.18 59,110-287
🔴   UserView.vue 46.34 33.33 68.75 48.64 84,97-137
🟡   UsersView.vue 60 7.69 62.5 66.66 91-92,105-107,123
🔴   ...ationView.vue 37.03 18.18 26.66 41.66 ...19,238,246-270
🔴  src/components 40.07 30.76 44.04 40.89
🔴   ...vityTable.vue 18.75 0 28.57 19.35 115-185
🟡   BaseLayout.vue 68.18 53.84 60 68.18 175,222-234
🔴   CopyButton.vue 14.28 0 0 14.28 5,37-49
🔴   GrantsTable.vue 45.03 38.77 45 46.85 ...51-455,463-543
🔴   ...dUploader.vue 24.24 33.33 37.5 24.24 16,25,74-111
🔴   SearchFilter.vue 40.74 28 37.5 40 ...52,64,67,72-82
🔴   UserAvatar.vue 40 25 100 37.5 31-37
🔴  ...ponents/Modals 26.64 43.08 31.91 30.18
🔴   ...anization.vue 17.39 65.62 30 22.22 1-14,149-178
🔴   AddTeam.vue 45.45 55.55 61.53 57.69 ...04,210,222-245
🔴   AddUser.vue 36.66 66.66 60 37.5 ...40,145,148-176
🔴   ...anization.vue 15.38 64.28 14.28 16.66 1-15,58-78
🔴   EditTeam.vue 18.18 26.43 33.33 20.83 ...29,208,216-301
🔴   EditUser.vue 21.05 66.66 25 22.22 1,101-128
🔴   ...ilsLegacy.vue 22.03 0 0 24.07 131,177,205-369
🟡   ImportTeams.vue 50 41.17 50 53.33 28,64-69,81-82
🔴   ImportUsers.vue 42.85 60 40 46.15 29,65-80
🔴   ...archPanel.vue 27.9 15.78 23.52 28.57 ...68-178,211-255
🔴   SearchPanel.vue 21.62 63.26 32 27.58 ...77-380,386-458
🔴  src/helpers 15.2 18.81 16.21 15.57
🟢   constants.js 100 100 100 100
🟢   currency.js 100 100 100 100
🟢   dates.js 100 100 100 100
🔴   fetchApi.js 5.71 16.66 5.88 5.71 9-97
🔴   filters.js 4 0 0 4.54 19-51
🔴   form-helpers.js 0 0 0 0 5-82
🟡   gtag.js 77.77 90 75 77.77 12,51
🔴   ...patWarning.js 0 0 0 0 39-61
🟢  ...s/featureFlags 90.9 100 80 90.9
🟡   index.js 66.66 100 66.66 66.66 8
🟢   utils.js 100 100 100 100
🔴  src/mixin 20 0 28.57 20
🔴   ...zableTable.js 20 0 28.57 20 16-31,36-37,42
🔴  src/router 18.42 16.66 10.52 18.42
🔴   index.js 18.42 16.66 10.52 18.42 ...93-194,198-217
🟢  src/store 100 100 100 100
🟢   index.js 100 100 100 100
🔴  src/store/modules 3.58 0 4.72 3.77
🔴   agencies.js 5.26 100 8.33 5.55 13-70
🔴   alerts.js 20 100 20 20 10-24
🔴   grants.js 1.41 0 1.05 1.49 58-352
🔴   organization.js 33.33 100 33.33 33.33 21-25
🔴   roles.js 20 100 20 25 13-22
🔴   tenants.js 11.11 100 14.28 12.5 13-32
🔴   users.js 2.43 0 4.76 2.5 17-100
🔴  src/views 41.61 33.68 30.62 43.39
🔴   ...orterView.vue 25.58 51.85 18.18 26.82 ...,62,84,109-151
🟡   ...boardView.vue 50 17.64 50 52 89-98,114-125
🔴   ...tailsView.vue 32.25 11.76 14 33.33 ...72-533,539-566
🟢   GrantsView.vue 100 100 100 100
🔴   LoginView.vue 34.61 33.33 22.22 36 1,22,57,109-136
🟡   MyGrantsView.vue 77.77 66.66 66.66 77.77 1,69
🟡   ...ofileView.vue 77.27 75 42.85 77.27 1,32,63,130-134
🟢   NotFoundView.vue 100 100 100 100
🔴   ...tionsView.vue 47.05 57.14 41.66 53.33 ...97-100,114-118
🔴   ...ivityView.vue 46.42 23.8 43.75 46.42 ...01,114,120-134
🔴   TeamsView.vue 44.44 88.88 41.66 53.33 1,58,142,156-163
🟡   UsersView.vue 50 66.66 36.36 53.84 ...16-121,133-139
Coverage report for `packages/server`
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🟡 All files 58.94 51.34 54.4 59.02
🟢  src 81.63 33.33 60 81.63
🟢   configure.js 81.63 33.33 60 81.63 42,61-68,97-99
🟢  src/arpa_reporter 98.75 66.66 100 98.75
🟢   configure.js 97.36 40 100 97.36 36
🟢   environment.js 100 100 100 100
🟢   use-request.js 100 100 100 100
🔴  src/arpa_reporter/db 38.58 32.92 44.44 40.16
🔴   arpa-subrecipients.js 13.15 4.34 15.38 14.28 23-92
🔴   reporting-periods.js 37.2 46.87 40 38.09 46,77-156
🟢   settings.js 100 83.33 100 100 13
🟡   uploads.js 50 28.57 52.38 51.42 18-29,84,99-124,141-150
🔴  src/arpa_reporter/lib 29.57 33.08 34.56 28.46
🟢   arpa-ec-codes.js 100 100 100 100
🔴   audit-report.js 21.44 19.35 24.19 21.32 ...28-529,554-684,732-758
🟡   ensure-async-context.js 75 100 50 100
🟢   format.js 90.62 90 90 91.3 41-42
🟡   log.js 75 50 50 75 13,25
🟡   preconditions.js 66.66 33.33 100 66.66 3
🔴   spreadsheet.js 9.09 0 0 9.09 15-32
🟢   validation-error.js 85.71 100 50 85.71 16
🔴  src/arpa_reporter/routes 40 14.92 14.28 40.6
🔴   agencies.js 22.58 0 0 23.33 13-21,26-53
🟡   application_settings.js 75 100 0 75 10-11
🟡   audit-report.js 68.91 58.33 100 68.91 57-58,64-78,100-116
🟢   exports.js 81.42 83.33 100 81.42 61-75,98-99
🔴   reporting-periods.js 20 0 0 20.43 ...25-137,143-149,154-180
🔴   subrecipients.js 23.8 0 0 23.8 12-13,17-27,31-48,52-63
🔴   uploads.js 28.28 7.89 9.09 29.16 ...33-154,164-166,173-180
🔴   users.js 19.6 0 0 20 15-35,39-44,48-81
🔴  src/arpa_reporter/services 43.47 30.82 46.42 43.73
🔴   generate-arpa-report.js 36.86 2.77 50 37.24 ...-975,984-997,1071-1138
🔴   get-template.js 21.62 0 0 21.62 18-79
🟡   persist-upload.js 68.6 90 69.56 68.67 ...58-200,221-235,273-295
🔴   records.js 20.75 0 11.11 21.15 38-204,221-276
🔴   revalidate-uploads.js 37.5 100 0 37.5 5-14
🔴   validate-upload.js 40.86 51.8 37.03 41.44 ...21,340,362,380-657,672
🟢   validation-rules.js 98.18 90 90.9 100 157,173
🟡  src/db 74.26 71.42 68.42 74.29
🟢   connection.js 100 50 100 100 6
🟢   constants.js 100 100 100 100
🟡   helpers.js 75 83.33 50 75 5,21-22
🟢   index.js 82.4 78.54 82.08 82.35 ...48-1414,1596-1597,1604
🟢   saved_search_migration.js 92 88.23 71.42 93.61 5,69,134
🔴   tenant_creation.js 10.58 2.7 0 11.11 15-40,48-210,220
🔴  src/db/arpa_reporter_db_shims 23.68 0 0 23.68
🔴   agencies.js 22.22 100 0 22.22 11-51
🔴   users.js 25 0 0 25 12-62
🟢  src/lib 86.24 79.44 91.42 86.2
🟢   access-helpers.js 93.54 89.18 100 93.54 96-97,102-103
🟢   agencyImporter.js 90.38 88.46 100 90.19 26,29,35,93-94
🟢   email.js 94.08 81.81 100 93.98 ...53,175-179,226,439-442
🔴   [gost-aws.js](h...[Comment body truncated]

Copy link

github-actions bot commented Aug 15, 2024

Terraform Summary

Step Result
🖌 Terraform Format & Style
⚙️ Terraform Initialization
🤖 Terraform Validation
📖 Terraform Plan

Hint: If "Terraform Format & Style" failed, run terraform fmt -recursive from the terraform/ directory and commit the results.

Output

Validation Output
stdout:
Success! The configuration is valid.


-------------------------------------
stderr:

Plan Summary
CHANGE RESOURCE
update datadog_logs_custom_pipeline.email_pipeline[0]
datadog_logs_metric.gost_ses_email_sending_event_send[0]
module.api.aws_ecs_service.default[0]
module.api.module.grant_digest_scheduled_task.aws_iam_role_policy.default[0]
module.api.module.grant_digest_scheduled_task.aws_scheduler_schedule.default[0]
module.arpa_audit_report.aws_ecs_service.default
module.arpa_treasury_report.aws_ecs_service.default
module.consume_grants.aws_ecs_service.default
module.website.aws_s3_object.deploy-config[0]
module.website.aws_s3_object.origin_dist_artifact["assets/currency-CxCF8RzK.js.map"]
module.website.aws_s3_object.origin_dist_artifact["assets/main-D75rod5K.js.map"]
module.website.aws_s3_object.origin_dist_artifact["assets/style-DJ79OuML.js.map"]
recreate module.api.aws_ecs_task_definition.default[0]
module.arpa_audit_report.aws_ecs_task_definition.consumer
module.arpa_treasury_report.aws_ecs_task_definition.consumer
module.consume_grants.aws_ecs_task_definition.consume_grants

Pusher: @greg-adams, Action: pull_request_target, Workflow: Continuous Integration

Copy link
Member

@TylerHendrickson TylerHendrickson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@greg-adams Left a few comments regarding the import strategy for calling getOrganizationNotesForGrant().

Also wanted to say that I like this testing strategy. As a non-scientific benchmark, it looks like it only adds about 11 seconds to the test suite execution in our GitHub Actions CI job (which is probably slower than most developers' machines). I personally think that's an acceptable tradeoff for the increased test isolation. One other idea I wanted to throw out in case you were interested in experimenting: given that Postgres supports nested transactions, I wonder if it would be possible to achieve similar isolation like this:

  1. Seed the database to whatever extent would be "common" for all tests
  2. Before each test, begin an outer (i.e. root-level) transaction
  3. Run a test
  4. After each test, roll back the transaction that was started in Step 2, bringing the state of the DB back to where it was after Step 1
  5. (Repeat 2-4 for each test)

Thoughts?

Comment on lines 1608 to 1611
function getOrganizationNotesForGrant(...args) {
return grantsCollaboration.getOrganizationNotesForGrant(knex, ...args);
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be cleaner to skip this db/index.js file altogether and just import directly from the lib/grantsCollaboration module in src/routes/grants.js.

Suggested change
function getOrganizationNotesForGrant(...args) {
return grantsCollaboration.getOrganizationNotesForGrant(knex, ...args);
}

@@ -1667,6 +1670,7 @@ module.exports = {
markGrantAsInterested,
unmarkGrantAsInterested,
getGrantAssignedAgencies,
getOrganizationNotesForGrant,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(See above)

Suggested change
getOrganizationNotesForGrant,

return;
}

const rows = await db.getOrganizationNotesForGrant(grantId, user.tenant_id, { afterRevision: paginateFrom, limit: limitInt });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As noted in my db/index.js comments, I think it would be preferable to require('../lib/grantsCollaboration') in this file and call its exported getOrganizationNotesForGrant() function directly rather than going through the db module.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated!

@greg-adams
Copy link
Contributor Author

greg-adams commented Aug 21, 2024

Yes - that is the ideal I wanted...sort of a global rollback that we could use after each test to only apply to changes made during the test. Knex does allow for nested transactions, but my read on it appears that nested transactions need to maintain reference to the originating transaction (to be subsequently rolled back). i.e.

knex.transaction((trx) => {
  // nested dml uses provided arg instead of a new knex.transaction()
  trx('grants').insert(....)
  
  // ...

  // after test
  trx.rollback();
})

I thought through a few approaches (we're not the first to have this issue, see https://github.com/bas080/knest), but some sort of dependency injection seemed too big of a refactor. I could envision a change in the exported knex where we wrap knex.transaction() that returns this global test transaction in tests, or the live one in production but I didn't want to hold up this story.

@TylerHendrickson
Copy link
Member

Yes - that is the ideal I wanted...sort of a global rollback that we could use after each test to only apply to changes made during the test. Knex does allow for nested transactions, but my read on it appears that nested transactions need to maintain reference to the originating transaction (to be subsequently rolled back). i.e.

knex.transaction((trx) => {
  // nested dml uses provided arg instead of a new knex.transaction()
  trx('grants').insert(....)
  
  // ...

  // after test
  trx.rollback();
})

I thought through a few approaches (we're not the first to have this issue, see https://github.com/bas080/knest), but some sort of dependency injection seemed too big of a refactor. I could envision a change in the exported knex where we wrap knex.transaction() that returns this global test transaction in tests, or the live one in production but I didn't want to hold up this story.

Agree with your rationale here – I think your current approach seems good for our needs.

Copy link
Member

@TylerHendrickson TylerHendrickson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@greg-adams Looks good!

@greg-adams greg-adams merged commit 309ba33 into main Aug 22, 2024
19 checks passed
@greg-adams greg-adams deleted the feat/grant-notes-retrieval branch August 22, 2024 22:58
@TylerHendrickson TylerHendrickson added the Grant Finder Issues related to the Grant Finder label Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Grant Finder Issues related to the Grant Finder javascript Pull requests that update Javascript code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants