11 13 2023 Tech Team Report

11-13-2023

Date	Task	Hours (Main)
6-Nov-2023	Reporting, meeting, all-hands mtg, fix test error in PrivateUrl/signedUrl PR, investigate login history a) broken csv on stage/prod - increase db max_allowed_packet value, b) missing name/mail values due to qdr_sso users-data being deleted instead of transferred - modify old code to make an update from ldap method in drupal; start debugging drupal deploy failure	6
7-Nov-2023	Fix Drupal deployment - multiple small issues, deploy D10 to stage, investigate/fix failure due to code typo, bad login link from Hang On page, blocked css/js access when reg form required, start investigating social login failure	5
8-Nov-2023	Fix social login failure, gb preview issue #10105, persistent Drupal session issues, deploy to dev/stage	4
9-Nov-2023	Run Update from LDAP on stage, coord re: still missing entries, deploy Drupal/Dataverse updates to prod, add Lax samesite checking, remove Google analytics, investigate log errors, workflow for adding anaytics to page.	3
10-Nov-2023	Deploy new Previewers from github, change prod externaltools to use signed Urls, investigate/fix issue with silent failure for long guestbook custom questions, make issue/PR#10117/#10118, update question size on prod, deploy/test code fix on dev.	3

Deployed D10 to stage and then prod, ran Update From LDAP method to restore login-history report info
Deployed recent Dataverse fixes to prod
Deployed signedUrl-based Previewers and new rich HTML previewer in github which makes them available on prod, updated the externltool manifests on prod to call the previewers with signedUrls (the Previewers themselves can use an apiKey or signedUrls).
Investigated/fixed issue with Social logins - setting the cookie samesite setting for Drupal to Strict apparently breaks social logins via Keycloak but not local account logins. Using "Lax" instead, which is recommended by some Drupal modules, allows both.
Finally removed Google Analytics on prod, investigated how often the file is loaded

Fixed additional login-history errors
- Increased the db max_allowed_packet size on stage/prod to allow larger csv to be created, coord w/Seba re fix
- Added an 'Update from LDAP' method to repopulate local Drupal account info (deleting the old qdr_sso module removed it's data w/o repopulating the same info for the qdr_oidc_sso module. Running Update from LDAP fixes that and thereby restores that info in the login-history report.)
Debugged silent Drupal deploy failure. The recent changes caused the deployment to leave the old version in place and still report success. I restored lines in the deploy script that correctly deleted old symbolic links, etc. allowing deployment to succeed.
Fixed old/incorrect login link in the Hang On (access denied) Drupal page

Fix test in PrivateUrlUsers w/ signedUrls PR #10098
Investigate/fix slow guestbook creation issue - Dataverse was reindexing all datasets in the root collection after any change to the collection. Coordinating with Leonid/IQSS, this has been turned off for any operations where it wasn't needed. Further, despite the code marking the indexing as asynchronous in some places, the reindexing was happening before the guestbook creation call returned. That issue has also be fixed for any time the collection still causes reindexing.
Fix download estimate query bug, create issue/PR ##10095/10096
Restore PrivateUrl functionality with OIDC SSO
Investigated/fixed issue with long guestbook questions causing a silent failure - increased the allowed size and fixed the reporting if/when this/other errors occur. (prod allows longer questions, dev has the error reporting fix as part of our next release, also in an issue/PR for IQSS)

In fixing the Drupal deployment, I also re-enabled/fixed the rollback backup of the db - should this stay?
I realized that Dataverse reads the Matomo analytics file from disk for every page load. Is that a performance issue give our setup?

Investigate options for HEAL to participate in SSO
Fix Stata-14 ingest by allowing file inspection during direct upload or adjusting the Stata ingester.
Fix #113 if possible
Matomo - investigate event-level tracking via tag manager, remove non-working google scripts
AnnoRep - explore round-trip, configure auto-start and log rotation
Ops
- check missing globalidcreationdates and fix via /modifyRegistration or alternative
Dataverse
- Make PR for guestbook adding datasetversion fix
- Popup info accessibility - IQSS likes the recommendations from the source I linked to, so this can be implemented along those lines.
QDAS Previewer
- Updates per request
- Investigate writing aux file/previewing lower-sensitivity version and/or other write options
TBD: FRDR Security