12 14 2020 Tech Team Report

12-14-2020

Logged Tasks

Date	Task	Hours (Main)	Hours (EOLS)	Hours (PII)
7-Dec-2020	Report, meeting, apply xss security update to previewers, test, cherry-pick pid failure logging to 5.1.1, deploy to stage	3
8-Dec-2020	Check 5.2 issues at IQSS, deploy pid reporting scripts to prod, setup crontab tests, initial 5.3 merge	3
9-Dec-2020	Finish ~5.3 merge, #59, debug/fix email dataset version issue #60, (#7467), check 5.2 issues, deploy 5.1.1-qdr4 to prod	5
10-Dec-2020	Further/final 5.3 merge, install payara-5.2020.7, deploy to dev, update metatag module in Drupal	4
13-Dec-2020	Check/merge D8 dependabot issue, prep for stage deploy of 5.3	1

Summary

Drupal

• Metatag module minor update
• Merged security update from dependabot - related to gulp used in managing the theme/not deployed on our machines, so this was mostly to get rid of the warning

Dataverse

• Merged security updates for previewers from GDCC - the underlying issue is that the file description field is not escaped/sanitized in Dataverse except for display, so previewers, when run as a separate page with a header, need to sanitize that field. FWIW: It appears that there are still a few people getting previewers from the QDR repository - used this as an opportunity to suggest again that they switch.
• #59 - changed text from "Replication Data for" to "Data for"
• #60 - investigated issue where emails about publication showed citations as "Draft". The issue was slightly broader in that any changes to creators, title, etc. between versions was also not showing in the citation in the email. The underlying issue was that the email logic retrieved a copy of the dataset version from the db, before the publication transaction completed, so it picked up the draft state and retrieved the title from the last published version, etc. Didn't check all the possible effects/permutations but moved the email to happen in the 'onSuccess' method which runs after the publication transaction completes, which should resolve things. Created IQSS issue/PR (#7467/#7468) to send the solution to them.
• Merged Dataverse changes through 5.3 release
• Added pid failure logging to 5.1.1 branch so it would be available on prod

Operations:

• deployed 5.1.1-qdr4 to stage, prod with pid failure logging
• installed pid logging scripts on prod and setup test cronjobs to fire them
• updated dev to 5.3 - includes an update to payara-5.2020.7 and changes to the way database connections are set up. Confirmed that updates to Dataverses started under glassfish can require an undocumented step (depends on the update process chosen).

##Discussion

• As noted in slack - it looks like 5.3 fixes/changes some of the restrict access issues, and the back-end config changes were less of a problem than I thought they could be, so I think testing/deploying 5.3 directly after 5.1.1 makes more sense than trying to fix up 5.2. Hoping to get things deployed on stage today.
• Re: the change to db connections:
  • This represents a change in domain.xml and following the release instructions, I added our password as an alias, which I think removes the need to have it in a file. The main reason I bring it up is that this overall change here is something that probably affects automating setup
  • The overall intent of this change, and more to come is to update Dataverse to make it easier to make configuration changes when Dataverse is deployed using containers. The basic change is to use a new tech - microprofiles - which I understand as making it configurable as to whether settings are sent through jvm options, db settings, profile files, environment variables, etc. The end result should be that Dataverse could work as is, with a combination of jvm options and db settings updated via API, but it will also be possible to do setup without having to use scripts to call the payara and/or Dataverse APIs, e.g. via files and env. variables, which will make Docker/Kubernetes easier to manage. There are more parts of this coming - that affect mail setup and will provide an alternative to the Dataverse settings API - so there will be some additional config changes coming in the next few releases.

Plans

• Finish deploying pid logging to prod
• 5.3 test/fix as issues are found
• AnnoRep planning/work

Still TBD:

• Drupal 9/composer 2/3