05.09.2022 TT Agenda

Admin:

• First week of the month - Seba submit hours
• New contracts are in process at QDR - should be ready for you to sign soon

Infrastructure:

• Outrage discussion --> restart had deployed old postgres 9.6 db on default port - changed to let postgres 13 have that port, restarted dbs, restarted DV
• We need to make sure there is an immediate rollback available

Operations: Jim Report

---

Notes:

• Flyway scripts all reverted
• Seba will remove / disable all 9.6 references (it shouldn't be able to do that 👎)
• Seba will spend 1 hour on a root cause analysis ... Will look into backup process to make
• And do we know why prod restarted?
• • Pagerduty worked ...

When restoring from AWS backups

• We are starting back up from a new machine
• The old data / config files are not lost - this is just no longer the instance that DNS for prod is pointed at...
• Being conservative to roll-backs makes sense, but we should have a playbook and documented policies for making decisions about when PROD is done...
• Longer retention of logs (once we get more than 6mb they stop being logged)
• • Right now we have an alert set at 80%

Operations:

• Python 3.9 installed on all machines ...
• Solr update across all machines (08.11.1

Question:

• Can you facet filter by a license?
• • YES - solves a good use case for QDR