Sending monthly email alerts

This process is janky in various ways and we would ideally like to automate it but for now this are the steps we go through.

ssh largeweb2.ebmdatalab.net

# Create a new screen session named like "evansd-20240802-alerts"
screen -S $(whoami)-$(date +%Y%m%d)-alerts

# Other instructions may tell you to use the "hello" user. That will probably work OK, but it makes
# more sense to run as the bot which does other similar tasks.
sudo su bennettbot

cd /webapps/openprescribing/openprescribing/
source ../.venv/bin/activate

./manage.py send_monthly_alerts

Important notes

• The process can take many hours (more than 24) to run because of the number of relatively slow chart images it generates.
• You will see a number of errors appearing in the console (see example below). These are of two sorts:
  • Transient errors due to timeouts when generating chart images.
  • Permanent errors due to charts which reference things (drugs, practices) which no longer exist. (In an ideal world we'd identify and remove the permanent failures, but the references here are buried in an encoded URL and so doing so is slightly more work than I've ever had time for.)
• Instead of dealing with these transient and permanent errors properly we take a simpler approach: after the command has finished we run it again. If the number of errors doesn't go down then we assume they're all permanent errors and call it a day.
• This is safe to do because the command will never send the same email twice.
• However, try to avoid killing the command mid-way through because it may fail to send some emails at all.
• Once you're done, exit the screen session (which requires pressing Ctrl-D twice: once to exit the su shell and then again to exit screen).

Example error

When a chart fails to render (either because of a transient or persistent error) this will generally manifest as a timeout from PhantomJS which looks something like:

Failed to send <SearchBookmark: Bookmark: Items for SMA Alfamino powder by NHS LIVERPOOL CCG and other CCGs>
phantomjs command failed with code 1

cmd:
/usr/local/bin/phantomjs --ignore-ssl-errors=true /webapps/openprescribing/openprescribing/frontend/management/commands/grab_chart.js "https://openprescribing.net/analyse/#org=CCG&orgIds=99A&numIds=090900000BBKWA0&denom=nothing&selectedTab=summary&source=button" /mnt/database/tmp/tmpdehyt1lb.png "#results .tab-pane.active" 800x600 500

stdout:
Error waiting for element #results .tab-pane.active


stderr: