Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aa storm feature / alert logic #1420

Draft
wants to merge 51 commits into
base: feature/add-new-AA-storm
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
c0ef39b
create a bash file to be triggered by cron scheduler
Jan 28, 2025
038d34c
create basic file structure
Jan 28, 2025
e69bccd
basic version for the email logic
Jan 28, 2025
6ae7b69
use db to persist latest reports; revisit the logic
Jan 30, 2025
ddd4a08
restructure files for tests; test getLatestAvailableReports()
Jan 31, 2025
46536cf
add more tests
Jan 31, 2025
88e4e7f
add another test
Jan 31, 2025
6210ea7
Merge branch 'feature/add-new-AA-storm' into aa-storm-feature/email-l…
ericboucher Jan 31, 2025
a6aa3bc
Merge branch 'feature/add-new-AA-storm' into aa-storm-feature/email-l…
ericboucher Feb 1, 2025
45d2537
Fix typeorm
ericboucher Feb 2, 2025
0755431
Merge branch 'feature/add-new-AA-storm' into aa-storm-feature/email-l…
ericboucher Feb 2, 2025
60b8902
pass status directly to email utility
Doniaab Feb 3, 2025
46f5831
fix
Doniaab Feb 3, 2025
2ba893e
WIP: update logic to new storage model
Feb 3, 2025
d35e38e
adapt logic to the new db model
Doniaab Feb 3, 2025
4773e5b
build prism url, refacto
Feb 3, 2025
2408407
update lastTriggeredAt only when emails are sent
Feb 4, 2025
798b801
use list of emails instead of single email
Feb 4, 2025
5a420d6
fix date formating
Feb 4, 2025
81e0b35
fix entity type for array; add screenshot settings
Feb 4, 2025
ec514b9
fix windspeed
Feb 4, 2025
0c2affc
rename activated_64 to activated_89
Feb 4, 2025
afd47e3
install chrome and dependencies for puppeteer
Doniaab Feb 4, 2025
ac01fde
responsive padding
Doniaab Feb 6, 2025
5a3382a
add env example
Doniaab Feb 6, 2025
1b1326a
Update email.test.ts
ericboucher Feb 6, 2025
76dbfd8
Fix linter
ericboucher Feb 7, 2025
d6faaa6
Update CI
ericboucher Feb 7, 2025
93c1b8c
Update alerting.yml
ericboucher Feb 7, 2025
f623195
Merge branch 'feature/add-new-AA-storm' into aa-storm-feature/email-l…
ericboucher Feb 7, 2025
27914df
Fix node20 typeorm error
ericboucher Feb 7, 2025
cf9e811
Allow any
ericboucher Feb 7, 2025
4ffd06c
Update alert.test.ts
ericboucher Feb 7, 2025
8979efc
Update alerting.yml
ericboucher Feb 7, 2025
a917b17
Update alerting.yml
ericboucher Feb 7, 2025
e8fc771
avoid blank capture and fix alert tests
Doniaab Feb 10, 2025
2a6ec4f
store common types in common package
Feb 10, 2025
6f33c47
remove useless console.log
Feb 10, 2025
eb1ea64
test filterOutAlreadyProcessedReports()
Feb 11, 2025
ee0b60c
test worker
Feb 11, 2025
52c84a1
add basic readme.md
Feb 11, 2025
b0ac3f7
Merge branch 'feature/add-new-AA-storm' into aa-storm-feature/email-l…
ericboucher Feb 12, 2025
f09087d
Finalize status update
ericboucher Feb 12, 2025
ee94853
Update alert.test.ts.snap
ericboucher Feb 12, 2025
951014b
Update common package
ericboucher Feb 13, 2025
f33d4d3
Merge branch 'feature/add-new-AA-storm' into aa-storm-feature/email-l…
ericboucher Feb 13, 2025
18e540f
Use common types
ericboucher Feb 13, 2025
f9dba54
Remove FeaturePropertyDataType
ericboucher Feb 13, 2025
c235af7
Update frontend.yml
ericboucher Feb 13, 2025
4e056a0
Updates for email alert (design, mailto link (#1432)
Doniaab Feb 13, 2025
50f7a17
add docstring to worker
Doniaab Feb 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions alerting/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
PRISM_ALERTS_EMAIL_HOST=smtp.gmail.com
[email protected]
PRISM_ALERTS_EMAIL_PASSWORD=test

POSTGRES_PORT=5432
POSTGRES_HOST=localhost
POSTGRES_USER=postgres
POSTGRES_DB=postgres
POSTGRES_PASSWORD="!ChangeMe!"
15 changes: 14 additions & 1 deletion alerting/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,21 @@
FROM node:16
FROM node:20

ARG CHROME_VERSION

COPY ./alerting /app
COPY ./common /common

# Install dependencies and Chrome for puppeteer
RUN set -e && \
apt-get update && apt-get install -y \
libnss3 libdbus-1-3 libatk1.0-0 libatk-bridge2.0-0 \
libcups2 libxcomposite1 libxdamage1 libgbm1 libxkbcommon-x11-0 \
libasound2 libxfixes3 libxrandr2 && \
apt-get clean && rm -rf /var/lib/apt/lists/*

RUN npx @puppeteer/browsers install chrome@$CHROME_VERSION && \
npx @puppeteer/browsers install chromedriver@$CHROME_VERSION

WORKDIR /common
RUN yarn --frozen-lockfile

Expand Down
13 changes: 13 additions & 0 deletions alerting/cron_aa_storm_alert_run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/bash
cd ~/prism-app/alerting

# source secrets from AWS
source ../api/set_envs.sh

docker compose run --entrypoint 'yarn aa-storm-alert-worker' alerting-node 2>&1 | tee -a ~/prism-app/alerting/aa_storm_alert_worker.log

## To set up the cron job, run the following command on the server:
# crontab -e
## and then add the following line to the crontab file:
# 0 * * * * ~/prism-app/alerting/cron_aa_storm_alert_run.sh
## This will run the alerting script every hour at minute 0.
2 changes: 1 addition & 1 deletion alerting/cron_alert_run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ source ../api/set_envs.sh
docker compose run --entrypoint 'yarn alert-worker' alerting-node 2>&1 | tee -a ~/prism-app/alerting/alert_worker.log

## To set up the cron job, run the following command on the server:
# contrab -e
# crontab -e
## and then add the following line to the crontab file:
# 0 1 * * * ~/prism-app/alerting/cron_alert_run.sh
## This will run the alerting script every day at 1:00 AM.
2 changes: 2 additions & 0 deletions alerting/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ services:
build:
context: ../
dockerfile: ./alerting/Dockerfile
args:
CHROME_VERSION: "126.0.6478.126"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not set this directly in the dockerfile?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both approaches work, but in this case, defining the Chrome version in docker-compose.yml makes more sense. It simplifies updates if any issues arise and makes it easier to test different versions without rebuilding the image.

restart: always
volumes:
- ../alerting:/app
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import { MigrationInterface, QueryRunner } from 'typeorm';

export class CreateAnticipatoryActionAlertsTable1738249210356
implements MigrationInterface
{
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`CREATE TABLE "anticipatory_action_alerts" (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wadhwamatic we decided to store the alerts in a new table since the behavior was quite different.
This will let us control mailing lists and target urls more easily. We made it so that it can be used for other countries in the future

"id" SERIAL NOT NULL,
"country" VARCHAR NOT NULL,
"emails" VARCHAR[] NOT NULL,
"prism_url" VARCHAR NOT NULL,
"last_triggered_at" TIMESTAMPTZ,
"last_ran_at" TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP,
"last_states" jsonb,
CONSTRAINT "PK_ad91cad659a3536465d564a4b3a" PRIMARY KEY ("id")
)`);
}

public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`DROP TABLE "anticipatory_action_alerts"`);
}
}
4 changes: 2 additions & 2 deletions alerting/ormconfig.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { SnakeNamingStrategy } from 'typeorm-naming-strategies';
import { join } from 'path';
import { DataSourceOptions } from 'typeorm';
import { ConnectionOptions } from 'typeorm';

// dotenv is a dev dependency, so conditionally import it (don't need it in Prod).
try {
Expand Down Expand Up @@ -51,4 +51,4 @@ export = {
migrationsDir: 'migration',
subscribersDir: 'subscriber',
},
} as unknown as DataSourceOptions;
} as unknown as ConnectionOptions;
4 changes: 3 additions & 1 deletion alerting/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
"migration:revert": "yarn typeorm migration:revert",
"typeorm-seeding": "ts-node $(yarn bin typeorm-seeding)",
"alert-worker": "yarn ts-node src/alert-worker.ts",
"aa-storm-alert-worker": "yarn ts-node src/aa-storm-alert-runner.ts",
"docker-alert": "source ../api/set_envs.sh && docker compose run --entrypoint 'yarn alert-worker' alerting-node"
},
"dependencies": {
Expand All @@ -36,13 +37,14 @@
"pg": "^8.5.1",
"prism-common": "file:../common",
"puppeteer": "^24.1.1",
"typeorm": "^0.3.0",
"typeorm": "0.2.31",
"typeorm-naming-strategies": "^1.1.0",
"xml-js": "^1.6.11"
},
"devDependencies": {
"@types/ejs": "^3.1.5",
"@types/jest": "^29.5.12",
"@types/node": "^22.13.0",
"@types/node-fetch": "^2.5.8",
"@types/nodemailer": "^6.4.1",
"faker": "^4.1.0",
Expand Down
3 changes: 3 additions & 0 deletions alerting/src/aa-storm-alert-runner.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import { run } from './aa-storm-alert/worker';

run();
65 changes: 65 additions & 0 deletions alerting/src/aa-storm-alert/__snapshots__/alert.test.ts.snap
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`alert mechanism buildEmailPayloads() does not return a payload when detailed report indicates that activation48 is triggered but there is no pilot activated districts for 48kt exposed 1`] = `Array []`;

exports[`alert mechanism buildEmailPayloads() does not return a payload when detailed report indicates that activation64 is triggered but there is no pilot activated districts for 64kt exposed 1`] = `Array []`;

exports[`alert mechanism buildEmailPayloads() returns a payload when detailed report indicates that activation48 is triggered and pilot activated districts for 48kt winds are parts of the exposed districts 1`] = `
Array [
Object {
"activatedTriggers": Object {
"districts48kt": Array [
"Angoche",
],
"districts64kt": Array [],
"windspeed": "",
},
"base64Image": "",
"cycloneName": "ELVIS",
"cycloneTime": "2025-01-31T06:00:00Z",
"email": "",
"readiness": false,
"redirectUrl": "",
},
]
`;

exports[`alert mechanism buildEmailPayloads() returns a payload when detailed report indicates that activation64 is triggered and pilot activated districts for 64kt winds are parts of the exposed districts 1`] = `
Array [
Object {
"activatedTriggers": Object {
"districts48kt": Array [],
"districts64kt": Array [
"Namacurra",
],
"windspeed": "",
},
"base64Image": "",
"cycloneName": "ELVIS",
"cycloneTime": "2025-01-31T06:00:00Z",
"email": "",
"readiness": false,
"redirectUrl": "",
},
]
`;

exports[`alert mechanism buildEmailPayloads() returns a payload when detailed report indicates that readiness is triggered 1`] = `
Array [
Object {
"activatedTriggers": Object {
"districts48kt": Array [],
"districts64kt": Array [],
"windspeed": "",
},
"base64Image": "",
"cycloneName": "ELVIS",
"cycloneTime": "2025-01-31T06:00:00Z",
"email": "",
"readiness": true,
"redirectUrl": "",
},
]
`;

exports[`alert mechanism buildEmailPayloads() returns no payload when detailed report indicates that activation is triggered but landfall already occured 1`] = `Array []`;
Loading
Loading