Speed alert emails #7318
Open
Speed alert emails #7318
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
starswan
force-pushed
the
speed-alert-emails
branch
from
70787d1 to
120c6ec
Compare
starswan
force-pushed
the
speed-alert-emails
branch
from
07333e2 to
4a7fd07
Compare
scruti
reviewed
Dec 3, 2024
Comment on lines
+116
to
118
| config.before(:each, disable_expensive_jobs: true) do | ||
| allow(DisableExpensiveJobs).to receive(:enabled?).and_return(true) | ||
| end |
scruti
reviewed
Dec 3, 2024
| @@ -1,19 +1,20 @@ | |||
| class AlertEmail::Base < ApplicationJob | |||
| MAXIMUM_RESULTS_PER_RUN = 500 | |||
There was a problem hiding this comment.
Is this still needed? I only see this removed call
There was a problem hiding this comment.
It was - I've put the usage back.
scruti
reviewed
Dec 3, 2024
scruti
reviewed
Dec 3, 2024
app/jobs/alert_email/base.rb
Outdated
Comment on lines
13
to
14
| subscriptions.find_each.reject { |sub| already_run_ids.include?(sub.id) }.each do |subscription| | ||
| vacancies = subscription.vacancies_matching default_scope |
There was a problem hiding this comment.
This seems to be looping twice over the subscriptions batch: once for the reject, and a second for finding/alerting.
We could avoid the first loop:
Suggested change
| subscriptions.find_each.reject { |sub| already_run_ids.include?(sub.id) }.each do |subscription| | |
| vacancies = subscription.vacancies_matching default_scope | |
| subscriptions.find_each.each do |subscription| | |
| next if already_run_ids.include?(subscription.id) | |
| vacancies = subscription.vacancies_matching default_scope |
There was a problem hiding this comment.
don't think it does. find_each is a lazy ierator IIRC so this pattern (which is pretty standard) works efficiently.
scruti
reviewed
Dec 3, 2024
Comment on lines
+138
to
+140
| YAML.unsafe_load_file(Rails.root.join("spec/fixtures/polygons.yml")).map(&:attributes).each { |s| LocationPolygon.create!(s) } | ||
| YAML.unsafe_load_file(Rails.root.join("spec/fixtures/liverpool_schools.yml")).map(&:attributes).each { |s| School.create!(s) } | ||
| create(:vacancy, :published_slugged, slug: "liv", contact_number: "0", organisations: [liverpool_school], job_roles: %w[headteacher], visa_sponsorship_available: false, ect_status: :ect_unsuitable, subjects: %w[German], phases: %w[secondary], working_patterns: %w[full_time]) |
There was a problem hiding this comment.
very nice test setup for a complex case! 👏
scruti
approved these changes
Dec 3, 2024
starswan
force-pushed
the
speed-alert-emails
branch
from
62d99d2 to
7055934
Compare
|
Review app deployed to https://teaching-vacancies-review-pr-7318.test.teacherservices.cloud on AKS |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Trello card URL
https://trello.com/c/w1KM928B/1414-refactor-daily-weeklyalerts-job-for-a-high-volume-of-subscriptions
Changes in this PR:
Currently each (of 100k) subscriptions triggers an SQL query for the same (small) subset of recent vacancies. This change turns that on its head - if there are no keyword or location filters, the filtering is now all done in memory on the 'recent' subset (around 2000 for the 'weekly' email).
Screenshots of UI changes:
Before
After
Next steps:
Terraform deployment required?
New development configuration to be shared?