Changes report evaluations for MTC from regional to subfeeds #3486
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
In 2023 a decision was made on the reports site to evaluate gtfs checks on MTC sub feed sites like SMART but the checks are on MTC's regional feed.
It's impossible for individual agencies to notice this and SMART has complained that it's not fair for their evaluation to get a fail when it's the regional MTC's gtfs's fault instead.
Resolves #cal-itp/reports#315
Type of change
How has this been tested?
poetry run dbt run --full-refresh --models +fct_daily_organization_combined_guideline_checks
Lots of sql query analysis in staging.
This produces a differential of what is taken out and what stays in:
SELECT * FROM `cal-itp-data-infra-staging.vb_staging.int_gtfs_quality__guideline_checks_long` where -- organization_name = 'Sonoma-Marin Area Rail Transit District' AND date = '2024-05-15' AND check = 'No errors in MobilityData GTFS Schedule Validator' AND organization_key IS NOT NULL and public_customer_facing_fixed_route AND key not in (SELECT key FROM `cal-itp-data-infra-staging.vb_staging.int_gtfs_quality__guideline_checks_long` where -- organization_name = 'Sonoma-Marin Area Rail Transit District' AND date = '2024-05-15' AND check = 'No errors in MobilityData GTFS Schedule Validator' AND organization_key IS NOT NULL AND ((use_subfeed_for_reports and public_customer_facing_or_regional_subfeed_fixed_route) or (not use_subfeed_for_reports and public_customer_facing_fixed_route)))
Post-merge follow-ups
Document any actions that must be taken post-merge to deploy or otherwise implement the changes in this PR (for example, running a full refresh of some incremental model in dbt). If these actions will take more than a few hours after the merge or if they will be completed by someone other than the PR author, please create a dedicated follow-up issue and link it here to track resolution.
Evaluate - This fix makes two agencies disappear from downstream sql tables, but neither of these are on the reports website anyways: