Skip to content

Replace "development" by "cal-itp-data-infra-staging" for AIRFLOW_ENV to match correct values #4439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Replace "development" by "cal-itp-data-infra-staging" for AIRFLOW_ENV to match correct values #4439

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions airflow/.test.env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
AIRFLOW__CORE__EXECUTOR=SequentialExecutor
AIRFLOW__SCHEDULER__CATCHUP_BY_DEFAULT=false
AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT='google-cloud-platform://'
AIRFLOW_ENV=cal-itp-data-infra-staging
CALITP_BUCKET__AGGREGATOR_SCRAPER=gs://calitp-staging-pytest
CALITP_BUCKET__AIRTABLE=gs://calitp-staging-pytest
CALITP_BUCKET__AMPLITUDE_BENEFITS_EVENTS=gs://calitp-staging-pytest
Expand Down
2 changes: 1 addition & 1 deletion airflow/dags/dags.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ def log_failure_to_slack(context):
latest_only=False,
user_defined_macros={
"image_tag": lambda: "development"
if os.environ["AIRFLOW_ENV"] == "development"
if os.environ["AIRFLOW_ENV"] == "cal-itp-data-infra-staging"
else "latest",
"env_var": os.environ.get,
},
Expand Down
2 changes: 1 addition & 1 deletion airflow/dags/download_gtfs_schedule_v2/_email_download_failures.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
# {html_report}
# """ # noqa: E231,E241

# if os.environ["AIRFLOW_ENV"] == "development":
# if os.environ["AIRFLOW_ENV"] == "cal-itp-data-infra-staging":
# print(
# f"Skipping since in development mode! Would have emailed {failures_df.shape[0]} failures."
# )
Expand Down
2 changes: 1 addition & 1 deletion kubernetes/apps/overlays/gtfs-rt-archiver-v3-test/archiver-channel-vars.yaml
View file
Open in desktop
Copy link
Contributor

@lauriemerrell lauriemerrell Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As noted in the ticket, I am just not sure what this env var does in this context, not sure if there's been a functional issue on the test archiver?

Copy link
Contributor Author

@erikamov erikamov Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I haven't work with that yet. I asked about your tests to see if any were related to this on.
We definitely need to test it before merging this change.

Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ kind: ConfigMap
metadata:
name: archiver-channel-vars
data:
AIRFLOW_ENV: development
AIRFLOW_ENV: cal-itp-data-infra-staging
CALITP_BUCKET__GTFS_DOWNLOAD_CONFIG: "gs://test-calitp-gtfs-download-config"
CALITP_BUCKET__GTFS_RT_RAW: "gs://test-calitp-gtfs-rt-raw-v2"
CALITP_HUEY_CONSUMER_WORKERS: "8"
Expand Down
2 changes: 1 addition & 1 deletion packages/calitp-data-infra/tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
import os

os.environ["AIRFLOW_ENV"] = "development"
os.environ["AIRFLOW_ENV"] = "cal-itp-data-infra-staging"
2 changes: 1 addition & 1 deletion services/gtfs-rt-archiver-v3/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ x-gtfs-rt-archiver-v3-common:
&gtfs-rt-archiver-v3-common
build: .
environment:
AIRFLOW_ENV: development
AIRFLOW_ENV: cal-itp-data-infra-staging
CALITP_BUCKET__AIRTABLE: "gs://test-calitp-airtable"
CALITP_BUCKET__GTFS_DOWNLOAD_CONFIG: "gs://test-calitp-gtfs-download-config"
CALITP_BUCKET__GTFS_RT_RAW: "gs://dev-calitp-gtfs-rt-raw"
Expand Down
Loading