Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
) This reverts commit f71d082. *Issue #, if available:* N/A *Description of changes:* Airflow startup is failing with: ``` mwaa-2101-scheduler | [2024-11-27T19:46:26.107+0000] {cmd.py:85} INFO - File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/models/connection.py", line 379, in set_extra mwaa-2101-scheduler | [2024-11-27T19:46:26.107+0000] {cmd.py:85} INFO - fernet = get_fernet() mwaa-2101-scheduler | [2024-11-27T19:46:26.108+0000] {cmd.py:85} INFO - ^^^^^^^^^^^^ mwaa-2101-scheduler | [2024-11-27T19:46:26.108+0000] {cmd.py:85} INFO - File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/models/crypto.py", line 90, in get_fernet mwaa-2101-scheduler | [2024-11-27T19:46:26.108+0000] {cmd.py:85} INFO - raise AirflowException(f"Could not create Fernet object: {value_error}") mwaa-2101-scheduler | [2024-11-27T19:46:26.108+0000] {cmd.py:85} INFO - airflow.exceptions.AirflowException: Could not create Fernet object: Fernet key must be 32 url-safe base64-encoded bytes. ``` If an airflow connection defines the `extra` field, it goes through an optional encoding/decoding step if a fernet key is defined [[reference](https://github.com/apache/airflow/blob/main/airflow/models/connection.py#L335)]. With #182, we are loading default provider connects, of which several define `extra`'s. This presents an issue when running the MWAA images locally, as we define a dummy fernet key in the docker compose [[reference](https://github.com/aws/amazon-mwaa-docker-images/blob/f71d082cac983385349f3e30b8a650d5f5c5d619/images/airflow/2.9.2/docker-compose.yaml#L23)] which is invalid as shown by the stack trace above. Reverting default connections loading until we implement a check for local image vs. mwaa environment. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
- Loading branch information