Enable installing latest patch versions for their minor releases of Apache Airflow in CI

[pankajkoti]

In our CI tests, when we create Hatch environments and install dependencies, we currently install specific x.y.0 versions of Apache Airflow. This happens because the constraints files used are tied to those versions, as shown in this script.

As a result, we are restricted from incorporating bug fixes and newer features introduced in subsequent patch releases for those minor versions, limiting the robustness of our CI runs. For instance, this limitation impacts one of our CI tests that executes example DAGs, forcing us to skip the test for Airflow 2.9.x entirely, as seen here.

Enabling the ability to fetch the latest patch versions for the corresponding minor versions of Apache Airflow in our CI would significantly enhance the comprehensiveness and effectiveness of our testing pipeline.