Integration between nomad-pipeline and apache airflow

[dob3001]

It might be worth considering some type of integration between nomad-pipeline and airflow...

This is the k8s provider for airflow.

https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/index.html

This could potentially mean that airflow can use nomad/nomad-pipeline to create DAGs etc. I have not specifically checked the functional ask of k8s so I'm assuming it's not too different in what we can do in nomad/nomad-pipeline.

[dob3001]

https://github.com/apache/airflow/tree/main/airflow/providers/hashicorp <- there is one for vault integration

[jsp-exhashi]

Any update for this integration request?

[waquidvp]

Currently I am not sure about what the integration with Airflow would look like. I am open to suggestions and would be happy to implement.

[mSALDANHAf]

Currently I am not sure about what the integration with Airflow would look like. I am open to suggestions and would be happy to implement.

Mainly the hability to run jobs (usually of batch type)