Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8s adjustments + fix setup script + more #122

Merged
merged 15 commits into from
Sep 16, 2024
Merged

Conversation

arthurprevot
Copy link
Owner

@arthurprevot arthurprevot commented Sep 16, 2024

K8s

  • new Dockerfile for k8s runs. Not validated.
  • new script to register k8s images to AWS ECR.

Other:

  • fix and improve setup script for EMR
  • improved logs.
  • delete tmp files in ec2 when loading pandas files.
  • add _created_at field in pandas output

Moved to v0.12.2

@arthurprevot arthurprevot merged commit 379c5ef into master Sep 16, 2024
2 checks passed
@arthurprevot arthurprevot deleted the airflow_k8s_adjust branch September 16, 2024 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant