Skip to content

Commit

Permalink
Add project boilerplate, and README
Browse files Browse the repository at this point in the history
  • Loading branch information
hammerhead authored and amotl committed Jun 27, 2023
1 parent 58ad0c6 commit 871810e
Show file tree
Hide file tree
Showing 4 changed files with 50 additions and 0 deletions.
6 changes: 6 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Each line is a file pattern followed by one or more owners.
# These owners will be the default owners for everything in
# the repo. Unless a later match takes precedence,
# The users listed below are global owners and will be
# requested for review when someone opens a pull request.
* @marijaselakovic @hammerhead
17 changes: 17 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
version: 2

updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "monthly"

- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "monthly"

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.venv*
*.egg-info
.eggs
__pycache__
23 changes: 23 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# crate-airflow-tutorial
Orchestration Project - Astronomer/Airflow tutorials


## About

This repository contains a few examples of Apache Airflow DAGs for automating recurrent
queries. All DAGs run on Astronomer infrastructure installed on Ubuntu 20.04.3 LTS.

In the [dags](dags) directory you can find a subset of our DAG examples.
Each DAG is accompanied by a tutorial:

* [data_retention_delete_dag.py](dags/data_retention_delete_dag.py) ([tutorial](https://community.crate.io/t/cratedb-and-apache-airflow-implementation-of-data-retention-policy/913)): implements a retention policy algorithm that drops expired partitions
* [data_retention_reallocate_dag.py](dags/data_retention_reallocate_dag.py) ([tutorial](https://community.crate.io/t/cratedb-and-apache-airflow-building-a-hot-cold-storage-data-retention-policy/934)): implements a retention policy algorithm that reallocates expired partitions from hot nodes to cold nodes
* [data_retention_snapshot_dag.py](dags/data_retention_snapshot_dag.py): implements a retention policy algorithm that snapshots expired partitions to a repository


## Testing

```shell
pip install --editable=.[develop,test]
poe check
```

0 comments on commit 871810e

Please sign in to comment.