Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

operations: Thanos replicate as a CronJob #597

Merged
merged 1 commit into from
Sep 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions resources/operations/bucket-replicate/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# What

This template deploys [Thanos Bucket Inspect](https://thanos.io/tip/components/tools.md/#bucket-insepct)
as a Kubernetes Job.
This template deploys [Thanos Bucket Replicate](https://thanos.io/tip/components/tools.md/#bucket-replicate)
as a Kubernetes Job or CronJob.

# SOP

Expand Down Expand Up @@ -49,3 +49,7 @@ oc process -f job-template.yaml | oc apply -f -
```


Alternatively, you can run it as a CronJob
```bash
oc process -f cron-job-template.yaml | oc apply -f -
```
107 changes: 107 additions & 0 deletions resources/operations/bucket-replicate/cron-job-template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
apiVersion: template.openshift.io/v1
kind: Template
metadata:
name: rhobs-thanos-bucket-replicate-cron
labels:
app.kubernetes.io/name: thanos-bucket-replicate
app.kubernetes.io/part-of: observatorium
description: |
Replicate data between object storage provider buckets on a schedule
parameters:
- name: NAME
description: The name of the CronJob.
value: 'thanos-bucket-replicate'
- name: NAMESPACE
description: The namespace where the Job should run.
value: 'observatorium-operations'
- name: SOURCE_OBJ_STORE_CONFIG_SECRET_NAME
value: 'thanos-bucket-replicate-config-source'
- name: DESTINATION_OBJ_STORE_CONFIG_SECRET_NAME
value: 'thanos-bucket-replicate-config-destination'
- name: SOURCE_OBJ_STORE_K8S_SECRET_KEY
value: 'config.yaml'
- name: DESTINATION_OBJ_STORE_K8S_SECRET_KEY
value: 'config.yaml'
- name: COMPACTION_MIN
value: '0'
- name: COMPACTION_MAX
value: '100'
- name: MIN_TIME
value: '0000-01-01T00:00:00Z'
- name: MAX_TIME
value: '9999-12-31T23:59:59Z'
- name: SCHEDULE
description: The schedule for the Job to run. Defaults to every 2 hours.
value: '0 */2 * * *'
- name: TENANT_ID
value: 'rhobs'
- name: IMAGE_TAG
value: 'v0.32.2'
- name: LOG_LEVEL
value: 'info'
- name: CPU_REQUEST
value: '1'
- name: CPU_LIMIT
value: '2'
- name: MEMORY_REQUEST
value: '500Mi'
- name: MEMORY_LIMIT
value: '3Gi'
objects:
- apiVersion: batch/v1
kind: CronJob
metadata:
name: ${NAME}
namespace: ${NAMESPACE}
labels:
app.kubernetes.io/name: thanos-bucket-inspect
app.kubernetes.io/part-of: observatorium
spec:
schedule: ${SCHEDULE}
concurrencyPolicy: Forbid
jobTemplate:
spec:
template:
spec:
containers:
- name: thanos-bucket-replicate
image: quay.io/thanos/thanos:${IMAGE_TAG}
resources:
requests:
memory: ${MEMORY_REQUEST}
cpu: ${CPU_REQUEST}
limits:
memory: ${MEMORY_LIMIT}
cpu: ${CPU_LIMIT}
ports:
- containerPort: 10902
name: metrics
volumeMounts:
- name: obj-store-from-config
readOnly: true
mountPath: "/var/lib/thanos/bucket-replicate-config/from"
- name: obj-store-to-config
readOnly: true
mountPath: "/var/lib/thanos/bucket-replicate-config/to"
args:
- 'tools'
- 'bucket'
- 'replicate'
- '--log.level=${LOG_LEVEL}'
- '--objstore.config-file=/var/lib/thanos/bucket-replicate-config/from/${SOURCE_OBJ_STORE_K8S_SECRET_KEY}'
- '--objstore-to.config-file=/var/lib/thanos/bucket-replicate-config/to/${DESTINATION_OBJ_STORE_K8S_SECRET_KEY}'
- '--single-run'
- '--matcher=tenant_id="${TENANT_ID}"'
- '--min-time=${MIN_TIME}'
- '--max-time=${MAX_TIME}'
- '--compaction-min=${COMPACTION_MIN}'
- '--compaction-max=${COMPACTION_MAX}'
restartPolicy: Never
volumes:
- name: obj-store-from-config
secret:
secretName: ${SOURCE_OBJ_STORE_CONFIG_SECRET_NAME}
- name: obj-store-to-config
secret:
secretName: ${DESTINATION_OBJ_STORE_CONFIG_SECRET_NAME}

Loading