Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DON'T MERGE - i852 demo deploy test branch #957

Closed
wants to merge 41 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
3316f89
queue derivative jobs in new :resource_intensive queue
bkiahstroud Jan 3, 2024
ab95c9a
attempt to only put video and audio jobs in separate queue
bkiahstroud Jan 4, 2024
cceb544
create child job class to use new Sidekiq queue
bkiahstroud Jan 4, 2024
acb09ad
rubocop fixes in CreateDerivativesJobDecorator
bkiahstroud Jan 4, 2024
70bdf48
remove CreateLargeDerivativesJob priority override
bkiahstroud Jan 4, 2024
debb2f1
new worker to run resource intensive jobs
bkiahstroud Jan 4, 2024
1abe7bd
deploy resource intensive worker with 4x the normal resources
bkiahstroud Jan 4, 2024
23202f1
no need to specify default file location
bkiahstroud Jan 4, 2024
38b4f8a
add code docs
bkiahstroud Jan 4, 2024
817657c
Revert "🩹 Temporary fix for video processing"
bkiahstroud Jan 4, 2024
307236a
Revert "🐛 HACK: Disable mp4 and webm derivative generation"
bkiahstroud Jan 4, 2024
69b4165
helm deployment of intensive worker
orangewolf Jan 5, 2024
47546b6
resource adjustments
orangewolf Jan 5, 2024
e176af1
undo changes to unused file
bkiahstroud Jan 5, 2024
f82c858
use ENV var for intensive worker thread count
bkiahstroud Jan 6, 2024
1c89d52
adjust intensive worker prod resources
bkiahstroud Jan 6, 2024
10e834b
fix disarranged YAML
bkiahstroud Jan 6, 2024
8171b37
update prod to Hyrax helm chart v3.5.1
bkiahstroud Jan 6, 2024
ea6a6e0
Merge branch 'i852-new-worker-to-run-resource-intensive-jobs' into i8…
bkiahstroud Jan 9, 2024
00b4508
Merge branch 'i852-sack-the-hack' into i852-demo-deploy-test-branch
bkiahstroud Jan 9, 2024
70a805b
Merge branch 'main' into i852-demo-deploy-test-branch
bkiahstroud Jan 9, 2024
a4d5347
fix demo redis deploy issue
bkiahstroud Jan 9, 2024
68a681d
Merge branch 'i852-new-worker-to-run-resource-intensive-jobs' into i8…
bkiahstroud Jan 9, 2024
dc7d470
env vars need to be strings
bkiahstroud Jan 17, 2024
0aea9de
env vars need to be strings
bkiahstroud Jan 17, 2024
f763d4d
correctly parse Redis host value
bkiahstroud Jan 24, 2024
f0f90fd
Merge branch 'i852-new-worker-to-run-resource-intensive-jobs' into i8…
bkiahstroud Jan 24, 2024
3576eb8
prevent variable substitution for db-wait step
bkiahstroud Jan 24, 2024
6e632d8
prevent variable substitution for db-wait step
bkiahstroud Jan 24, 2024
f0b45b3
Merge branch 'i852-new-worker-to-run-resource-intensive-jobs' into i8…
bkiahstroud Jan 24, 2024
31e2564
attempt to escape variable substitution again
bkiahstroud Jan 24, 2024
7a4dedb
attempt to escape variable substitution again
bkiahstroud Jan 24, 2024
38642d1
Merge branch 'i852-new-worker-to-run-resource-intensive-jobs' into i8…
bkiahstroud Jan 24, 2024
9998530
use helm to hopefully avoid env variable substitution
bkiahstroud Jan 25, 2024
a7b07b9
use helm to hopefully avoid env variable substitution
bkiahstroud Jan 25, 2024
66476fc
Merge branch 'i852-new-worker-to-run-resource-intensive-jobs' into i8…
bkiahstroud Jan 25, 2024
eb3b2fa
let's try this again...
bkiahstroud Jan 25, 2024
989e5a5
Merge branch 'i852-new-worker-to-run-resource-intensive-jobs' into i8…
bkiahstroud Jan 25, 2024
f07f23b
add redis port
bkiahstroud Jan 25, 2024
4733738
add redis port
bkiahstroud Jan 25, 2024
322f1c9
Merge branch 'i852-new-worker-to-run-resource-intensive-jobs' into i8…
bkiahstroud Jan 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix disarranged YAML
bkiahstroud committed Jan 6, 2024
commit 10e834b78fafac7331e8091565e24abb86cfb300
14 changes: 7 additions & 7 deletions ops/production-deploy.tmpl.yaml
Original file line number Diff line number Diff line change
@@ -198,6 +198,13 @@ worker:
requests:
memory: "2Gi"
cpu: "150m"
extraVolumeMounts: *volMounts
extraEnvVars: *envVars
podSecurityContext:
runAsUser: 1001
runAsGroup: 101
fsGroup: 101
fsGroupChangePolicy: "OnRootMismatch"

workerIntensive:
replicaCount: 1
@@ -214,13 +221,6 @@ workerIntensive:
- name: SIDEKIQ_INTENSIVE_THREAD_COUNT
value: 1

extraVolumeMounts: *volMounts
extraEnvVars: *envVars
podSecurityContext:
runAsUser: 1001
runAsGroup: 101
fsGroup: 101
fsGroupChangePolicy: "OnRootMismatch"
podSecurityContext:
runAsUser: 1001
runAsGroup: 101