Skip to content

Commit

Permalink
feat: Add semaphore to limit concurrent basemaps import workflows BM-822
Browse files Browse the repository at this point in the history
 (#157)

#### Description

Adds a semaphore to limit how many basemaps import workflows can run at
a time.

#### Intention

As we are about to bulk import the georeferenced historical imagery
surveys, it would be beneficial to add a semaphore to limit how many of
these can be run at once.

The limit of 10 is entirely arbitrary: very happy to change it to
whatever it seen as a better value before merging.

#### Checklist
- [ ] Tests updated
- [ ] Docs updated
- [x] Issue linked in Title
  • Loading branch information
Andrew Jacombs authored Aug 23, 2023
1 parent 90bb6b6 commit f7edcac
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 0 deletions.
1 change: 1 addition & 0 deletions config/semaphores.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ data:
standardising: "2" # Limit of how many standardising workflow instances can run at the same time
bulk: "4" # Limit of how many bulk workflow instances can run at the same time
bulkcopy: "8" # Limit of how many publish-copy workflow instances can run at the same time
basemaps_import: "10" # Limit of how many basemaps import workflow instances can run at the same time
5 changes: 5 additions & 0 deletions workflows/basemaps/imagery-import-cogify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ metadata:
spec:
parallelism: 100
entrypoint: main
synchronization:
semaphore:
configMapKeyRef:
name: semaphores
key: basemaps_import
templateDefaults:
container:
imagePullPolicy: Always
Expand Down

0 comments on commit f7edcac

Please sign in to comment.