Skip to content

Commit

Permalink
feat: Celery worker concurrency setting
Browse files Browse the repository at this point in the history
This allows the user to configure how many Celery workers are spawned
independently of how many CPUs there are in the system. The default is
to spawn as many workers as there are CPUs, which in some cases can
consume too many resources.

(The setting should be particularly useful to people running Tutor for
development on Linux machines, where reducing the concurrency to "1" can
reduce RAM usage significantly.)
  • Loading branch information
arbrandes committed Mar 1, 2024
1 parent 13c420c commit 0382877
Show file tree
Hide file tree
Showing 6 changed files with 26 additions and 4 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- [Feature] Allow the user to configure Celery worker concurrency. (by @arbrandes)
5 changes: 5 additions & 0 deletions docs/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,11 @@ This defines the version that will be pulled from just the Open edX platform git

By default, there are 2 `uwsgi worker processes <https://uwsgi-docs.readthedocs.io/en/latest/Options.html#processes>`__ to serve requests for the LMS and the CMS. However, each worker requires upwards of 500 Mb of RAM. You should reduce this value to 1 if your computer/server does not have enough memory.

- ``OPENEDX_LMS_CELERY_WORKERS`` (default: ``"0"``)
- ``OPENEDX_CMS_CELERY_WORKERS`` (default: ``"0"``)

This sets the number of Celery workers at startup. Defaults to the number of CPUs detected on the system (represented by "0"). If your system's RAM is limited, you may need to set this to a low integer value in order to conserve it. (On development deployments, it is suggested to use "1" for both.)

- ``OPENEDX_CELERY_REDIS_DB`` (default: ``0``)
- ``OPENEDX_CACHE_REDIS_DB`` (default: ``1``)

Expand Down
14 changes: 14 additions & 0 deletions docs/dev.rst
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,20 @@ Now, use the ``tutor dev ...`` command-line interface to manage the development

Read more about bind-mounts :ref:`below <bind_mounts>`.

Conserving system resources
---------------------------

When developing, you'll usually want the development environment to take up as little system resources as possible. To do so, we recommend setting the following Tutor configuration and relaunching your environment::

tutor config save \
--set OPENEDX_CMS_CELERY_WORKERS=1 \
--set OPENEDX_LMS_CELERY_WORKERS=1 \
--set OPENEDX_CMS_UWSGI_WORKERS=1 \
--set OPENEDX_LMS_UWSGI_WORKERS=1 \
--set ELASTICSEARCH_HEAP_SIZE=100m

This has been known to reduce RAM usage by more than 70%, specially if you're running Docker natively and have a lot of CPU cores.

Stopping the platform
---------------------

Expand Down
2 changes: 2 additions & 0 deletions tutor/templates/config/defaults.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ OPENEDX_AWS_ACCESS_KEY: ""
OPENEDX_AWS_SECRET_ACCESS_KEY: ""
OPENEDX_CACHE_REDIS_DB: 1
OPENEDX_CELERY_REDIS_DB: 0
OPENEDX_CMS_CELERY_WORKERS: 0
OPENEDX_CMS_UWSGI_WORKERS: 2
OPENEDX_LMS_CELERY_WORKERS: 0
OPENEDX_LMS_UWSGI_WORKERS: 2
OPENEDX_MYSQL_DATABASE: "openedx"
OPENEDX_MYSQL_USERNAME: "openedx"
Expand Down
4 changes: 2 additions & 2 deletions tutor/templates/k8s/deployments.yml
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ spec:
containers:
- name: cms-worker
image: {{ DOCKER_IMAGE_OPENEDX }}
args: ["celery", "--app=cms.celery", "worker", "--loglevel=info", "--hostname=edx.cms.core.default.%%h", "--max-tasks-per-child", "100", "--exclude-queues=edx.lms.core.default"]
args: ["celery", "--app=cms.celery", "worker", "--loglevel=info", "--hostname=edx.cms.core.default.%%h", "--concurrency={{ OPENEDX_CMS_CELERY_WORKERS }}", "--max-tasks-per-child", "100", "--exclude-queues=edx.lms.core.default"]
env:
- name: SERVICE_VARIANT
value: cms
Expand Down Expand Up @@ -250,7 +250,7 @@ spec:
containers:
- name: lms-worker
image: {{ DOCKER_IMAGE_OPENEDX }}
args: ["celery", "--app=lms.celery", "worker", "--loglevel=info", "--hostname=edx.lms.core.default.%%h", "--max-tasks-per-child=100", "--exclude-queues=edx.cms.core.default"]
args: ["celery", "--app=lms.celery", "worker", "--loglevel=info", "--hostname=edx.lms.core.default.%%h", "--concurrency={{ OPENEDX_LMS_CELERY_WORKERS }}", "--max-tasks-per-child=100", "--exclude-queues=edx.cms.core.default"]
env:
- name: SERVICE_VARIANT
value: lms
Expand Down
4 changes: 2 additions & 2 deletions tutor/templates/local/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ services:
environment:
SERVICE_VARIANT: lms
DJANGO_SETTINGS_MODULE: lms.envs.tutor.production
command: celery --app=lms.celery worker --loglevel=info --hostname=edx.lms.core.default.%%h --max-tasks-per-child=100 --exclude-queues=edx.cms.core.default
command: celery --app=lms.celery worker --loglevel=info --hostname=edx.lms.core.default.%%h --concurrency={{ OPENEDX_LMS_CELERY_WORKERS }} --max-tasks-per-child=100 --exclude-queues=edx.cms.core.default
restart: unless-stopped
volumes:
- ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
Expand All @@ -177,7 +177,7 @@ services:
environment:
SERVICE_VARIANT: cms
DJANGO_SETTINGS_MODULE: cms.envs.tutor.production
command: celery --app=cms.celery worker --loglevel=info --hostname=edx.cms.core.default.%%h --max-tasks-per-child 100 --exclude-queues=edx.lms.core.default
command: celery --app=cms.celery worker --loglevel=info --hostname=edx.cms.core.default.%%h --concurrency={{ OPENEDX_CMS_CELERY_WORKERS }} --max-tasks-per-child 100 --exclude-queues=edx.lms.core.default
restart: unless-stopped
volumes:
- ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
Expand Down

0 comments on commit 0382877

Please sign in to comment.