Background tasks

[javierdelapuente]

Applicable spec:
ISD181 - Background tasks in 12 factor apps

Overview

This PR implement background services in 12 factor apps, using the paas-app-charmer library.

Services defined in rockcraft.yaml with their name ending in -worker will be injected the same environment variables as the main service and will these background services will run in all units.

Services defined in rockcraft.yaml with their name ending in -scheduler will be injected the same environment variables as the main service, but will only run in the unit ending in name 0. This pod will always exists if there at least one unit (juju does not set .spec.ordinals.start, see here for reference.

The integration test for the PR is complex, as it uses Celery beat and Celery workers. The Celery beat will send tasks to the workers, with an argument that is the Celery beat hostname. The Celery workers will put in a Redis set the hostname of the beat scheduler (received as an argument) and its own hostname in another set. Checking the sets we can see how many schedulers and worker services are working in all the units in the application.

As a simpler alternative, it could be possible to directly check which services are started in each unit and if they contain environment variables.

Documentation will be handled in different PRs/posts.

Rationale

Juju Events Changes

Module Changes

Library Changes

Checklist

• The charm style guide was applied
• The contributing guide was applied
• The changes are compliant with ISD054 - Managing Charm Complexity
• The documentation is generated using src-docs
• The documentation for charmhub is updated
• The PR is tagged with appropriate label (urgent, trivial, complex)

[jdkandersson]

do you think we should be case-insensitive here?

[jdkandersson]

Also here regarding case sensitivity

[jdkandersson]

I saw this change for Flask, Django and go tests, but not for fast API?

[github-actions]

Test coverage for dae8563

Name                                            Stmts   Miss Branch BrPart  Cover   Missing
-------------------------------------------------------------------------------------------
paas_app_charmer/__init__.py                       32     17      0      0    47%   13-14, 19-20, 26-27, 33-37, 43-44, 50-51, 57-58
paas_app_charmer/_gunicorn/__init__.py              0      0      0      0   100%
paas_app_charmer/_gunicorn/charm.py                15      0      0      0   100%
paas_app_charmer/_gunicorn/webserver.py            83      4     16      1    95%   175, 187-193
paas_app_charmer/_gunicorn/workload_config.py       8      0      0      0   100%
paas_app_charmer/_gunicorn/wsgi_app.py             16      0      0      0   100%
paas_app_charmer/app.py                           151      0     64      3    99%   109->exit, 159->165, 332->334
paas_app_charmer/charm.py                         232     25     52      3    89%   33-34, 41-42, 205-206, 208-209, 242-246, 303-305, 359-360, 365, 370, 375, 385, 390, 395, 400, 405, 430
paas_app_charmer/charm_state.py                   108      2     20      2    97%   187, 276
paas_app_charmer/charm_utils.py                    23      0      0      0   100%
paas_app_charmer/database_migration.py             35      0      2      0   100%
paas_app_charmer/databases.py                      25      2     11      1    92%   89-90
paas_app_charmer/django/__init__.py                 2      0      0      0   100%
paas_app_charmer/django/charm.py                   42      4     10      2    88%   44, 104, 119-120
paas_app_charmer/exceptions.py                      5      0      0      0   100%
paas_app_charmer/fastapi/__init__.py                2      0      0      0   100%
paas_app_charmer/fastapi/charm.py                  29      0      0      0   100%
paas_app_charmer/flask/__init__.py                  2      0      0      0   100%
paas_app_charmer/flask/charm.py                    24      0      0      0   100%
paas_app_charmer/go/__init__.py                     2      0      0      0   100%
paas_app_charmer/go/charm.py                       26      0      0      0   100%
paas_app_charmer/observability.py                  24      3      8      1    81%   41->45, 72-76
paas_app_charmer/rabbitmq.py                       78      2     22      5    93%   117->exit, 122->exit, 157->exit, 159-160, 181->175
paas_app_charmer/secret_storage.py                 50      2     16      4    91%   55->54, 56->58, 86, 105
paas_app_charmer/utils.py                          21      4     14      2    71%   30, 32, 52-53
-------------------------------------------------------------------------------------------
TOTAL                                            1035     65    235     24    92%

Static code analysis report

Run started:2024-09-20 07:59:51.839447

Test results:
  No issues identified.

Code scanned:
  Total lines of code: 2241
  Total lines skipped (#nosec): 1
  Total potential issues skipped due to specifically being disabled (e.g., #nosec BXXX): 0

Run metrics:
  Total issues (by severity):
  	Undefined: 0
  	Low: 0
  	Medium: 0
  	High: 0
  Total issues (by confidence):
  	Undefined: 0
  	Low: 0
  	Medium: 0
  	High: 0
Files skipped (0):