Skip to content

Commit

Permalink
Merge branch '5.0' into start-component
Browse files Browse the repository at this point in the history
  • Loading branch information
agronholm authored May 2, 2024
2 parents ec1331e + 9047a25 commit ada1369
Showing 1 changed file with 66 additions and 59 deletions.
125 changes: 66 additions & 59 deletions docs/userguide/deployment.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ Configuration and deployment

.. py:currentmodule:: asphalt.core
As your application grows more complex, you may find that you need to have different settings for
your development environment and your production environment. You may even have multiple
deployments that all need their own custom configuration.
As your application grows more complex, you may find that you need to have different
settings for your development environment and your production environment. You may even
have multiple deployments that all need their own custom configuration.

For this purpose, Asphalt provides a command line interface that will read a YAML_ formatted
configuration file and run the application it describes.
For this purpose, Asphalt provides a command line interface that will read a YAML_
formatted configuration file and run the application it describes.

.. _YAML: https://yaml.org/

Expand Down Expand Up @@ -43,7 +43,8 @@ Writing a configuration file

A production-ready configuration file should contain at least the following options:

* ``component``: a dictionary containing the class name and keyword arguments for its constructor
* ``component``: a dictionary containing the class name and keyword arguments for its
initializer
* ``logging``: a dictionary to be passed to :func:`logging.config.dictConfig`

Suppose you had the following component class as your root component::
Expand All @@ -53,10 +54,10 @@ Suppose you had the following component class as your root component::
super().__init__(components)
self.data_directory = data_directory

async def start(ctx):
async def start() -> None:
self.add_component('mailer', backend='smtp')
self.add_component('sqlalchemy')
await super().start(ctx)
await super().start()

You could then write a configuration file like this::

Expand Down Expand Up @@ -90,32 +91,33 @@ In the above configuration you have three top level configuration keys: ``max_th
``component`` and ``logging``, all of which are directly passed to
:func:`run_application` as keyword arguments.

The ``component`` section defines the type of the root component using the specially processed
``type`` option. You can either specify a setuptools entry point name (from the
``asphalt.components`` namespace) or a text reference like ``module:class`` (see
The ``component`` section defines the type of the root component using the specially
processed ``type`` option. You can either specify a setuptools entry point name (from
the ``asphalt.components`` namespace) or a text reference like ``module:class`` (see
:func:`resolve_reference` for details). The rest of the keys in this section are
passed directly to the constructor of the ``MyRootComponent`` class.

The ``components`` section within ``component`` is processed in a similar fashion.
Each subsection here is a component type alias and its keys and values are the constructor
arguments to the relevant component class. The per-component configuration values are merged with
those provided in the ``start()`` method of ``MyRootComponent``. See the next section for a more
elaborate explanation.
Each subsection here is a component type alias and its keys and values are the
constructor arguments to the relevant component class. The per-component configuration
values are merged with those provided in the ``start()`` method of ``MyRootComponent``.
See the next section for a more elaborate explanation.

With ``max_threads: 20``, the maximum number of threads in the event loop's default thread pool
executor is set to 20.
With ``max_threads: 20``, the maximum number of threads that functions like
:func:`anyio.to_thread.run_sync` can have running, to 20.

The ``logging`` configuration tree here sets up a root logger that prints all log entries of at
least ``INFO`` level to the console. You may want to set up more granular logging in your own
configuration file. See the
:ref:`Python standard library documentation <python:logging-config-dictschema>` for details.
The ``logging`` configuration tree here sets up a root logger that prints all log
entries of at least ``INFO`` level to the console. You may want to set up more granular
logging in your own configuration file. See the
:ref:`Python standard library documentation <python:logging-config-dictschema>` for
details.

Using data from environment variables and files
-----------------------------------------------

Many deployment environments (Kubernetes, Docker Swarm, Heroku, etc.) require applications to input
configuration values and/or secrets using environment variables or external files. To support this,
Asphalt extends the YAML parser with three custom tags:
Many deployment environments (Kubernetes, Docker Swarm, Heroku, etc.) require
applications to input configuration values and/or secrets using environment variables or
external files. To support this, Asphalt extends the YAML parser with three custom tags:

* ``!Env``: substitute with the value of an environment variable
* ``!TextFile`` substitute with the contents of a (UTF-8 encoded) text file (as ``str``)
Expand All @@ -138,8 +140,9 @@ If a file path contains spaces, you can just quote it::
type: !!python/name:myproject.MyRootComponent
param_from_text_file: !TextFile "/path with spaces/to/file.txt"

.. note:: This does **not** allow you to include other YAML documents as part of the configuration,
except as text/binary blobs. See the next section if this is what you want.
.. note:: This does **not** allow you to include other YAML documents as part of the
configuration, except as text/binary blobs. See the next section if this is what you
want.

.. versionadded:: 4.5.0

Expand All @@ -154,31 +157,33 @@ Component configuration can be specified on several levels:
* ...
* Command line configuration options to ``asphalt run --set``

Any options you specify on each level override or augment any options given on previous levels.
The command line configuration options have precedence over the configuration files.
To minimize the effort required to build a working configuration file for your application, it is
suggested that you pass as many of the options directly in the component initialization code and
leave only deployment specific options like API keys, access credentials and such to the
configuration file.
Any options you specify on each level override or augment any options given on previous
levels. The command line configuration options have precedence over the configuration
files. To minimize the effort required to build a working configuration file for your
application, it is suggested that you pass as many of the options directly in the
component initialization code and leave only deployment specific options like API keys,
access credentials and such to the configuration file.

With the configuration presented in the earlier paragraphs, the ``mailer`` component's constructor
gets passed three keyword arguments:
With the configuration presented in the earlier paragraphs, the ``mailer`` component's
constructor gets passed three keyword arguments:

* ``backend='smtp'``
* ``host='smtp.mycompany.com'``
* ``ssl=True``

The first one is provided in the root component code while the other two options come from the YAML
file. You could also override the mailer backend in the configuration file if you wanted, or at the
command line (with the configuration file saved as ``config.yaml``):
The first one is provided in the root component code while the other two options come
from the YAML file. You could also override the mailer backend in the configuration file
if you wanted, or at the command line (with the configuration file saved as
``config.yaml``):

.. code-block:: bash
asphalt run config.yaml --set component.components.mailer.backend=sendmail
.. note::
Note that if you want a ``.`` to be treated as part of an identifier, and not as a separator,
you need to escape it at the command line with ``\``. For instance, in both commands:
Note that if you want a ``.`` to be treated as part of an identifier, and not as a
separator, you need to escape it at the command line with ``\``. For instance, in
both commands:

.. code-block:: bash
Expand All @@ -187,24 +192,26 @@ command line (with the configuration file saved as ``config.yaml``):
The logging level for the ``asphalt.templating`` logger will be set to ``DEBUG``.

The same effect can be achieved programmatically by supplying the override configuration to the
container component via its ``components`` constructor argument. This is very useful when writing
tests against your application. For example, you might want to use the ``mock`` mailer in your test
suite configuration to test that the application correctly sends out emails (and to prevent them
from actually being sent to recipients!).
The same effect can be achieved programmatically by supplying the override configuration
to the container component via its ``components`` constructor argument. This is very
useful when writing tests against your application. For example, you might want to use
the ``mock`` mailer in your test suite configuration to test that the application
correctly sends out emails (and to prevent them from actually being sent to
recipients!).

Defining multiple services
--------------------------

.. versionadded:: 4.1.0

Sometimes it may be more convenient to use a single configuration file for launching your
application with different configurations or entry points. To this end, the runner supports the
notion of "service definitions" in the configuration file. This is done by replacing the
``component`` dictionary with a ``services`` dictionary at the top level of the configuration file
and either setting the ``ASPHALT_SERVICE`` environment variable or by passing the ``--service``
(or ``-s``) option when launching the runner. This approach provides the additional advantage of
allowing the use of YAML references, like so::
Sometimes it may be more convenient to use a single configuration file for launching
your application with different configurations or entry points. To this end, the runner
supports the notion of "service definitions" in the configuration file. This is done by
replacing the ``component`` dictionary with a ``services`` dictionary at the top level
of the configuration file and either setting the ``ASPHALT_SERVICE`` environment
variable or by passing the ``--service`` (or ``-s``) option when launching the runner.
This approach provides the additional advantage of allowing the use of YAML references,
like so::

---
services:
Expand All @@ -231,9 +238,9 @@ allowing the use of YAML references, like so::
auth_id: clientuser
auth_secret: clientpass

Each section under ``services`` is like its own distinct top level configuration. Additionally, the
keys under each service are merged with any top level configuration, so you can, for example,
define a logging configuration there.
Each section under ``services`` is like its own distinct top level configuration.
Additionally, the keys under each service are merged with any top level configuration,
so you can, for example, define a logging configuration there.

Now, to run the ``server`` service, do:

Expand All @@ -247,11 +254,11 @@ The ``client`` service is run in the same fashion:
asphalt run -s client config.yaml
You can also define a service with a special name, ``default``, which is used in case multiple
services are present and no service has been explicitly selected.
You can also define a service with a special name, ``default``, which is used in case
multiple services are present and no service has been explicitly selected.

.. note:: The ``-s/--service`` command line switch overrides the ``ASPHALT_SERVICE`` environment
variable.
.. note:: The ``-s/--service`` command line switch overrides the ``ASPHALT_SERVICE``
environment variable.

Performance tuning
------------------
Expand All @@ -264,6 +271,6 @@ Add the following piece to your application's configuration:
.. code-block:: yaml
backend_options:
use_uvloop: true
use_uvloop: true
.. _uvloop: https://magic.io/blog/uvloop-make-python-networking-great-again/

0 comments on commit ada1369

Please sign in to comment.