Skip to content

Commit

Permalink
chore(docs): update new datasource developer docs
Browse files Browse the repository at this point in the history
Reorder ds-identify docs above python DataSource module as the
python module needs to reflect the same ds-identify detection logic
in a ds_detect module.

Add that ds-identify does rely on identification alternatives to DMI such as
attached device labels, kernel command line and virt type.
Add example reference to simple NWCS datasource commit for ds-identify docs.

Drop mentions of 'you' and 'your' from docs.
  • Loading branch information
blackboxsw committed Sep 18, 2024
1 parent 82efd3e commit 6c0dea8
Showing 1 changed file with 57 additions and 45 deletions.
102 changes: 57 additions & 45 deletions doc/rtd/development/datasource_creation.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _datasource_creation:

Supporting your cloud or platform
*********************************
Supporting a new cloud or platform
**********************************

The upstream cloud-init project regularly accepts code contributions for new
platforms that wish to support cloud-init.
Expand Down Expand Up @@ -29,17 +29,19 @@ cloud-init support.
Technical requirements
----------------------

A cloud needs to be able to identify itself to cloud-init at runtime, and that
the cloud be able to provide configuration to the instance.
A cloud needs to be able to identify itself to cloud-init at runtime and
provide unique configuration to the instance.

A mechanism for self-identification
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A mechanism for identification
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Each cloud platform must positively identify itself to the guest. This allows
the guest to make educated decisions based on the platform on which it is
running. On the x86 and arm64 architectures, many clouds identify themselves
through `DMI`_ data. For example, Oracle's public cloud provides the string
``'OracleCloud.com'`` in the DMI chassis-asset field.
``'OracleCloud.com'`` in the DMI chassis-asset field. Some platforms present
attached devices with well known filesystem label, kernel command line flags or
virtualization types which uniquely identify a particular cloud platform.

``Cloud-init``-enabled images produce a log file with details about the
platform. Reading through this log in :file:`/run/cloud-init/ds-identify.log`
Expand All @@ -66,9 +68,9 @@ Testing access
^^^^^^^^^^^^^^

A platform that isn't available for testing cannot be independently validated.
You will need to provide some means for community members and upstream
developers to test and verify this platform. Code that cannot be used cannot be
supported.
Datasource platform maintainers need to provide some means for community
members and upstream developers to test and verify this platform. Code that
cannot be used cannot be supported.

Maintainer support
^^^^^^^^^^^^^^^^^^
Expand All @@ -86,25 +88,43 @@ There are multiple ways to provide `user data`, `metadata`, and
`vendor data`, and each cloud solution prefers its own way. A datasource
abstract base class defines a single interface to interact with the different
clouds. Each cloud implementation must inherit from this base class to use this
shared functionality and interface. See :file:`cloud-init/sources/__init__.py`
shared functionality and interface. See `cloudinit/sources/__init__.py`_
to see this class.

If you are interested in adding a new datasource for your cloud platform you
will need to do all of the following:
For cloud-init upstream to support a new datasource, all of the the following
are needed:

Update ``ds-identify``
----------------------

In ``systemd``, Alpine and BSD environments, ``ds-identify`` runs in early boot
to detect which datasource should be enabled, or if ``cloud-init`` should run
at all. Add early identification support for the platform via a
``dscheck_<CloudPlatform>`` function in :file:`tools/ds-identify`.
For example, see `NWCS support`_

Add tests for ``ds-identify``
-----------------------------

Add relevant tests in a new class to
`tests/unittests/test_ds_identify.py`_. Use ``TestOracle`` as an example.

Add datasource module cloudinit/sources/DataSource<CloudPlatform>.py
--------------------------------------------------------------------

We suggest you start by copying one of the simpler datasources
such as ``DataSourceHetzner``.
Use one of the simpler datasources such as ``DataSourceHetzner`` as a guiding
template for style and expectations. The DataSource module should implement a
``ds_detect`` method validates the same identification conditions defined
in ds-identify and returns ``True`` when met. This allows cloud-init to support
environments without ds-identify run as part of the init system.

Re-run datasource detection
^^^^^^^^^^^^^^^^^^^^^^^^^^^

While developing a new datasource it may be helpful to manually run datasource
detection without rebooting the system.

To re-run datasource detection, you must first force :file:`ds-identify` to
To re-run datasource detection, first force :file:`ds-identify` to
re-run, then clean up any logs, and finally, re-run ``cloud-init``:

.. code-block:: bash
Expand All @@ -118,33 +138,19 @@ Add tests for datasource module
-------------------------------

Add a new file with some tests for the module to
:file:`cloudinit/sources/test_<yourplatform>.py`. For example, see
:file:`cloudinit/sources/tests/test_oracle.py`

Update ``ds-identify``
----------------------

In ``systemd`` systems, ``ds-identify`` is used to detect which datasource
should be enabled, or if ``cloud-init`` should run at all. You'll need to
make changes to :file:`tools/ds-identify`.

Add tests for ``ds-identify``
-----------------------------

Add relevant tests in a new class to
:file:`tests/unittests/test_ds_identify.py`. You can use ``TestOracle`` as
an example.
:file:`tests/unittests/sources/test_<cloudplatform>.py`. For example, see
`tests/unittests/sources/test_oracle.py`_

Add your datasource name to the built-in list of datasources
------------------------------------------------------------
Add new datasource name to the built-in list of datasources
-----------------------------------------------------------

Add your datasource module name to the end of the ``datasource_list``
entry in :file:`cloudinit/settings.py`.
Add the new datasource module name to the end of the ``datasource_list``
entry in `cloudinit/settings.py`_.

Add your cloud platform to apport collection prompts
----------------------------------------------------
Add cloud platform to apport collection prompts
-----------------------------------------------

Update the list of cloud platforms in :file:`cloudinit/apport.py`. This list
Update the list of cloud platforms in `cloudinit/apport.py`_. This list
will be provided to the user who invokes :command:`ubuntu-bug cloud-init`.

Enable datasource by default in Ubuntu packaging branches
Expand All @@ -155,17 +161,17 @@ Ubuntu packaging branches contain a template file,
``datasource_list`` that is installed by distros that use the upstream
packaging configuration.

Add documentation for your datasource
-------------------------------------
Add documentation for the new datasource
----------------------------------------

You should update the following docs:
Update the following docs:
1. Add a new file in :file:`doc/rtd/reference/datasources/<cloudplatform>.rst`
2. Reference `<cloudplatform>.rst` in :file:`doc/rtd/reference/datasources.rst`
3. Add an alphebetized dsname entry in representing your datasource
3. Add an alphabetized dsname entry in representing the datasource
:file:`doc/rtd/reference/datasource_dsname_map.rst`

Benefits of including your datasource in upstream cloud-init
============================================================
Benefits of including a datasource in upstream cloud-init
=========================================================

Datasources included in upstream cloud-init benefit from ongoing maintenance,
compatibility with the rest of the codebase, and security fixes by the upstream
Expand All @@ -180,3 +186,9 @@ If this is not possible, one can add
.. _Proxmox: https://pve.proxmox.com/wiki/Cloud-Init_Support
.. _DatasourceNoCloud.py: https://github.com/canonical/cloud-init/blob/main/cloudinit/sources/DataSourceNoCloud.py
.. _nocloud: https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html
.. _NWCS support: https://github.com/canonical/cloud-init/commit/d0cae67b
.. _cloudinit/apport.py: https://github.com/canonical/cloud-init/tree/main/cloudinit/apport.py
.. _cloudinit/settings.py: https://github.com/canonical/cloud-init/tree/main/cloudinit/settings.py
.. _cloudinit/sources/__init__.py: https://github.com/canonical/cloud-init/tree/main/cloudinit/sources/__init__.py
.. _tests/unittests/test_ds_identify.py: https://github.com/canonical/cloud-init/tree/main/tests/unittests/test_ds_identify.py
.. _tests/unittests/sources/test_oracle.py: https://github.com/canonical/cloud-init/tree/main/tests/unittests/sources/test_oracle.py

0 comments on commit 6c0dea8

Please sign in to comment.