Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[docs] Add explanation about cloud-config #5403

Closed
wants to merge 7 commits into from
Closed

[docs] Add explanation about cloud-config #5403

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
c057066
Add explanation about cloud-config
s-makin Jun 13, 2024
3f600db
Apply review feedback
s-makin Jul 11, 2024
1cfd8e8
Address more review feedback
s-makin Jul 11, 2024
5977475
Fix spelling
s-makin Jul 11, 2024
63bafd5
Trailing spaces
s-makin Jul 11, 2024
a66402b
doc: redact storage, fix invalid user/chpasswd schema
blackboxsw Jul 24, 2024
2df14ce
Merge branch 'main' into cloud-config-explanation
s-makin Aug 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 163 additions & 0 deletions doc/rtd/explanation/about-cloud-config.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
.. _about-cloud-config:

About the cloud-config file
s-makin marked this conversation as resolved.
Show resolved Hide resolved
***************************

The ``#cloud-config`` file is a type of user data that cloud-init can consume
to automatically set up various aspects of the system. It is the preferred way
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preferred by who? I don't think the answer is "users" as I remember a previous maintainer telling me that the vast majority of user data is just simple bash scripts. We could poll the maintainers, but I don't have any particular preference for cloud-config.

to pass this type of configuration to cloud-init, and is commonly referred to
as "cloud config".

Note that networks are not configurable via the ``#cloud-config`` file because
:ref:`network configuration <network_config>` comes from the cloud.

How do I create a cloud-config file?
====================================

The cloud-config file uses `YAML version 1.1`_. The file is composed of a
**header** and one or more **modules**.

* **The header**:
The first line **must start** with ``#cloud-config``. This line identifies
the file to cloud-init and ensures that it will be processed as intended.

* **The modules**:
After the header, every aspect of the system's configuration is controlled
through specific cloud-init modules.

Each module is declared through the use of its **top-level key**, and the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if we were this consistent, but not all modules have a single top-level key. E.g., the set passwords module reacts to the password, chpasswd, and ssh_pwauth keys.

Additionally, there are multiple modules that may react to the same key. E.g., write_files key is used by both the write_files and write_files_deferred module and hostname and fqdn keys are used by both the set_hostname and update_hostname modules.

configuration options are set using key-value pairs (or list keys) according
to the config schema. It follows this general format:

.. code-block:: yaml

#cloud-config
top-level-key:
config-key-1: config-value-1
config-key-2: config-value-2
list-key:
- list-value-1
additional-list-value-1
- list-value-2

Let us consider a real example using the :ref:`Keyboard <mod_cc_keyboard>`
module. The top-level key for this module is ``keyboard:``, and beneath that
are the various configuration options for the module shown as key-value pairs:

.. code-block:: yaml

#cloud-config
keyboard:
s-makin marked this conversation as resolved.
Show resolved Hide resolved
layout: us
model: pc105
variant: nodeadkeys
options: compose:rwin

A full list of modules can be found :ref:`on our modules page <modules>`. This
list also shows the valid schema keys for every module, and YAML examples.

.. note::
When a module declares no `Activate only on keys:` section, that module will
attempt to run regardless of specific schema keys.


Module ordering
---------------

The order of the ``cloud-config`` keys is unimportant and modules can be
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to what I commented above, keys don't correspond 1:1 with modules, so I don't think it makes sense to talk about the cloud-config as if we're writing or specifying modules.

written in any order.

.. note::
The :ref:`Users and Groups <mod_cc_users_groups>` module is a special case
Copy link
Member

@holmanb holmanb Jul 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this note?

There are other examples where ordering matters too - any place in the schema where a list is used rather than a k/v pair is likely to have an ordering effect (consider runcmd, for example). I'm not sure that enumerating all of the places that lists are used really makes sense. Maybe we just need to point out how lists and key/value pairs behave differently and that's all that we need?

where ordering matters if you want to add users to a group list.

Checking your cloud-config file
===============================

Once you have constructed your cloud-config file, you can check it against
the :ref:`cloud-config validation tool <check_user_data_cloud_config>`. This
tool tests the YAML in your file against the cloud-init schema and will warn
you of any errors.

Example cloud-config file
=========================

The following code is an example of a complete user data cloud-config file.
The :ref:`cloud-config example library <yaml_examples>` contains further
examples that can be copy/pasted and adapted to your needs.

.. code-block:: yaml

#cloud-config

# Basic system setup
hostname: example-host
fqdn: example-host.example.com

# User setup configuration
users:
- name: exampleuser
gecos: Example User
sudo: ['ALL=(ALL) NOPASSWD:ALL']
blackboxsw marked this conversation as resolved.
Show resolved Hide resolved
groups: sudo
home: /home/exampleuser
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
home: /home/exampleuser
homedir: /home/exampleuser

shell: /bin/bash
ssh_authorized_keys:
- ssh-rsa AAAAB3...restofpublickey user@host

# Change passwords for exampleuser using chpasswd
chpasswd:
expire: false
users:
- {name: exampleuser, password: terriblepassword12345, type: text}

# Package management
package_update: true
package_upgrade: true
packages:
- git
- nginx
- python3

# Commands to run at the end of the cloud-init process
runcmd:
- echo "Hello, world!" > /etc/motd
- systemctl restart nginx
- mkdir -p /var/www/html
- echo "<html><body><h1>Welcome to the party, pal!</h1></body></html>" > /var/www/html/index.html

# Write files to the instance
write_files:
- path: /etc/example_config.conf
content: |
[example-config]
key=value
- path: /etc/motd
content: |
Some text that will appear in your MOTD!

# Final message, shown after cloud-init completes
final_message: "The system is up, after $UPTIME seconds"

# Reboot the instance after configuration
power_state:
mode: reboot
message: Rebooting after initial setup
timeout: 30
condition: True

Ubuntu installer cloud-config
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that this section makes sense in cloud-init's documentation about cloud-config. Cloud-init doesn't do anything special when autoinstall is included, it is just ignored (and subiquity does whatever it does with it later).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A counterpoint here is that we are getting lots of questions in IRC in cloud-init about this and a lot of keyword searches past 30 days: autoinstall (1 result) 136 searches. I'd rather have a link handy in our docs that says, not our deal so that hopefully it is discoverable before people ask in #cloud-init channel in IRC.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a lot of keyword searches past 30 days: autoinstall (1 result) 136 searches. I'd rather have a link handy in our docs that says, not our deal so that hopefully it is discoverable before people ask in #cloud-init channel in IRC.

Why is is this page the right place for it? There are many things that one could say about cloud-config, but there is value in keeping it short and on-topic without cluttering it with random details. Also, go search autoinstall in the docs. I get exactly one hit. We don't need to say "not our deal" in two places, so I'd rather not add another unless we're planning on removing the existing one - and if we're going to relocate it lets put it where it belongs rather than from one section to the next. Your points make it sound like it is a frequently asked question, which is exactly the purpose of the page that that section is currently on.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this doesn't belong here. I made a similar comment on an earlier version of this PR and would prefer to keep this info buried in the FAQ, but FAQs don't fit nicely into diataxis and are seen by some as an indication of bad documentation.

Could we keep it in the FAQ for now and perhaps grow a "Projects that use cloud-init" or similar page that point to other well known (and not just Canonical) projects that use cloud-init along with important things to be aware of?

-----------------------------

For the special case where your cloud-config file will be consumed by the
Ubuntu live installer, you will need to include the ``autoinstall:`` top level
key. The presence of this key instructs cloud-init not to process
the user-data itself, but instead to pass it directly to the installer for
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The presence of this key instructs cloud-init not to process
the user-data itself, but instead to pass it directly to the installer for
processing.

That's not true, is it? The presence of this key doesn't prevent cloud-init from processing the user-data.

Cloud-init just ignores keys nested under the autoinstall key.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good pt. cloud-init just ignores it, it doesn't take any action to "pass it to the installer"

processing.

For more detailed instructions for this case, refer to the installer
documentation on using `cloud-init with the autoinstaller`_.

.. LINKS
.. _cloud-init with the autoinstaller: https://canonical-subiquity.readthedocs-hosted.com/en/latest/tutorial/providing-autoinstall.html#autoinstall-by-way-of-cloud-config
.. _YAML version 1.1: https://yaml.org/spec/1.1/current.html
1 change: 1 addition & 0 deletions doc/rtd/explanation/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ knowledge and become better at using and configuring ``cloud-init``.
:maxdepth: 1

introduction.rst
about-cloud-config.rst
format.rst
configuration.rst
boot.rst
Expand Down
Loading