The /etc/hostname.* files should have the mtu on
a separate line otherwise it gives error:

  ifconfig: mtu: bad value

The lines are executed in order by ifconfig and
mtu should be on it's own line.

Fixes: canonicalGH-5413

)

When there is no IPv6 addr given in the customization configuration,
we shall set IPv6 type to dhcp6, then customized Linux network will be
set to dhcp IPv6 explicitly.

* Remove `unittest` constructs and remove base classes.
* Replace tests that don't test things with tests that do
* Add fstab and mounts combinations test

canonicalGH-5445

There are no call sites requesting not decoding the environment vars.
This change decodes then always, simplifying typing and logic.

Instead of a broad try/except, do properly check for conditions that
invalidate a mount location.

Support for jumbo frames requires that the underlying physical interfaces
and the parent bond interface all have the larger MTU configured, not just
the physical interfaces.

…canonical#5501)

At least one of (or both) 'baseurl' or 'metalink' should be provided for yum
repository specification. Add schema changes to enforce it. Without this,
with just 'metalink' property set, one would get the schema validator error

\---
Error: Cloud config schema errors: yum_repos.epel-release: 'baseurl' is a required property
\---

Signed-off-by: Ani Sinha <[email protected]>

On systemd, services are started by PID 1. When this doesn't happen, cloud-init
is in an unknown run state and should warn the user.

Reorder pid log to be able to reuse Distro information.

Add docstring deprecating util.is_Linux().

…5209)

Implement verify_clean_boot() to ignore certain expected logs
in a platform-specific way.

…anonical#5209)

Ensure ignore_warnings=True or ignore_errors=True is honored and
not overridden by supplemental warning texts appended.

…l#5209)

Avoid using warning level messages as there may be some
use-cases in the wild that need to invoke cloud-init boot
stages after boot for some reason unknown to upstream.

Provide a detailed warning message informing admins to file
issues against cloud-init to better represent those feature
needs before dropping this feature altogether.

…onical#5504)

Commit 604d80e introduced assertions expecting exit 2 from the
CLI when calling cloud-init init --local. Revert this test assertion
as only cloud-init status command exits (2) on deprecations/warnings.

Invoking cloud-init's boot stages on the commmand line will only exit
1 if critical errors are encountered to avoid degrading overall
systemd health as seen from cloud-init systemd units. When cloud-init
boot stages encounter recoverable_errors of any type, there is no
need to exit non-zero as those deprecation logs are not-critical to
the health of the system as a whole.

It is pretty consistently failing due to canonical#5373 with no fix in
sight.

Ensure DNS server addresses are parsed from the proper location
of network_data.json

Fixes canonical#5386

Co-authored-by: Alberto Contreras <[email protected]>

If DNS information is added to a NetworkManager managed interface where
the given protocol family is disabled, NetworkManager will be unable to
activate the interface.

canonical#5387

'mirrorlist' config can be specified instead or along with 'baseurl' in the yum
repository config. Add support for specifying mirrorlist instead of 'baseurl'.

Fixes canonicalGH-5520
Signed-off-by: Ani Sinha <[email protected]>

9929a00 added the ability to used a cached datasource when none is
found. This was supposed to be per-datasource, but the lack of cache
cleaning got applied universally. This commit makes it so cache will be
cleaned as it was before if fallback isn't implemented in datasource.

Fixes canonicalGH-5486

This is useful for logs we want hidden by default but can be turned
on via configuration.

Switch to pathlib where appropriate and call consistently

…5515)

With this change, the following config in cloud.cfg.d/ will select NoCloud in
network stage.

```
datasource_list: [ GCE, NoCloud, None ]
datasource:
  NoCloud:
    seedfrom: http://0.0.0.0:8000/
```

Previously a two or less datasources in the datasource_list were required to
get this behavior, which was undocumented and not intuitive.

The ds-identify already allowed inline user-data and meta-data to
trigger detection.

Add ds-identify unittests for seedfrom and inline user-data.
Add DataSourceNoCloud.ds_detect() unittests for seedfrom and inline
user-data.

The nocloud datasource logs messages that are sometimes confused by users
for errors. Clarify them.

Also, remove redundant information from the logs:

- simplify log wording
- only include seed and dsmode information in nocloud string when
  non-default values are used

ds-identify does not support the fs_label key. This key is only partially
supported.

Deprecate it. Users of custom labels may switch to cidata or CIDATA.

Note: Tools such as cloud-localds hard-code the label.

)

UP4W business logic is so that its data overrides user at a key (module) 
level.
That means the entire Landscape config is overriden if both agent data
and user data contains config for that module.
Yet, for better usability, computer tags must be assignable per instance.
That's not possible with agent.yaml, because it's meant to be global.
Its config data affects all Ubuntu WSL instances.

Thus this aims to make a special case for landscape.client.tags,
if present in user provided data (either Landscape or local user -
  whatever is picked up before merging with agent.yaml)
its value overwrites any tags set by agent.yaml.

Only landscape.client.tags are treated specially.
The pre-existing merge rules still apply for any other value present in
both agent.yaml and user provided data.

Fixes UDENG-2464

Commit 441d8f8 adds openeuler to the list of supported distros in
cc_spacewalk.py, but there is one omission.

This change adds an optional `source` key to the `write_files` module,
allowing users to specify a URI from which to load file contents. This
facilitates more flexible multi-part configurations, as file contents
can be managed via external sources such as independent Git
repositories.

Fixes canonicalGH-5500

Change default usr_lib_exec from /usr/libexec/ to /usr/lib
as azurelinux installs cloud-init tool binaries under
/usr/lib/cloud-int instead of /usr/libexec/cloud-init

After 7703634 log messages migrated
from:

"Detected platform DataSource..." to "Detected DataSource..."

Adapt integration tests accordingly.

Adapt to conform with 7703634

…nonical#5537)

Got swallowed by
https://github.com/canonical/cloud-init/pull/5116/commits/
da6b5c4

The former commit resulted in usage error from the wslpath
command thus we never found WSL specific data,
disabling cloud-init.

Create explicit file to define and pin the actions/labeler's version.

Upgrade breaking changes from action/labeler@v5.

The suggested qemu command line in our local execution example
is rather old.

Change the discouraged -hd* options to the new
-device instead.

Further add a chance to use KVM acceleration to speed up the
example.

Finally we had several occasions to be working on x86 only.
We dropped arguments that can not work on cross-arch and
furthermore added a hint at how one could again native
performance on these platforms.

Fixes canonicalGH-5050

SC-1835
Fixes canonicalGH-5334

Fixes canonicalGH-5513

…ical#5543)

Improve explaination on #cloud-boothook for end-users.

SC-1657
Fixes canonicalGH-4542

Co-authored-by: Calvin Mwadime <[email protected]>

Fixes canonicalGH-5467

Without this explicit config option, an untracked and unused
`doc/rtd/spelling_wordlist.txt` file is created while running
`tox -e doc-spelling`.

…cal#5545)

…#5521)

Formally document providing runtime configuration in system configuration.
Introduce names to identify previously unnamed NoCloud concepts.
Add more structure - discrete sections for:
- runtime configuration types
- discovery configuration
- configuration sources

Add group of pages for drop-in custom modules and
restructure existing docs under it.

Add doc for custom datasources and config modules.
    
SC-1836
Fixes canonicalGH-4649

Fixes canonicalGH-4739

…nonical#5568)

Unspecified base match in labeler assumes 'any' for each match
clause. When specifying base-branch and --any-glob-to-any-file either
one of these cases would result in a successful match which would label
all PRs again main as documentation. We need to explicitly specify
'all:' in our labeler match config to ensure BOTH:

 * matching file paths related to documentation
         -AND-
 * targeting a merge against 'main' branch

…5569)

Both were version gated and do not run.

…cal#5570)

Also drop undesirable former doc-autolabel.yml workflow

When instance id hasn't changed and datasource hasn't changed, don't
forcibly reload the configuration.

This enables support for network config v2 and v1 to NoCloud
when used with http / ftp / etc. 

BREAKING_CHANGE: Adds an additional network request to NoCloud.

String output changed in 7703634.
Instance-id doesn't change on LXD / Focal.

The handle function of cc_mounts was hard to grok and had one of the
highest cyclomatic complexity scores in the codebase. Functionally,
the code should be unchanged.

…#5573)

Python interpreter initialization and module import time 
contributes a significant amount of wall clock time to
cloud-init's runtime (and therefore to total boot time).

Cloud-init has four stages. Each stage starts its own Python
interpreter and loads the same libraries. To eliminate the
redundant work of starting an interpreter and loading libraries,
this changes cloud-init to run as a single process. Systemd
service ordering is retained by using the existing cloud-init
services as shims which use a synchronization protocol to start
each cloud-init stage and to communicate that each stage is
complete to the init system. Since multiple cloud-init processes
sit in the critical chain of starting the system, this reduces
boot time (including time to ssh login and time to cloud-init
completion).

Currently only systemd is supported, but the synchronization
protocol should be capable of supporting other init systems
as well with minor changes.

Note: This enables many additional follow-on improvements that
eliminate redundant work. However, these potential improvements
are temporarily ignored. This commit has been structured to
minimize the changes required to capture the majority of primary
performance savings while preserving correctness and the ability
to preserve backwards compatibility.

Since this changes the semantics of the existing cloud-init unit
files, this change takes the opportunity to rename one of its
systemd units which causes frequent user confusion. The unit named
cloud-init.service is often mistaken by users for being the only
cloud-init service, when it is simply one of four stages. This
stage is documented as the "network" stage, so this service will
be renamed to "cloud-init-network.service". A new notify service
is added as part of this implementation which contains the
cloud-init process. This unit is named "cloud-init-main.service".
 
Synchronization protocol
========================

- create one Unix socket for each systemd service stage
- send sd_notify()
- For each of the four stages (local, network, config, final):
   - when init system sends "start" to the Unix socket, start the
     stage
   - when running stage is complete, send "done" to Unix socket

File changes
============

socket.py (new)
---------------

- define a systemd-notify helper function
- define a context manager which implements a multi-socket
  synchronization protocol

cloud-init.service -> cloud-init-network.service (renamed)
----------------------------------------------------------

- renamed to cloud-network.service

cloud-{init-local,init-network,config,final}.services
-------------------------------------------

- change ExecStart to use netcat to connect to Unix socket and:
  - send a start message
  - wait for completion response
- note: a pure Python equivalent is possible for any downstreams
  which do not package openbsd's netcat

cloud-init-main.service (new)
-----------------------------

 - use service type to 'notify'
 - invoke cloud-init in single process mode
 - adopt systemd ordering requirements from cloud-init-local.service
 - adopt KillMode from cloud-final.service

main.py
-------

 - Add command line flag to indicate "all stages" mode
 - In this mode run each stage followed by an IPC
   synchronization protocol step

cloud-final.services
--------------------

- drop KillMode

cloud-init-local.services
-------------------------

- drop dependencies made redundant by ordering after
  cloud-init-main.service

Performance Impact
==================

On Ubuntu 24.04, Python's wall clock start up time as measured with
`time python3 -c 'import cloudinit.cmd.main' on a few cloud types:

lxc container: 0.256s
QEMU machine:  0.300s
gce instance:  0.367s
ec2 instance:  0.491s

This change eliminates x1 this start up time from time to ssh.
This change eliminates x3 this start up time from cloud-init's total
completion. Total benefit varies based on the platform that the
instance is hosted by, but all platforms will measurably benefit from
this change.

BREAKING_CHANGE: Run all four cloud-init services as a single systemd service.

Commit 5322dca introduced an assumption to read_seeded that
network-config must always be present for NoCloud datasource.
Since it is still considered and optional supplemental configuration
allow the read_seeed calls to succeed in the absence of network-config.

Avoids failures seen in tests/integration-tests/datasources/test_nocloud.py::
  test_nocloud_seedfrom_vendordata