Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ubuntu/focal pre-upload packaging review 24.3.1 versus released 24.2-0ubuntu1~20.04.1 #27

Closed
wants to merge 133 commits into from

Conversation

blackboxsw
Copy link
Owner

blocks canonical#5679 for upstream release of 24.3.1 into Focal

leavelet and others added 30 commits July 3, 2024 14:10
The /etc/hostname.* files should have the mtu on
a separate line otherwise it gives error:

  ifconfig: mtu: bad value

The lines are executed in order by ifconfig and
mtu should be on it's own line.

Fixes: canonicalGH-5413
)

When there is no IPv6 addr given in the customization configuration,
we shall set IPv6 type to dhcp6, then customized Linux network will be
set to dhcp IPv6 explicitly.
* Remove `unittest` constructs and remove base classes.
* Replace tests that don't test things with tests that do
* Add fstab and mounts combinations test
There are no call sites requesting not decoding the environment vars.
This change decodes then always, simplifying typing and logic.
Instead of a broad try/except, do properly check for conditions that
invalidate a mount location.
Support for jumbo frames requires that the underlying physical interfaces
and the parent bond interface all have the larger MTU configured, not just
the physical interfaces.
…canonical#5501)

At least one of (or both) 'baseurl' or 'metalink' should be provided for yum
repository specification. Add schema changes to enforce it. Without this,
with just 'metalink' property set, one would get the schema validator error

\---
Error: Cloud config schema errors: yum_repos.epel-release: 'baseurl' is a required property
\---

Signed-off-by: Ani Sinha <[email protected]>
On systemd, services are started by PID 1. When this doesn't happen, cloud-init
is in an unknown run state and should warn the user.

Reorder pid log to be able to reuse Distro information.

Add docstring deprecating util.is_Linux().
…5209)

Implement verify_clean_boot() to ignore certain expected logs
in a platform-specific way.
…anonical#5209)

Ensure ignore_warnings=True or ignore_errors=True is honored and
not overridden by supplemental warning texts appended.
…l#5209)

Avoid using warning level messages as there may be some
use-cases in the wild that need to invoke cloud-init boot
stages after boot for some reason unknown to upstream.

Provide a detailed warning message informing admins to file
issues against cloud-init to better represent those feature
needs before dropping this feature altogether.
…onical#5504)

Commit 604d80e introduced assertions expecting exit 2 from the
CLI when calling cloud-init init --local. Revert this test assertion
as only cloud-init status command exits (2) on deprecations/warnings.

Invoking cloud-init's boot stages on the commmand line will only exit
1 if critical errors are encountered to avoid degrading overall
systemd health as seen from cloud-init systemd units. When cloud-init
boot stages encounter recoverable_errors of any type, there is no
need to exit non-zero as those deprecation logs are not-critical to
the health of the system as a whole.
It is pretty consistently failing due to canonical#5373 with no fix in
sight.
Ensure DNS server addresses are parsed from the proper location
of network_data.json

Fixes canonical#5386

Co-authored-by: Alberto Contreras <[email protected]>
If DNS information is added to a NetworkManager managed interface where
the given protocol family is disabled, NetworkManager will be unable to
activate the interface.

canonical#5387
'mirrorlist' config can be specified instead or along with 'baseurl' in the yum
repository config. Add support for specifying mirrorlist instead of 'baseurl'.

Fixes canonicalGH-5520
Signed-off-by: Ani Sinha <[email protected]>
9929a00 added the ability to used a cached datasource when none is
found. This was supposed to be per-datasource, but the lack of cache
cleaning got applied universally. This commit makes it so cache will be
cleaned as it was before if fallback isn't implemented in datasource.

Fixes canonicalGH-5486
This is useful for logs we want hidden by default but can be turned
on via configuration.
Switch to pathlib where appropriate and call consistently
…5515)

With this change, the following config in cloud.cfg.d/ will select NoCloud in
network stage.

```
datasource_list: [ GCE, NoCloud, None ]
datasource:
  NoCloud:
    seedfrom: http://0.0.0.0:8000/
```

Previously a two or less datasources in the datasource_list were required to
get this behavior, which was undocumented and not intuitive.

The ds-identify already allowed inline user-data and meta-data to
trigger detection.

Add ds-identify unittests for seedfrom and inline user-data.
Add DataSourceNoCloud.ds_detect() unittests for seedfrom and inline
user-data.
The nocloud datasource logs messages that are sometimes confused by users
for errors. Clarify them.

Also, remove redundant information from the logs:

- simplify log wording
- only include seed and dsmode information in nocloud string when
  non-default values are used
blackboxsw and others added 15 commits August 20, 2024 10:38
Adapt to new annotation formating from
a2193da.
…onical#5636)

Directly calling execute("cloud-init clean --logs --reboot") on
an integration instances also involves awaiting a new boot id upon
next interaction with with instance to ensure a reboot has actually
taken place already on this target machine.

Slow responding test instances/platforms may not completed the shutdown
restart sequence yet when trying to iteract with an immediate blocking
call to execut("cloud-init status --wait") which may exit early if accessing
the prior instance boot before the reboot occurred.

It is preferable to use inspect /proc/sys/kernel/random/boot_id before
issuing a reboot request and block until a delta is seen in boot_id.
This blocking wait on reboot and new boot_id is encapsulated inside
pycloudlib.BaseInstance.restart which will inspect
/proc/sys/kernel/random/boot_id before restart and block until a delta
in boot_id across the requested restart.

Fix test_status_block_through_all_boot_status to call instance.clean()
and restart() to ensure we do not beat the instance reboot race with
our post-boot assertions.
Add PPS support for azure-proxy agent and improve error logging.
…nical#5633)

Do not treat the emptiness of .cloud-init/ as an error in the logs
if agent.yaml is present.

Fixes canonicalGH-5632
…ply (canonical#5622)

Perform the same steps that cloud-init daily recipe builds performs
to assert any packaging branch updates will not break daily builds
due to quilt patch apply issues.

Steps of daily build recipe reflected in this workflow:
- checkout main
- merge packaging branch topmost commit
- quilt push -a
- run unittests (via tox -e py3)
- quilt pop -a
…cal#5641)

Reintroduce strict assert that cloud-init's cert in userdata is the
only root cert defined on the platform. Google guest agent was
installed a secondary root cert in ca_certifications.crt for a period of
time and this was determined to be less than ideal practice.

Allow cloud-init's integration tests to remain strict validation of
cert checksum to provide a signal if other platforms or agents
attempt to extend or alter the system-wide CA.
…d field (canonical#5355)

Currently cc_user_groups assumes that "useradd" never locks the password
field of newly created users. This is an incorrect assumption.

Change add_user (in both __init__.py and alpine.py) to
explicitly call either lock_passwd or unlock_passwd at all times to
achieve the desired final result.

For existing users with empty or empty locked passwords, no
password unlock will be performed and warnings will be issued.
To support empty password validation, provide functionality to
parse /etc/shadow and /var/lib/extrausers/shadow to assert
existing users do not have empty passwords before unlocking.

Additionally in this commit: 
- add NetworkBSD.ifs property to avoid subp side-effect in ___init__
  which calls ifconfig -a at every instance initialization

Useradd background:

From the useradd manpage:

'-p, --password PASSWORD
The encrypted password, as returned by crypt(3). The default is to
disable the password.'

That is, if cloud-init runs 'useradd' but does not pass it the "-p"
option (with an encrypted password) then the new user's password field
will be locked by "useradd".

cloud-init only passes the "-p" option when calling "useradd" when
user-data specifies the "passwd" option for a new user. For user-data
that specifies either the "hashed_passwd" or "plain_text_passwd"
options instead then cloud-init calls "useradd" without the "-p" option
and so the password field of such a user will be locked by "useradd".

For user-data that specifies "hashed_passwd" for a new user then
"useradd" is called with no "-p" option, so causing "useradd" to lock the
password field, however then cloud-init calls "chpasswd -e" to set the
encrypted password which also results in the password field being
unlocked.

For user-data that specifies either "plain_text_passwd" for a new user
then "useradd" is called with no "-p" option, so causing "useradd" to
lock the password. cloud-init then calls "chpasswd" to set the password
which also results in the password field being unlocked.

For user-data that specifies no password at all for a new user then
"useradd" is called with no "-p" option, so causing "useradd" to lock
the password. The password field is left locked.

In all the above scenarios "passwd -l" may be called later by
cloud-init to enforce "lock_passwd: true").

Conversely where "lock_passwd: false" applies the above "usermod"
situation (for "hash_passwd", "plain_text_passwd" or no
password) means that newly created users may have password
fields locked when they should be unlocked.

For Alpine, "adduser" does not support any form of password being
passed and it always locks the password field (the same point
applies about password field being unlocked when/if "chpasswd" is
called). Therefore in some situations (i.e. no password specified
in user-data) the password needs to be unlocked if
"lock_passwd: false".
Bump the version in cloudinit/version.py to 24.3 and
update ChangeLog.
Bump the version in cloudinit/version.py to 24.3.1 and
update ChangeLog.
@blackboxsw blackboxsw changed the title Ubuntu/focal pre-upload packaging review 24.3.1 Ubuntu/focal pre-upload packaging review 24.3.1 versus released 24.2-0ubuntu1~20.04.1 Sep 6, 2024
patches:
debian/patches/cli-retain-file-argument-as-main-cmd-arg.patch
debian/patches/drop-unsupported-systemd-condition-environment.patch
debian/patches/netplan99-cannot-use-default.patch
debian/patches/no-nocloud-network.patch
debian/patches/no-single-process.patch
debian/patches/revert-551f560d-cloud-config-after-snap-seeding.patch
Revert remaning functional references to cloud-init-network service
which will not exist on stable releases.
Copy link

@panlinux panlinux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pre-SRU +1

@blackboxsw
Copy link
Owner Author

Thank you @panlinux. I'll close this branch as approved and merge the upstream PR canonical#5678 once our upload gets accepted into -proposed

@blackboxsw blackboxsw closed this Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.