This repository has been archived by the owner on Jan 6, 2023. It is now read-only.
meta-swupd #95
Merged
meta-swupd #95
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When building without the "smack" image feature in a distro configuration where Smack is enabled (the default in Ostro OS), the resulting image still had (unused) Smack xattrs, because the packages were compiled with Smack support and set Smack xattrs during installation. This turned out to be problematic while working with swupd image creation when xattrs were not yet supported. In addition, unused filesystem content takes up unnecessary space. Therefore it is better to remove the unused Smack xattrs. Signed-off-by: Patrick Ohly <[email protected]>
There is a standard OE-core variable for the first part of the file
name used for symlinks to the latest image file. The dsk image code
should use that instead of hard-coding ${BPN}-${MACHINE}, because then
it automatically does the right thing in image recipes which modify
IMAGE_BASENAME.
Signed-off-by: Patrick Ohly <[email protected]>
Adds meta-swupd to the default set of layers used for building Ostro OS and bumps the config file version to ensure that users with an out-dated conf/bblayer.conf are notified explicitly during the sanity check phase, instead of getting more obscure build errors. Signed-off-by: Patrick Ohly <[email protected]>
The new 3.x is available in meta-swupd, but has not been tested enough and at least at this time lacks a critical fix (the hardlink workaround). Signed-off-by: Patrick Ohly <[email protected]>
When building swupd bundles locally, developers need a way to serve the resulting "www" directory. This script adds some boilerplate code around Python's SimpleHTTPServer to simplify starting and stopping such a server. Signed-off-by: Patrick Ohly <[email protected]>
These recipes are supposed to be a part of the meta-swupd layer that cn be used not only in Ostro. Signed-off-by: Dmitry Rozhkov <[email protected]>
Ostro uses an update mechanism which differs from what's used in ClearLinux. This patch adds Ostro specific EFI updater to the swupd client. Signed-off-by: Dmitry Rozhkov <[email protected]> swupd-client: add rdep on gptfdisk to satisfy EFI updater Ostro's swupd uses efi_combo_updater app to update the boot partitions. This apps relies on sgdisk presence which is a part of gptfdisk. Signed-off-by: Dmitry Rozhkov <[email protected]>
Recording volatile information in the os-release package is conceptually problematic: either the package gets updated during each build and thus causes constant image rebuilds, or it does not get updated and then old, incorrect information gets copied into images. In Ostro OS, we do not have to keep packaged files unmodified during image creation, so the solution is to record static strings in the os-release package and replace that with the actual, current values during image creation. This now works in the CI and in local builds, therefore recording BUILD_ID can be enabled for both. Signed-off-by: Patrick Ohly <[email protected]>
|
👎 must be removed. |
|
On Thu, 2016-04-07 at 05:58 -0700, Alexander D. Kanevskiy wrote:
Can't test that here, so I triggered a test build with such a change |
|
We anyway would need to re-test this PR after ostroproject/ostro-os#51 merged. |
The bundle information created by meta-swupd and the VERSION_ID used
in the os-release file must match, and therefore the value must be set
at the distro level. OS_VERSION is the variable used by meta-swupd,
so that is what is getting used in all places.
How OS_VERSION is set differs between CI and local builds. For the CI,
the expectation is that long-term the CI system will determine the
right OS_VERSION and set it explicitly. As fallback until then, the
OS_VERSION is derived from the integer part of the BUILD_ID set in the
Jenkins CI by multiplying it by 10.
During a local build, we can explicitly use ${DATETIME} to have a
reasonable default if developers do not set OS_VERSION manually. The
default is to use a modified OS_VERSION if something else triggers an
image build, but a OS_VERSION change itself won't trigger an image and
bundle build.
Signed-off-by: Patrick Ohly <[email protected]>
…images
The same build configuration can be used to build images with or without
swupd support. When building images with swupd support, swupd bundles
are automatically created, too.
Image variants are no longer supported. Instead, most of the Ostro
image code is now in ostro-image.bbclass and gets inherited by
separate image recipes. This was necessary because the swup utility
class cannot be combined with the image variant class (both use the
same "virtual image" mechanism). More importantly, having an
ostro-image.bbclass also allows OEMs to define their own custom image
recipes, which was not possible before.
Right now, bundles are defined such that they match the previous image
features where image features where used to select additional
software. Image features which change the OS behavior still exist and
need to be set for the base image.
This large number of bundles currently slows down builds considerably.
Developers working with bundle images are expected to customize the
set of bundles in their local.conf or their custom image recipe.
Developers building locally without swupd are expected to pick which
components and features they want in their custom "ostro-image" and
directly build that.
The following image recipes are currently provided:
* ostro-image-swupd.bb - the base image for swupd images and bundles.
All images derived from it share the same swupd update stream,
so it is possible to add and remove content. Actual images
just differ in the set of pre-installed bundles.
Right now, those images are:
* ostro-image-swupd-reference - minimal reference image with shell
access enabled.
* ostro-image-swupd-dev - image with all components pre-installed
together with development tools and files. Used for QA testing
and on-target development.
* ostro-image-noswupd.bb - for local builds. When using development
mode, additional components get selected such that the content
matches ostro-image-swupd-reference.
While cleaning up, OSTRO_EXTRA_IMAGE_FEATURES got renamed to
OSTRO_IMAGE_EXTRA_FEATURES because that is consistent with other
variables in ostro-image.bb and elsewhere (prefix derived from file).
tools-profile was not used and does not compile in Ostro OS when
trying to use it, therefore it gets removed.
Signed-off-by: Patrick Ohly <[email protected]>
Builds are too slow in the CI to be useful (> 3 hours). Performance enhancements are possible and planned, but not available yet. As the initial goal is primarily about whole image update with bundle support as stretch goal, reducing the number of bundles to just those that are needed to build the example images is acceptable and will reduce build times again. Signed-off-by: Patrick Ohly <[email protected]>
The automatic CI testing looks for a compressed "ostro-image" under various names (depending on the platform), both in the download area and (in the case of Edison) also inside archives. We no longer build that and instead want to get "ostro-image-swupd-dev" tested. Until the CI testing becomes configurable, provide the expected files via a symlink. Signed-off-by: Patrick Ohly <[email protected]>
Ostro OS tries to build minimal images and thus prefers Busybox or Toybox over coreutils or any other package providing similar functionality. For technical reasons, this is crucial when building swupd bundles: the component that gets chosen must be consistent across bundles and must be installed in the os-core bundle, otherwise we end up with dangling symlinks when the symlink is in the os-core but the component it points to is not. The choice is made in the ostro-image.bbclass to ensure consistency between local, personal developer builds, regardless whether it is with or without swupd, and CI builds. Developers can still pick coreutils in custom images by completely removing Busybox/Toybox. The priorities of Toybox/Busybox must be higher than those of any alternatives. procps uses 200, so we start at 300. Both systemd and the efi_combo_updater have problems when "mount" is provided by busybox: systemd fails to remount the rootfs read/write and the updater segfaults because it does not parse the output correctly. For now we avoid these problems by sticking to the traditional mount utilities from util-linux. Signed-off-by: Patrick Ohly <[email protected]>
The EFI combo update works only on dsk images. Therefore it should only get installed and activated on architectures using the dsk image format, and that choice should be made in a central place like ostro.conf. For this to work, the .service file which hooks the efi combo updater into cannot be in the allarch oe-swupd-helpers, because that must have the same content regardless of the architecture. It seems simpler to do it all in a single .bbappend anyway, so now efi-combo-trigger.service is part of the swupd-client.bbappend, just like the efi-combo-trigger binary itself. A side effect is that developers can override that choice by setting the new OSTRO_USE_DSK_IMAGES, which may be useful for debugging. Upstream swupd provides a "check-update" systemd unit which runs regularly to check for updates. In Ostro OS, that unit is not configured and thus gets disabled entirely. Signed-off-by: Patrick Ohly <[email protected]>
Command line tools were all referenced by absolute paths instead of relying on PATH. Ostro OS does no specify where exactly each tool is located (although there is an OE-core convention for tools provided via update-alternatives), so better rely on searching via PATH. Signed-off-by: Patrick Ohly <[email protected]>
When replacing busybox with toybox (to be committed separately), several commands will disappear from the base image (ping, top, ps, vi, etc.). This is acceptable for a base image, but not for something that people use interactively or the QA test scripts. Therefore a new "tools-interactive" image feature gets introduced: - it is active in all image variants at the moment - the corresponding package group installs commands not provided anymore by toybox, choosing the corresponding individual packages instead - bash is installed but not used as /bin/sh, which always points to dash The goal is to ensure consistent behavior, regardless whether coreutils, busybox or toybox are installed. Signed-off-by: Patrick Ohly <[email protected]>
The initial commit of packagegroup-tools-interactive.bb only had the essentials. To make interactive use more convenient, we can and should also include additional commands that may be expected, like successor(s) of top and the corresponding tools for TCP and I/O. Also restores sort order in PNWHITELIST_openembedded-layer. Signed-off-by: Patrick Ohly <[email protected]>
…/rsync) wget is required for QA tests, the other two might be expected by users. Signed-off-by: Patrick Ohly <[email protected]>
Unless explicitly requested, tools like ps, wget, curl and thus also their dependencies like ca-certificates are not installed in the "ostro-image-swupd-reference" and thus also not in "ostro-image-noswup". As the reference image is expected to be used interactively, better add the corresponding feature. Signed-off-by: Patrick Ohly <[email protected]>
The normal log level only puts progress messages into the log file where is is not visible unless developers know where to look. In a CI system the output is not accessible at all. Using bbplain for progress messages is a bit unusual and thus not done by default in meta-swupd, but okay (they just do not get formatted as NOTES). A dedicated logging level for progress notifications would be better, but does not exist in bitbake at the moment. Signed-off-by: Patrick Ohly <[email protected]>
Dangling symlinks have been a problem in particular in swupd-based images, because it can happen that the actual file that gets pointed to is only part of a bundle, while the link is part of the os-core. This new check covers this. It gets applied to all images, whether they are based on swupd or not, because we also do not want dangling symlinks in non-swupd images. Signed-off-by: Patrick Ohly <[email protected]>
|
Yes, this PR needs to be tested. @mythi please merge ostroproject/ostro-os#51 as soon as possible. If the test build in PR #25 passes, then I will add that additional commit also here. |
| iotivity \ | ||
| java-jdk \ | ||
| node-runtime \ | ||
| nodejs-runtime-tools \ | ||
| python-runtime \ | ||
| qatests \ | ||
| smack \ | ||
| ssh-server \ | ||
| soletta \ | ||
| soletta-tools \ |
There was a problem hiding this comment.
what is soletta-tools x soletta? In this image only installing the sol-fbp-runner is enough, it should bring the dependencies such as other libraries and modules. and I'd call it similar to other runtimes (python-runtime, java-runtime, soletta-runtime).
For -dev images, installing soletta-dev, which should include sol-fbp-generator and other scripts would be necessary.
Last but not least, java-jdk, node-runtime and nodejs-runtime-tools, can we have a single name pattern?
There was a problem hiding this comment.
In this PR, I just followed the existing naming of image features. Let's clean this up separately.
There was a problem hiding this comment.
To answer to @barbieri: soletta-tools is simply soletta-dev-app. soletta in turn is what soletta.bb gives (the package group used to have more packages but they are now pulled in with RDEPENDS)
| # Add bash because it is more convenient to use than dash. | ||
| # This does not change /bin/sh due to re-organized update-alternative | ||
| # priorities. | ||
| FEATURE_PACKAGES_tools-interactive = "packagegroup-tools-interactive bash" |
There was a problem hiding this comment.
please add more useful helpers for interactive use, like bash-completion? :-)
I'm thinking the case where we have all daemons split in command line clients and actual server, in that case should we add the cmdline clients here? like connmanctl, systemctl, systemd console agents... It would be nice to make this more automatic, so cmdline tools split packages can be auto pulled in.
nothing to be done now, but let's think about it.
|
hi @pohly, Overall I like it and would recommend to include ASAP. I did some minor comments, some are not even related to your PR (like efi_combo_updater, which I've created an issue #96). Those comments about interactive and shell can be reviewed later, as we clean some parts the others will become cleaner. |
|
test this please |
When configuring the image to be tested, symlinks are no longer needed. Signed-off-by: Patrick Ohly <[email protected]>
|
The latest build in #25 with symlink workaround removed failed on Gigabytes and Minnowmax because of ssh connectivity issues between test host and test devices. Not sure whether that was related to anything in the PR itself. In this PR, I kicked of two builds: https://ostroproject.org/jenkins/view/Pull-Requests/job/dsl-meta-ostro_pull-requests/200/ with the symlink workaround, https://ostroproject.org/jenkins/view/Pull-Requests/job/dsl-meta-ostro_pull-requests/201/ with the workaround removed via commit 15aa34d. |
This was referenced Apr 9, 2016
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This completes the integration of meta-swupd. The PR is ready for merging.