wip/3mdeb/qubes-wrapper

[Asiderr]

This patch is adding the fwupd wrapper for Qubes OS. The wrapper provides fwupd functionalities for Qubes R4.1.
It creates three packages (two RPMs and one deb):
fwupd-qubes-dom0 (RPM)
fwupd-qubes-vm (RPM)
fwupd-qubes-vm-whonix (deb)
More information about the wrapper could be found in the contrib/qubes/README.md

Signed-off-by: Norbert Kamiński [email protected]

Type of pull request:

• New plugin (Please include new plugin checklist)
• Code fix
• Feature
• Documentation

[lgtm-com]

This pull request introduces 3 alerts when merging b91d725 into f3510f5 - view on LGTM.com

new alerts:

• 3 for Duplication in regular expression character class

[hughsie]

Just the GPG keys? I think you want the PKCS ones too, no?

[Asiderr]

GPG keys are used three times to verify the metadata and firmware update files. The first time, they are used when the files are downloaded, the second time after copying the files from UpdateVM to Dom0, and the third time when Dom0 copies files to sys-usb VM. For now, we are not using the PKCS keys, but if we want to use jcat-tool they are available in standard fwupd directory /etc/pki/fwupd

[Asiderr]

I deleted these lines. GPG verification has been replaced with jcat verification.

[hughsie]

both?

[Asiderr]

We need both files to cover fwupd versions from 0.9.5 to the current one.

[superm1]

Why support such old fwupd versions?

[Asiderr]

Ferdora 25 is base for dom0 in Qubes R4.0 (https://www.qubes-os.org/doc/supported-versions/) and it provides fwupd 0.9.5.

[Asiderr]

The file doesn't exist. It's replaced with the python script. There is no need to download the GPG signature, thus I left only jcat file. The wrapper works correctly with the current libjcat master and the changes connected to the hughsie/libjcat#48 and hughsie/libjcat#49 need to be available on Fedora 32.

[hughsie]

libjcat update just went stable for f32: https://bodhi.fedoraproject.org/updates/FEDORA-2021-a83da4c566

[superm1]

Are there other packages in the Debian archive that have set precedent for this already?

[Asiderr]

There are no such packages.

[superm1]

I'm leaning on we should have some sort of flag that determines whether this binary package gets built as part of https://github.com/fwupd/fwupd/blob/master/contrib/ci/generate_debian.py

I don't think it should be done in general purpose Debian or Ubuntu, right? It should only be when the Debian derivatives Whonix generates the binary packages.

[Asiderr]

I don't think it should be done in general purpose Debian or Ubuntu, right?

That's right. It's Qubes specific package.

I'm leaning on we should have some sort of flag that determines whether this binary package gets built as part of https://github.com/fwupd/fwupd/blob/master/contrib/ci/generate_debian.py

It could be handled as another OS environment variable e.g. Debian-x86_64-Whonix and it would be added to the TARGET split: https://github.com/fwupd/fwupd/blob/master/contrib/ci/generate_debian.py#L24

[superm1]

Sounds like a good enough approach to me. If you find while you're doing it that doesn't end up scaling well enough feel free to add argparse support there instead though. The environment parsing was just a convenient way for our first CI jobs to do it.

[Asiderr]

There is a possibility to generate separate fedora-qubes and debian-x86_64-qubes containers that build qubes packages. Standard containers don't provide qubes DEBs and RPMs.

[lgtm-com]

This pull request introduces 4 alerts when merging 17f44be into 8fa65d9 - view on LGTM.com

new alerts:

• 3 for Duplication in regular expression character class
• 1 for Conflicting attributes in base classes

[lgtm-com]

This pull request introduces 4 alerts when merging 1ab7324 into 8fa65d9 - view on LGTM.com

new alerts:

• 3 for Duplication in regular expression character class
• 1 for Conflicting attributes in base classes

[superm1]

Similarly to the Debian case I think we want to special case the generation of the qubes binary package. This won't go into real Fedora archives.

[hughsie]

Completely agree.

[Asiderr]

There is a possibility to generate separate fedora-qubes and debian-x86_64-qubes containers that build qubes packages. Standard containers don't provide qubes DEBs and RPMs.

[hughsie]

The ColorHug2 firmware updates must provide SHA256 for testing purposes

Which version do you need resigned? I don't want to do them all ideally to avoid churn.

[lgtm-com]

This pull request introduces 3 alerts when merging d5ca239 into 4d8163c - view on LGTM.com

new alerts:

• 3 for Duplication in regular expression character class

[Asiderr]

Which version do you need resigned? I don't want to do them all ideally to avoid churn.

@hughsie Ideally one for update (2.0.7) and one for downgrade (2.0.6 or 2.0.5).

[hughsie]

@Asiderr: All three done!

[hughsie]

3 for Duplication in regular expression character class

There's also this warning from LGTM; but we're getting there now! Nearly there.

[lgtm-com]

This pull request introduces 1 alert when merging b336edb into 0cde61d - view on LGTM.com

new alerts:

• 1 for Unused import

[Asiderr]

@hughsie @superm1

There's also this warning from LGTM; but we're getting there now! Nearly there.

I fixed LGTM problems.

I've dropped GPG verification and now metadata and firmware updates are validated with jcat verification. I'm still waiting for metadata sync (fwupdagent still doesn't provide sha256 for ColorHug2 cabinets).

The wrapper has been successfully tested on Qubes R4.1. For test purposes, I've replaced jcat-tool in the following lines with the directories of the jcat-tool binary, built from the current master:

• fwupd_common_vm.py
• fwupd_receive_updates.py

[hughsie]

I'm still waiting for metadata sync

This should have happened yesterday, no? "fwupdmgr refresh --force"

[Asiderr]

@hughsie I guess so, but fwupdagent still doesn't provide the second checksum:

$ fwupdmgr refresh --force
Fetching metadata https://cdn.fwupd.org/downloads/firmware.xml.gz
Downloading…             [***************************************]
Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc

Successfully downloaded new metadata: 1 local device supported

$ fwupdagent get-updates
{
  "Devices" : [
    {
      "Name" : "ColorHug2",
      "DeviceId" : "983c3cffc6fd36d32b00b62928d30721eaeb93db",
      "Guid" : [
        "2082b5e0-7a64-478a-b1b2-e3404fab6dad",
        "aa4b4156-9732-55db-9500-bf6388508ee3",
        "101ee86a-7bea-59fb-9f89-6b6297ceed3b",
        "2fa8891f-3ece-53a4-adc4-0dd875685f30",
        "182ab329-1976-5d99-a662-ce618d2ccf88"
      ],
      "Summary" : "An open source display colorimeter",
      "Plugin" : "colorhug",
      "Protocol" : "com.hughski.colorhug",
      "Flags" : [
        "updatable",
        "supported",
        "registered",
        "self-recovery"
      ],
      "Vendor" : "Hughski Ltd.",
      "VendorId" : "USB:0x273F",
      "Version" : "2.0.6",
      "VersionFormat" : "triplet",
      "Icons" : [
        "colorimeter-colorhug"
      ],
      "InstallDuration" : 8,
      "Created" : 1612800403,
      "Releases" : [
        {
          "AppstreamId" : "com.hughski.ColorHug2.firmware",
          "RemoteId" : "lvfs",
          "Summary" : "Firmware for the Hughski ColorHug2 Colorimeter",
          "Description" : "<p>This release fixes prevents the firmware returning an error when the remote SHA1 hash was never sent.</p>",
          "Version" : "2.0.7",
          "Filename" : "658851e6f27c4d87de19cd66b97b610d100efe09",
          "Protocol" : "com.hughski.colorhug",
          "Checksum" : [
            "80bddeb898cda5b87d9837e13a9ace19846053bf"
          ],
          "License" : "GPL-2.0+",
          "Size" : 16384,
          "Uri" : "https://fwupd.org/downloads/0a29848de74d26348bc5a6e24fc9f03778eddf0e-hughski-colorhug2-2.0.7.cab",
          "Homepage" : "http://www.hughski.com/",
          "SourceUrl" : "https://github.com/hughski/colorhug2-firmware",
          "Vendor" : "Hughski Limited",
          "Flags" : [
            "is-upgrade"
          ],
          "InstallDuration" : 8
        }
      ]
    }
  ]
}

[hughsie]

Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc

That looks like a very old fwupd version; possibly one without the SHA256 support?

[Asiderr]

@hughsie The same result on Fedora32, fwupd 1.5.5 (updates-testing). Metadata contains the SHA256 of the .cab archive, but fwupdagent doesn't provide the checksum.

[hughsie]

Hmm, will investigate. https://bodhi.fedoraproject.org/updates/FEDORA-2021-a83da4c566 now exists for libjcat if that helps.

[hughsie]

Hmm, will investigate

Ha, so I think we accidentally fixed this in master a few weeks ago when we added the artifact support. I Can tag a new tarball release this week if that helps.

[Asiderr]

I Can tag a new tarball release this week if that helps.

This will help a lot. I will run the wrapper tests after the release.

[hughsie]

@Asiderr I've built 1.5.7 for F32 just now: https://bodhi.fedoraproject.org/updates/FEDORA-2021-449618a8a8 -- can you rebase this patchset, do the testing to fix up the remaining issues, and then I think we should aim to get this upstream before it bitrots any more than it has already.

[Asiderr]

@hughsie Tomorrow I'll rebase and test the newest changes and we can focus on the upstream. Also until the end of this week I want to push fwupd for BSD fixes.

[hughsie]

@superm1 can you verify the Debian stuff please.

[Asiderr]

@hughsie Wrapper is working fine. I added some minor fixes to the tests and docs. I'll push force one more change in a minute. The RPM packages must require specific versions of the libjcat and fwupd.