Mender integration for reflashing OTA update

[atharvanan1]

In past weeks, we had few people talking about integrating Matt's work on OTA reflashing with Mender update scheme.

Current way to upgrade highlighted in the above-mentioned document follows booting an upgrader-initramfs which contains tools - tegra-sysinstall - for reflashing.

A suggestion came on OE4T Slack, which suggested kicking off tegra-sysinstall script by setting up the running rootfs for a flashing update.

I'm starting this discussion to discuss better ways to integrate this with Mender.

As discussed with @dwalkes, one way to handle this would be to provide the initramfs + images as part of mender package, boot the initramfs such that:

1. it checks for images, verifies them, and reflashes the system
2. if images are not found, then boots the system

[arnstein]

This is the flow of the way I have currently solved this:

1. I have made a flash-image rootfs, which is a quite small image that have the most important tools for being able to run tegra-sysinstall.
2. I provide this image (as tar.gz) in a Mender update for the devices running warrior branch.
3. As a part of ArtifactReboot_Leave state in Mender update, I mount this rootfs as tmpfs, pivot roots and enables the "reflash systemd service"
4. This reflash.sh copies the contents in /data to a backupfolder, unmounts the eMMC partitions, runs tegra-sysinstall, make sure that the data partition is mounted after and copies back the /data backup to the new /data, then reboots

A couple of things to note here:

1. So far I have everything working, except for the fact that I didn't run it as an ArtifactReboot_Leave script yet. Will test that today.
2. It seems that sometimes after running tegra-sysinstall the first time it doesn't properly partition the eMMC. I have a "while mmcblk0p34 not mounted" check before restoring the contents to /data. I need this to fail after some time, and systemd will make sure the script is run again. I also think I saw @madisongh mention something about repartitioning save logic not being done, so maybe that is what is happening here. So far it has worked perfectly fine the second time I have ran the script after failing the first.
3. It is possible to set up a way to connect to your device running the tmpfs, for example through SSH, if you want to manually check everything before hitting reboot.

Steps going forward:

1. I want to make the reflash script a bit more robust and then test it with an actual Mender update. Once this is verified to be working, I will post the flash-image.bb, ArtifactReboot_Leave, reflash.service and reflash.sh files here.
2. I will look into the repartitioning save logic and adding so tegra-sysinstall will backup the UDA partlabel as well. Currently I am just manually backing it up as part of the reflash.sh.

[arnstein]

Update on this:
This is what I have been able to do a successful update from warrior-l4t-r32.2 branch to dunfell-l4t-r32.4.3 with.

The dunfell-rootfs and dunfell-bootloader are the recipes for the dunfell rootfs and dunfell BUP payload. I put them in /installer/dunfell-bup-payload and /installer/image.tar.gz. The reflash.sh will move the dunfell-bup-payload to /opt/ota_package/bl_update_payload, and tegra-sysinstall will use /installer/image/tar.gz.

NOTE:
These scripts aren't really robust, tested a lot or beautiful. This is the product of working fast to solve a problem. It seems to work fine for our usecase on a TX2. We will test it a lot more before rolling it out to production. While we are doing testruns I will look into the repartitioning save logic and UDA partlabel backup in tegra-sysinstall.
otareflash.zip