Skip to content

Commit

Permalink
doc: compression format
Browse files Browse the repository at this point in the history 
describes internals of signed image with compressed payload.

Signed-off-by: Mateusz Michalek <[email protected]>
Signed-off-by: Anna Wojdylo <[email protected]>
  • Loading branch information
@michalek-no @nvlsianpu
michalek-no authored and nvlsianpu committed Feb 10, 2025
1 parent 724ee7e commit 25e2761
Show file tree
Hide file tree
Showing 3 changed files with 263 additions and 35 deletions.
172 changes: 172 additions & 0 deletions docs/compression_format.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# Compressed binary file internals

This article describes the structure of the
`zephyr.signed.bin` file when image
compression is enabled. You do not need to know these details to use the
image compression subsystem, but they can be beneficial if you want to
use them for verification or custom integration purposes.

For an example, see the following structure of the file:

![LZMA header](./images/decomp.png)

## [LZMA Header](#LZMA-Header)

The Lempel-Ziv-Markov chain Algorithm (LZMA) header is crucial for files
compressed using the LZMA method. It contains metadata essential for
decompression. The `lzma2_header` encodes compression parameters using
two bytes.

### [Calculating compression parameters](#Calculating-compression-parameters)

Compression parameters can be calculated, retrieved, or changed
depending on your needs. For details, see the following sections.

#### [Default values](#Default-values)

Compression parameters have the following default values:

- `dict_size`: 131072
- `pb`: 2
- `lc`: 3
- `lp`: 1

#### [Adjusting dictionary size](#Adjusting-dictionary-size)

You can calculate the `dict_size` using the following method:

``` {.c}
unsigned int i = 0;
for (i = 0; i < 40; i++) {
if (raw_dict_size <= (((uint32_t)2 | ((i) & 1)) << ((i) / 2 + 11))) {
break;
}
}
dict_size = (uint8_t)i;
```

With this method, `dict_size` can have one of the following values:

|Hex Value | Size |
|-----------|------------|
|0x00 | 4096 |
|0x01 | 6144 |
|0x02 | 8192 |
|0x03 | 12288 |
|0x04 | 16384 |
|0x05 | 24576 |
|0x06 | 32768 |
|0x07 | 49152 |
|0x08 | 65536 |
|0x09 | 98304 |
|0x0a | 131072 |
|0x0b | 196608 |
|0x0c | 262144 |
|0x0d | 393216 |
|0x0e | 524288 |
|0x0f | 786432 |
|0x10 | 1048576 |
|0x11 | 1572864 |
|0x12 | 2097152 |
|0x13 | 3145728 |
|0x14 | 4194304 |
|0x15 | 6291456 |
|0x16 | 8388608 |
|0x17 | 12582912 |
|0x18 | 16777216 |
|0x19 | 25165824 |
|0x1a | 33554432 |
|0x1b | 50331648 |
|0x1c | 67108864 |
|0x1d | 100663296 |
|0x1e | 134217728 |
|0x1f | 201326592 |
|0x20 | 268435456 |
|0x21 | 402653184 |
|0x22 | 536870912 |
|0x23 | 805306368 |
|0x24 | 1073741824 |
|0x25 | 1610612736 |
|0x26 | 2147483648 |
|0x27 | 3221225472 |

#### [Calculating literal context, literal pos, and pos bits](#Calculating-literal-context-literal-pos-and-pos-bits)

The second byte of the `lzma2_header` carries the following parameters:

- `lc`, which specifies a number of literal context bits

- `lp`, which specifies a number of literal pos bits

- `pb`, which specifies a number of pos bits

These parameters are encoded with the following formula:

``` {.c}
pb_lp_lc = (uint8_t)((pb * 5 + lp) * 9 + lc);
```
To decode these values from the combined `pb_lp_lc` byte, run the
following code:
``` {.c}
lc = pb_lp_lc % 9;
pb_lp_lc /= 9;
pb = pb_lp_lc / 5;
lp = pb_lp_lc % 5;
```
## [Extracting LZMA stream from image](#Extracting-LZMA-stream-from-image)
To extract and decompress the LZMA stream from the image, follow these
steps:
1. Determine the offset of the compressed stream by adding the
`lzma2_header` size and the value stored under
`image_header.ih_hdr_size`. For the size of the compressed stream,
see `image_header.ih_img_size`.
2. If the compressed stream is isolated and stored in a file named
`raw.lzma`, you can perform
decompression using the following commands:
- Without an ARM thumb filter:
``` {.bash}
unlzma --lzma2 --format=raw --suffix=.lzma raw.lzma
```
- With an ARM thumb filter:
``` {.bash}
unlzma --armthumb --lzma2 --format=raw --suffix=.lzma raw.lzma
```
Once the command is executed you will see a newly created file named
`raw`, which is identical to the
image before compression.
## [TLVs](#TLVs)
The following Type-Length-Values (TLVs) are used in the context of
decompressed images:
- `DECOMP_SIZE (0x70)`: Specifies the size of the decompressed image.
- `DECOMP_SHA (0x71)`: Contains the hash of the decompressed image.
- `DECOMP_SIGNATURE (0x72)`: Holds the signature of either the hash or
the entire image.
These TLVs are placed in the protected TLV section, ensuring they are
included in the hashing and signature calculations during the
verification process. The process for choosing the type of cryptographic
signature and hash algorithm used for securing the image is the same,
regardless of whether the image has undergone compression.
## [Sample](#Sample)
For practical implementation, you can find a simple stand-alone
verification program under the following path
`bootloader/mcuboot/samples/compression_test/independent_cmp.c`
This program demonstrates how to independently verify the integrity and
authenticity of a decompressed image using the specified TLVs.
Binary file added docs/images/decomp.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
126 changes: 91 additions & 35 deletions docs/imgtool.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,50 +51,100 @@ enabled, this last step is unnecessary and can be skipped.
Image signing takes an image in binary or Intel Hex format intended for the
primary slot and adds a header and trailer that the bootloader is expecting:

Usage: imgtool.py sign [OPTIONS] INFILE OUTFILE
Usage: imgtool sign [OPTIONS] INFILE OUTFILE

Create a signed or unsigned image

INFILE and OUTFILE are parsed as Intel HEX if the params have .hex
extension, otherwise binary format is used

Options:
--vector-to-sign [payload|digest]
send to OUTFILE the payload or payloads
digest instead of complied image. These data
can be used for external image signing
--sha [auto|256|384|512] selected sha algorithm to use; defaults to
"auto" which is 256 if no cryptographic
signature is used, or default for signature
type
--sig-out filename Path to the file to which signature will be
written. The image signature will be encoded
as base64 formatted string
--pure Expected Pure variant of signature; the Pure
variant is expected to be signature done
over an image rather than hash of that
image.
--fix-sig-pubkey filename public key relevant to fixed signature
--fix-sig filename fixed signature for the image. It will be
used instead of the signature calculated
using the public key
-k, --key filename
--public-key-format [hash|full]
--align [1|2|4|8|16|32] Alignment used by swap update modes.
-v, --version TEXT [required]
-s, --security-counter TEXT Specify the value of security counter. Use
the `auto` keyword to automatically generate
it from the image version.
-d, --dependencies TEXT
--pad-sig Add 0-2 bytes of padding to ECDSA signature
(for MCUboot <1.5)
-H, --header-size INTEGER [required]
--pad-header Add --header-size zeroed bytes at the
beginning of the image
-S, --slot-size INTEGER Size of the slot where the image will be
written [required]
--pad Pad image to --slot-size bytes, adding
trailer magic
--confirm When padding the image, mark it as confirmed
-M, --max-sectors INTEGER When padding allow for this amount of
sectors (defaults to 128)
--boot-record sw_type Create CBOR encoded boot record TLV. The
sw_type represents the role of the software
component (e.g. CoFM for coprocessor
firmware). [max. 12 characters]
--overwrite-only Use overwrite-only instead of swap upgrades
-e, --endian [little|big] Select little or big endian
-E, --encrypt filename Encrypt image using the provided public key
--save-enctlv When upgrading, save encrypted key TLVs
instead of plain keys. Enable when
BOOT_SWAP_SAVE_ENCTLV config option was set.
-L, --load-addr INTEGER Load address for image when it should run
from RAM.
-x, --hex-addr INTEGER Adjust address in hex output file.
-R, --erased-val [0|0xff] The value that is read back from erased
flash.
-h, --help Show this message and exit.
In what format to add the public key to the
image manifest: full key or hash of the key.
--max-align [8|16|32] Maximum flash alignment. Set if flash
alignment of the primary and secondary slot
differ and any of them is larger than 8.
--align [1|2|4|8|16|32] Alignment used by swap update modes.
-v, --version TEXT [required]
-s, --security-counter TEXT Specify the value of security counter. Use
the `auto` keyword to automatically generate
it from the image version.
-d, --dependencies TEXT Add dependence on another image, format:
"(<image_ID>,<image_version>), ... "
--pad-sig Add 0-2 bytes of padding to ECDSA signature
(for mcuboot <1.5)
-H, --header-size INTEGER [required]
--pad-header Add --header-size zeroed bytes at the
beginning of the image
-S, --slot-size INTEGER Size of the slot. If the slots have
different sizes, use the size of the
secondary slot. [required]
--pad Pad image to --slot-size bytes, adding
trailer magic
--confirm When padding the image, mark it as confirmed
(implies --pad)
-M, --max-sectors INTEGER When padding allow for this amount of
sectors (defaults to 128)
--boot-record sw_type Create CBOR encoded boot record TLV. The
sw_type represents the role of the software
component (e.g. CoFM for coprocessor
firmware). [max. 12 characters]
--overwrite-only Use overwrite-only instead of swap upgrades
-e, --endian [little|big] Select little or big endian
-c, --clear Output a non-encrypted image with encryption
capabilities,so it can be installed in the
primary slot, and encrypted when swapped to
the secondary.
--skip-encryption Set encryption flags and TLV's without
applying encryption.
--compression [disabled|lzma2|lzma2armthumb]
Enable image compression using specified
type. Will fall back without image
compression automatically if the compression
increases the image size.
--encrypt-keylen [128|256] When encrypting the image using AES, select
a 128 bit or 256 bit key len.
-E, --encrypt filename Encrypt image using the provided public key.
(Not supported in direct-xip or ram-load
mode.)
--save-enctlv When upgrading, save encrypted key TLVs
instead of plain keys. Enable when
BOOT_SWAP_SAVE_ENCTLV config option was set.
-F, --rom-fixed INTEGER Set flash address the image is built for.
-L, --load-addr INTEGER Load address for image when it should run
from RAM.
-x, --hex-addr INTEGER Adjust address in hex output file.
-R, --erased-val [0|0xff] The value that is read back from erased
flash.
--custom-tlv [tag] [value] Custom TLV that will be placed into
protected area. Add "0x" prefix if the value
should be interpreted as an integer,
otherwise it will be interpreted as a
string. Specify the option multiple times to
add multiple TLVs.
--non-bootable Mark the image as non-bootable.
-h, --help Show this message and exit.

The main arguments given are the key file generated above, a version
field to place in the header (1.2.3 for example), the alignment of the
Expand All @@ -111,6 +161,12 @@ the load address (in Intel Hex terms, the Extended Linear Address record) to
adjust for the new bytes prepended to the file. The load address of all data
existing in the file should not change.

The `--compression` option enables LZMA compression over payload. Details
about internals of image generated with this option can be found here
[here](./compression_format.md)
This isn't fully supported on the embedded side but can be utilised when
project is built on top of the mcuboot.

The `--slot-size` argument is required and used to check that the firmware
does not overflow into the swap status area (metadata). If swap upgrades are
not being used, `--overwrite-only` can be passed to avoid adding the swap
Expand Down

0 comments on commit 25e2761

Please sign in to comment.