[REQUEST] Ability to upgrade flash Texas Instrument CC2530 and CC2531 firmware via its bootloader interface

[Hedda]

@JelmerT Requesting this now as @puddly posted in zigpy/zigpy-znp#14 saying that he implemented the MT command based serial bootloader protocol (f67e8a4) implemented in CC2530 and CC2531 bootloader interface as used by the SerialBootTool (Windows only) utility by Texas Instruments mentioned in Koenkk/zigbee2mqtt#320 into zigpy-znp.

zigpy-znp is new open-source TI ZNP 3.x client written in Python that is in early development for zigpy and ZHA in Home Assistant. For more information check out https://github.com/zha-ng/zigpy-znp/

zigpy-znp developer was able to backup (read) and upgrade (write) flash his already pre-flashed Texas Instruments CC2531 adapter with a newer firmware without using any external tools, so as a standalone application to flash an adapter (in his case he tested it with a CC2531 adapter from ITead which is now known to be shipped with an older firmware from the factory)

• zigpy/zigpy-znp@f67e8a4

ip install zigpy-znp
$ python -m zigpy_znp.tools.flash_read /dev/serial/by-id/radio -o flash.bin
$ python -m zigpy_znp.tools.flash_write /dev/serial/by-id/radio -i flash.bin

Suggest considering adding the ability to upgrade these older Texas Instrument CC2530 and CC2531 coordinator firmware via cc2538-bsl to make it also compatible with those as well.

(TI's rialBootTool which also comes with IAR can otherwise be download separately here http://processors.wiki.ti.com/index.php/File:SerialBootTool_1_3_2.zip ).

[JelmerT]

Thanks for pointing this out @Hedda
Expanding compatibility to other micros has been a goal for a while now, and the code base should be able to support it (with some slight adjustments)

So if I understand this correctly, this bootloader is part of the TI zigbee stack (Z-stack) and has to be flashed to the chip before you can actually use it. (Blank chips out of the factory don't come with it).
It looks like the protocol is a lot simpeler than what is implemented in the CC2538 and up and this BSL was already used for the CC2510, CC2511, CC1110 and CC1111. (https://www.ti.com/lit/an/swra371a/swra371a.pdf)

Is there a document available about the protocol of this bootloader? (I found this: https://e2echina.ti.com/cfs-file/__key/communityserver-discussions-components-files/104/Serial_5F00_Boot_5F00_Loader_5F00_for_5F00_CC253x.pdf) Or do we need to look at the actual source of the bootloader itself.
And is there a list of all the chips it is / can be implemented on?

We would probably need a way to recognize this bootloader over serial and then figure out which chip it's running on. If this is possible I don't see a reason why not to implement it.

[puddly]

It's the Z-Stack management and testing protocol (MT). I have the latest spec copied in zigpy-znp, which includes a description of the frame format.

The bootloader isn't documented so you'll have to stare at sb_exec_v2.c from Z-Stack 3.0.1 to see how it works. It also doesn't appear to be present in all Z-Stack versions (namely, the latest) and appears to have an older version with different commands, which I've not seen or tested.

Detecting it should be possible by sending the handshake command. Unfortunately, they decided to implement it as an asynchronous request, meaning that Z-Stack will not respond to it if it doesn't understand it. It also seems to leave bootloader mode if any frames have been received that aren't from the bootloader command set.

If you change the logging level of python -m zigpy_znp.tools.flash_write to logging.DEBUG - 5, it will print out exactly the frames and bytes sent/received over serial, which should match the Windows tool. You can see a high-level description of the frames in zigpy_znp/commands/ubl.py.

[JelmerT]

The bootloader isn't documented so you'll have to stare at sb_exec_v2.c from Z-Stack 3.0.1 to see how it works. It also doesn't appear to be present in all Z-Stack versions (namely, the latest) and appears to have an older version with different commands, which I've not seen or tested.

Well that's not great.

It does seem to be the only serial bootloader that's widespread for the CC2530 and 31? And a derived (compatible?) version for CC2510, CC2511, CC1110 and CC1111 exists too according to https://www.ti.com/lit/an/swra371a/swra371a.pdf
Looks like the v2 version commands are defined here: https://github.com/ClementXu/Z-Stack-Home-1.2.2a.44539/blob/master/Projects/zstack/Utilities/BootLoad/Source/sb_exec_v2.h

I don't think this repo is the place to implement the whole Z-Stack management and testing protocol (MT), but adding the bootloader interfacing part sounds good to me.
I do like having things be as fool-proof as possible, so people don't lock themselves out. Which seems an issue with this bootloader, since its even pretty hard to figure out which chip exactly you're talking to?

I currently don't have a CC2530 dev board or usb-stick (I think). So this is pretty hard for me to implement. But I'm welcoming pull requests, and willing to get some hardware to test it.

[Hedda]

Maybe ask if @Koenkk and @kirovilya nicely if they happen to know any tricks on figuring out which chip exactly you're talking to as I know they have loads of experience from zigbee-herdsman. Plus IIRC I read somewhere that @Koenkk was thinking about implementing TI chip flashing in Zigbee2mqt(?).

PS: Tip on hardware to buy is the pre-flashed CC2531 dongle that ITead sells https://www.itead.cc/cc2531-usb-dongle.html as it's inexpensive, have distributors worldwide, and known to ship with an old firmware from factory so from that perspective would make a good target.

[Hedda]

Anyway, to clarify the limitations a couple of key factors:

• It is a requirement that this bootloader has to have already been flashed to the chip before you can actually use it these MT commands (TI management and testing protocol).

• It only present in Z-Stack Home 1.2.x and Z-Stack 3.0.x (so no longer in Z-Stack 3.1.x or newer).

In any case, as I understand it, Z-Stack 3.0.x is officially the latest version of Z-Stack that TI officially supports on any CC253x chip.

• https://www.ti.com/tool/Z-STACK

So the later Z-Stack 3.1.x or newer is will officially not be supported on CC2530 or CC2531 anyway.

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator

It looks like CC2538 is the only CC253x powerful enough to run Z-Stack 3.1.x in any case, so technically you can port Z-Stack 3.1.x to work on CC2538, as @kirovilya already did with @diyruz

• https://github.com/diyruz/diyruz_rt

• https://www.cnx-software.com/2020/05/25/zigbee-firmware-news-ti-z-stack-3-0-zigbee-for-cc2530-ptvo-zigbee-fw-configuration-tool/

[puddly]

Oh, I guess it is documented in swra371a.pdf, including the CRC location. That could have saved me some time. I still can't get my CRC verification code working (the bootloader computes it over all the image bytes + b'\x00\x00', skipping the two CRC bytes).

[Hedda]

Any further thoughts on this?

https://e2e.ti.com/support/wireless-connectivity/sub-1-ghz/f/156/t/585437

http://www.ti.com/litv/zip/swra371a (swra371a.zip with example code)

[puddly]

Finally got it: the CRC is computed by skipping the two pairs of CRC bytes: https://github.com/zha-ng/zigpy-znp/blob/7e9367480ec3a7602c215c59c27a9bafe604a28b/zigpy_znp/tools/flash_write.py#L21-L56

At this point the implementation in zigpy_znp.tools.flash_write is stable-ish.

I do like having things be as fool-proof as possible, so people don't lock themselves out. Which seems an issue with this bootloader, since its even pretty hard to figure out which chip exactly you're talking to?

The bootloader won't let you overwrite itself so you probably won't get locked out but if you manage to catch the chip while it's still in its bootloader, you should be able to tell a little bit about what chip you're talking to:

==>  UBL.HandshakeReq.Req()  # b'\xFE\x00\x4D\x04\x49'
<==  UBL.HandshakeRsp.Callback(  # b'\xFE\x12\x4D\x84\x00\x00\x00\x00\x00\x02\x40\x00\x00\x00\x00\x08\x00\x00\x00\x00\x00\x00\x91'
         Status=<BootloaderStatus.SUCCESS: 0>,
         BootloaderRevision=0,
         DeviceType=<BootloaderDeviceType.CC2530: 2>,  # it's hard to tell from the source code what this really is
         BufferSize=64,
         PageSize=2048,
         BootloaderCodeRevision=0
     )

Otherwise, you can ask the device if it even has a bootloader after it has booted by sending it a SYS.Version request. The response looks like this:

==>  SYS.Version.Req()
<==  SYS.Version.Rsp(
         TransportRev=2,
         ProductId=1,
         MajorRel=2,
         MinorRel=7,
         MaintRel=1,
         CodeRevision=20200417,
         BootloaderBuildType=<BootloaderBuildType.NON_BOOTLOADER_BUILD: 0>,
         BootloaderRevision=4294967295
     )

You might be able to send a hard reset request and have the device jump back into bootloader, though this might just restart the application code instead. Implementing the five or so MT commands is doable but it will take a little bit of work to get the command framing and parsing working.

[Hedda]

@JelmerT any further thoughts on application firmware upgrade for C2530 and CC2531 via your cc2538-bsl tool?

As @puddly wrote above, his latest revisions of zigpy_znp.tools.flash_write are now "stable-ish" enough to call "ALPHA":

https://github.com/zha-ng/zigpy-znp/tree/dev/zigpy_znp/tools

"You can flash CC2531ZNP-without-SBL.bin to your stick directly with zigpy_znp: python -m zigpy_znp.tools.flash_write -i /path/to/CC2531ZNP-without-SBL.bin /dev/serial/by-id/YOUR-CC2531"

[JelmerT]

Sorry for the delay on this.

I'm open to PRs to support this bootloader. As long as current features and functions of the already supported chips aren't changed and implementation is as user friendly as possible (read "it just works").

I currently don't have the bandwidth to do any active development on this though. I can do code review.