Merge branch 'main' into dev

Scott Powell · Scott Powell · commit e31017be1ad6 · 2025-06-29T20:04:03.000+10:00
diff --git a/docs/faq.md b/docs/faq.md
@@ -64,9 +64,13 @@ author: https://github.com/LitBomb<!-- omit from toc -->
   - [6.3. Q: How to connect to a repeater via BLE (Bluetooth)?](#63-q-how-to-connect-to-a-repeater-via-ble-bluetooth)
   - [6.4. Q: I can't connect via Bluetooth, what is the Bluetooth pairing code?](#64-q-i-cant-connect-via-bluetooth-what-is-the-bluetooth-pairing-code)
   - [6.5. Q: My Heltec V3 keeps disconnecting from my smartphone.  It can't hold a solid Bluetooth connection.](#65-q-my-heltec-v3-keeps-disconnecting-from-my-smartphone--it-cant-hold-a-solid-bluetooth-connection)
+  - [6.6. Q: My RAK/T1000-E/xiao\_nRF52 device seems to be corrupted, how do I wipe it clean to start fresh?](#66-q-my-rakt1000-exiao_nrf52-device-seems-to-be-corrupted-how-do-i-wipe-it-clean-to-start-fresh)
+  - [6.7. Q: WebFlasher fails on Linux with failed to open](#67-q-webflasher-fails-on-Linux-with-failed-to-open)
+
 - [7. Other Questions:](#7-other-questions)
-  - [7.2 Q: How to update ESP32-based devices over the air?](#72-q-how-to-update-esp32-based-devices-over-the-air)
   - [7.1 Q: How to update nRF (RAK, T114, Seed XIAO) repeater and room server firmware over the air using the new simpler DFU app?](#71-q-how-to-update-nrf-rak-t114-seed-xiao-repeater-and-room-server-firmware-over-the-air-using-the-new-simpler-dfu-app)
+  - [7.2 Q: How to update ESP32-based devices over the air?](#72-q-how-to-update-esp32-based-devices-over-the-air)
+  - [7.3 Q: Is there a way to lower the chance of a failed OTA device firmware update (DFU)?](#73-q-is-there-a-way-to-lower-the-chance-of-a-failed-ota-device-firmware-update-dfu)
 
 ## 1. Introduction
 
@@ -533,9 +537,57 @@ You can get the epoch time on <https://www.epochconverter.com/> and use it to se
 
 **A:** Heltec V3 has a very small coil antenna on its PCB for Wi-Fi and Bluetooth connectivity.  It has a very short range, only a few feet.  It is possible to remove the coil antenna  and replace it with a 31mm wire.  The BT range is much improved with the modification.
 
+### 6.6. Q: My RAK/T1000-E/xiao_nRF52 device seems to be corrupted, how do I wipe it clean to start fresh?
+
+**A:** 
+1. Connect USB-C cable to your device, per your device's instruction, get it to flash mode:
+    - For RAK, double click its reset button
+    - For T1000-e, quickly disconnect and reconnect the magnetic side of the cable from the device TWICE
+    - For Heltec T114, click the reset button once (the bottom button)
+    - For Xiao nRF52, click the reset button once.  If that doesn't work, quickly double click the reset button twice.  If that doesn't work, disconnection the board from your PC and reconnect again ([seeed studio wiki](https://wiki.seeedstudio.com/XIAO_BLE/#access-the-swd-pins-for-debugging-and-reflashing-bootloader))
+5. A new folder will appear on your computer's desktop
+6. Download the `flash_erase*.uf2` file for your device on flasher.meshcore.co.uk 
+    - RAK WisBlock and Heltec T114: `Flash_erase-nRF32_softdevice_v6.uf2`
+    - Seeed Studio Xiao nRF52 WIO: `Flash_erase-nRF52_softdevice_v7.uf2`
+8. drag and drop the uf2 file for your device to the root of the new folder
+9. Wait for the copy to complete.  You might get an error dialog, you can ignore it
+10. Go to https://flasher.meshcore.co.uk/, click `Console` and select the serial port for your connected device 
+11. In the console, press enter.  Your flash should now be erased
+12. You may now flash the latest MeshCore firmware onto your device
+
+Separately, starting in firmware version 1.7.0, there is a CLI Rescue mode.  If your device has a user button (e.g. some RAK, T114), you can activate the rescue mode by hold down the user button of the device within 8 seconds of boot.  Then you can use the 'Console' on flasher.meshcore.co.uk 
+
+
+### 6.7. Q: WebFlasher fails on Linux with failed to open
+
+**A:** If the usb port doesn't have the right ownership for this task, the process fails with the following error:
+`NetworkError: Failed to execute 'open' on 'SerialPort': Failed to open serial port.`
+
+Allow the browser user on it:
+`# setfacl -m u:YOUR_USER_HERE:rw /dev/ttyUSB0`
+
 ---
 ## 7. Other Questions:
 
+### 7.1 Q: How to update nRF (RAK, T114, Seed XIAO) repeater and room server firmware over the air using the new simpler DFU app?
+
+**A:** The steps below work on both Android and iOS as nRF has made both apps' user interface the same on both platforms:
+
+1. Download nRF's DFU app from iOS App Store or Android's Play Store, you can find the app by searching for `nrf dfu`, the app's full name is `nRF Device Firmware Update`
+2. On flasher.meshcore.co.uk, download the **ZIP** version of the firmware for your nRF device (e.g. RAK or Heltec T114 or Seeed Studio's Xiao)
+3. From the MeshCore app, login remotely to the repeater you want to update with admin priviledge
+4. Go to the Command Line tab, type `start ota` and hit enter.
+5. you should see `OK` to confirm the repeater device is now in OTA mode
+6. Run the DFU app,tab `Settings` on the top right corner
+7. Enable `Packets receipt notifications`, and change `Number of Packets` to 10 for RAK, 8 for T114.  8 also works for RAK.  
+9. Select the firmware zip file you downloaded
+10. Select the device you want to update. If the device you want to updat is not on the list, try enabling`OTA` on the device again
+11. If the device is not found, enable `Force Scanning` in the DFU app
+12. Tab the `Upload` to begin OTA update
+13. If it fails, try turning off and on Bluetooth on your phone.  If that doesn't work, try rebooting your phone.  
+14. Wait for the update to complete.  It can take a few minutes.
+
+
 ### 7.2 Q: How to update ESP32-based devices over the air?
 
 **A:** For ESP32-based devices (e.g. Heltec V3):
@@ -548,22 +600,19 @@ You can get the epoch time on <https://www.epochconverter.com/> and use it to se
 8. From a browser, go to http://192.168.4.1/update and upload the non-merged bin from the flasher
 
 
-### 7.1 Q: How to update nRF (RAK, T114, Seed XIAO) repeater and room server firmware over the air using the new simpler DFU app?
+### 7.3 Q: Is there a way to lower the chance of a failed OTA device firmware update (DFU)?
+
+**A:** Yes, developer `che aporeps` has an enhanced OTA DFU bootloader for nRF52 based devices.  With this bootloader, if it detects that the application firmware is invalid, it falls back to OTA DFU mode so you can attempt to flash again to recover.  This bootloader has other changes to make the OTA DFU process more fault tolerant. 
+
+Refer to https://github.com/oltaco/Adafruit_nRF52_Bootloader_OTAFIX for the latest information.
+
+Currently, the following boards are supported:
+- Nologo ProMicro
+- Seeed Studio XIAO nRF52840 BLE
+- Seeed Studio XIAO nRF52840 BLE SENSE
+- RAK 4631
 
-**A:** The steps below work on both Android and iOS as nRF has made both apps' user interface the same on both platforms:
 
-1. Download nRF's DFU app from iOS App Store or Android's Play Store, you can find the app by searching for `nrf dfu`, the app's full name is `nRF Device Firmware Update`
-2. On flasher.meshcore.co.uk, download the **ZIP** version of the firmware for your nRF device (e.g. RAK or Heltec T114 or Seeed Studio's Xiao)
-3. From the MeshCore app, login remotely to the repeater you want to update with admin priviledge
-4. Go to the Command Line tab, type `start ota` and hit enter.
-5. you should see `OK` to confirm the repeater device is now in OTA mode
-6. Run the DFU app,tab `Settings` on the top right corner
-7. Enable `Packets receipt notifications` and change `Number of Packets` to 10 for RAK, 8 for T114.  8 also works for RAK.  
-8. Select the firmware zip file you downloaded
-9. Select the device you want to update. If the device you want to updat is not on the list, try enabling`OTA` on the device again
-10. Tab the `Upload` to begin OTA update
-11. If it fails, try turning off and on Bluetooth on your phone.  If that doesn't work, try rebooting your phone.  
-12. Wait for the update to complete.  It can take a few minutes.
 
 
 ---
diff --git a/docs/packet_structure.md b/docs/packet_structure.md
@@ -0,0 +1,54 @@
+# Packet Structure
+
+| Field    | Size (bytes)                     | Description                                               |
+|----------|----------------------------------|-----------------------------------------------------------|
+| header   | 1                                | Contains routing type, payload type, and payload version. |
+| path_len | 1                                | Length of the path field in bytes.                        |
+| path     | up to 64 (`MAX_PATH_SIZE`)       | Stores the routing path if applicable.                    |
+| payload  | up to 184 (`MAX_PACKET_PAYLOAD`) | The actual data being transmitted.                        |
+
+Note: see the [payloads doc](./payloads.md) for more information about the content of payload.
+
+## Header Breakdown
+
+bit 0 means the lowest bit (1s place)
+
+| Bits  | Mask   | Field           | Description                                   |
+|-------|--------|-----------------|-----------------------------------------------|
+| 0-1   | `0x03` | Route Type      | Flood, Direct, Reserved - see below.          |
+| 2-5   | `0x3C` | Payload Type    | Request, Response, ACK, etc. - see below.     |
+| 6-7   | `0xC0` | Payload Version | Versioning of the payload format - see below. |
+
+## Route Type Values
+
+| Value  | Name                          | Description                          |
+|--------|-------------------------------|--------------------------------------|
+| `0x00` | `ROUTE_TYPE_TRANSPORT_FLOOD`  | Flood routing mode + transport codes |
+| `0x01` | `ROUTE_TYPE_FLOOD`            | Flood routing mode (builds up path). |
+| `0x02` | `ROUTE_TYPE_DIRECT`           | Direct route (path is supplied).     |
+| `0x03` | `ROUTE_TYPE_TRANSPORT_DIRECT` | direct route + transport codes       |
+
+## Payload Type Values
+
+| Value  | Name                      | Description                                   |
+|--------|---------------------------|-----------------------------------------------|
+| `0x00` | `PAYLOAD_TYPE_REQ`        | Request (destination/source hashes + MAC).    |
+| `0x01` | `PAYLOAD_TYPE_RESPONSE`   | Response to REQ or ANON_REQ.                  |
+| `0x02` | `PAYLOAD_TYPE_TXT_MSG`    | Plain text message.                           |
+| `0x03` | `PAYLOAD_TYPE_ACK`        | Acknowledgment.                               |
+| `0x04` | `PAYLOAD_TYPE_ADVERT`     | Node advertisement.                           |
+| `0x05` | `PAYLOAD_TYPE_GRP_TXT`    | Group text message (unverified).              |
+| `0x06` | `PAYLOAD_TYPE_GRP_DATA`   | Group datagram (unverified).                  |
+| `0x07` | `PAYLOAD_TYPE_ANON_REQ`   | Anonymous request.                            |
+| `0x08` | `PAYLOAD_TYPE_PATH`       | Returned path.                                |
+| `0x09` | `PAYLOAD_TYPE_TRACE`      | trace a path, collecting SNI for each hop.    |
+| `0x0F` | `PAYLOAD_TYPE_RAW_CUSTOM` | Custom packet (raw bytes, custom encryption). |
+
+## Payload Version Values
+
+| Value  | Version | Description                                       |
+|--------|---------|---------------------------------------------------|
+| `0x00` | 1       | 1-byte src/dest hashes, 2-byte MAC.               |
+| `0x01` | 2       | Future version (e.g., 2-byte hashes, 4-byte MAC). |
+| `0x02` | 3       | Future version.                                   |
+| `0x03` | 4       | Future version.                                   |
diff --git a/docs/payloads.md b/docs/payloads.md
@@ -1,25 +1,29 @@
 # Meshcore payloads
 Inside of each [meshcore packet](./packet_structure.md) is a payload, identified by the payload type in the packet header. The types of payloads are:
 
+* Node advertisement.
+* Acknowledgment.
+* Returned path.
 * Request (destination/source hashes + MAC).
 * Response to REQ or ANON_REQ.
 * Plain text message.
-* Acknowledgment.
-* Node advertisement.
+* Anonymous request.
 * Group text message (unverified).
 * Group datagram (unverified).
-* Anonymous request.
-* Returned path.
 * Custom packet (raw bytes, custom encryption).
 
 This document defines the structure of each of these payload types
 
+## Important concepts:
+
+* Node/channel hash: the first byte of the node or channel's public key
+
 # Node advertisement
 This kind of payload notifies receivers that a node exists, and gives information about the node
 
 | Field         | Size (bytes)    | Description                                              |
 |---------------|-----------------|----------------------------------------------------------|
-| public key    | 32              | Ed25519 public key                                       |
+| public key    | 32              | Ed25519 public key of the node                           |
 | timestamp     | 4               | unix timestamp of advertisement                          |
 | signature     | 64              | Ed25519 signature of public key, timestamp, and app data |
 | appdata       | rest of payload | optional, see below                                      |
@@ -37,20 +41,29 @@ Appdata
 
 Appdata Flags
 
-| Value  | Name      | Description                           |
-|--------|-----------|---------------------------------------|
-| `0x10` | location  | appdata contains lat/long information |
-| `0x20` | feature 1 | Reserved for future use.              |
-| `0x40` | feature 2 | Reserved for future use.              |
-| `0x80` | name      | appdata contains a node name          |
+| Value  | Name           | Description                           |
+|--------|----------------|---------------------------------------|
+| `0x01` | is chat node   | advert is for a chat node             |
+| `0x02` | is repeater    | advert is for a repeater              |
+| `0x03` | is room server | advert is for a room server           |
+| `0x10` | has location   | appdata contains lat/long information |
+| `0x20` | has feature 1  | Reserved for future use.              |
+| `0x40` | has feature 2  | Reserved for future use.              |
+| `0x80` | has name       | appdata contains a node name          |
 
 # Acknowledgement
+
+An acknowledgement that a message was received. Note that for returned path messages, an acknowledgement will be sent in the "extra" payload (see [Returned Path](#returned-path)) and not as a discrete ackowledgement. CLI commands do not require an acknowledgement, neither discrete nor extra.
+
 | Field    | Size (bytes) | Description                                                |
 |----------|--------------|------------------------------------------------------------|
 | checksum | 4            | CRC checksum of message timestamp, text, and sender pubkey |
 
 
 # Returned path, request, response, and plain text message
+
+Returned path, request, response, and plain text messages are all formatted in the same way. See the subsection for more details about the ciphertext's associated plaintext representation.
+
 | Field            | Size (bytes)    | Description                                          |
 |------------------|-----------------|------------------------------------------------------|
 | destination hash | 1               | first byte of destination node public key            |
@@ -60,11 +73,13 @@ Appdata Flags
 
 ## Returned path
 
+Returned path messages provide a description of the route a packet took from the original author. Receivers will send returned path messages to the author of the original message.
+
 | Field       | Size (bytes) | Description                                                                                  |
 |-------------|--------------|----------------------------------------------------------------------------------------------|
 | path length | 1            | length of next field                                                                         |
-| path        | see above    | a list of node hashes (one byte each) describing the route from us to the packet author      |
-| extra type  | 1            | extra, bundled payload type, eg., acknowledgement or response. See packet structure spec     |
+| path        | see above    | a list of node hashes (one byte each) |
+| extra type  | 1            | extra, bundled payload type, eg., acknowledgement or response. Same values as in [packet structure](./packet_structure.md) |
 | extra       | rest of data | extra, bundled payload content, follows same format as main content defined by this document |
 
 ## Request
@@ -156,18 +171,14 @@ Plaintext message
 
 # Group text message / datagram
 
-| Field        | Size (bytes)    | Description                              |
-|--------------|-----------------|------------------------------------------|
-| channel hash | 1               | TODO                                     | 
-| cipher MAC   | 2               | MAC for encrypted data in next field     |
-| ciphertext   | rest of payload | encrypted message, see below for details |
+| Field        | Size (bytes)    | Description                                |
+|--------------|-----------------|--------------------------------------------|
+| channel hash | 1               | the first byte of the channel's public key | 
+| cipher MAC   | 2               | MAC for encrypted data in next field       |
+| ciphertext   | rest of payload | encrypted message, see below for details   |
 
-Plaintext for text message
+The plaintext contained in the ciphertext matches the format described in [plain text message](#plain-text-message). Specifically, it consists of a four byte timestamp, a flags byte, and the message. The flags byte will generally be `0x00` because it is a "plain text message". The message will be of the form `<sender name>: <message body>` (eg., `user123: I'm on my way`).
 
-| Field     | Size (bytes)    | Description                      |
-|-----------|-----------------|----------------------------------|
-| timestamp | 4               | send time (unix timestamp)       |
-| content   | rest of message | plain group text message content |
 
 TODO: describe what datagram looks like
 
