Add OTA metadata validation v2 #4998

willmmiles · 2025-10-11T14:56:29Z

Replaces #4960, fixes #4929.

Expand the OTA subsystem to add a framework for validating binaries before installing. Expands from #4960, squashed to make porting to 0.15 easier.

Key changes:

Adds a new metadata struct placed in a section where it is guaranteed to appear early in the .bin file
When performing an OTA update, validate that the release agrees with the current value (prevents ETH -> non-ETH downgrades, etc.)
Add a tickbox to the update UI to override checking
Print release info on update page.
Convert update page to using JSON API for retrieving metadata
Fix races and cleanup handling of OTA web requests

This implementation validates only release names. Additional validations can be added to shouldAllowOTA() in future PRs.

Summary by CodeRabbit

New Features
- Added robust OTA update flow with compatibility validation and clearer success/error feedback.
- Update page now auto-fetches device info (brand, version, release) and shows dynamic values.
- Option to ignore firmware validation during OTA.
Improvements
- Prevents forced reconnects while an OTA is in progress.
- Unified OTA handling for ESP8266/ESP32 with streamlined upload processing.
Bug Fixes
- Fixed inconsistent handling of update page scripts and version display.
Chores
- Consolidated build metadata/version handling into a single pre-build step.

Implement a comprehensive solution for validating a firmware before an OTA updated is committed. WLED metadata such as version and release is moved to a data structure located at near the start of the firmware binary, where it can be identified and validated. Co-authored-by: netmindz <[email protected]>

Improves cache utilization as fewer things are passed via CFLAGS to all files. In the event that no metadata is available, let the cpp file handle warning about default usage.

coderabbitai · 2025-10-11T14:56:37Z

Walkthrough

Consolidates version/repo metadata handling into a single build script and introduces a new build-time/runtime metadata system. Adds a stateful OTA update pipeline with metadata validation and integrates it into the server. Updates the OTA UI to show version/release and a validation bypass. Removes legacy version globals and related UI/XML handling.

Changes

Cohort / File(s)	Summary
Build scripts and config `pio-scripts/set_metadata.py`, `pio-scripts/set_version.py`, `platformio.ini`	New metadata script adds WLED_REPO/WLED_VERSION via middleware; removes old set_version script; updates PlatformIO to use set_metadata instead of set_version/set_repo.
OTA core integration `wled00/ota_update.h`, `wled00/ota_update.cpp`, `wled00/wled_server.cpp`, `wled00/wled.cpp`	Adds AsyncWebServer-based OTA context, chunked handling, validation hooks, and finalization; server routes use new context APIs; loop avoids reconnect during active Update.
Build metadata system `wled00/wled_metadata.h`, `wled00/wled_metadata.cpp`, `wled00/wled.h`	Introduces embedded metadata struct/section, compile-time/runtime hash checks, extraction and validation APIs; exposes repo/product/brand strings; removes legacy version/release/repo globals from header.
OTA UI and tooling `tools/cdata.js`, `wled00/data/update.htm`, `wled00/xml.cpp`	Update page implements GetV() fetching info and shows version/release with “Ignore firmware validation”; removes PAGE_update-specific mangle and XML subpage handling.
Minor adjustments `wled00/dmx_input.cpp`, `wled00/e131.cpp`	Renames local version string var in DMX config; adds cast for version parsing in E1.31.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

Revert disable OTA logic & optional Arduino OTA #4748 — Both PRs modify OTA-related code paths and handlers, overlapping in wled_server.cpp and OTA flow changes.

Suggested reviewers

rapek412-prog
blazoncek
DedeHai

Pre-merge checks and finishing touches

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title succinctly captures the primary enhancement of the pull request, namely the introduction of version 2 of OTA metadata validation, and directly reflects the main change without extraneous detail.
Linked Issues Check	✅ Passed	The changes fully satisfy issue #4929 by embedding release-name metadata at build time, extracting and validating it during OTA uploads, rejecting mismatches unless the user opts out via the new UI checkbox, and basing validation on binary metadata rather than filenames.
Out of Scope Changes Check	✅ Passed	All modified files and code paths relate directly to embedding, extracting, and validating OTA metadata or updating the OTA UI and server flow to support release-name checks; there are no unrelated feature changes outside the scope of OTA metadata validation.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 7

🧹 Nitpick comments (12)

wled00/data/update.htm (2)
27-28: Address TODO comments before merging.

The TODO comments suggest incomplete functionality:

"assemble update URL"

"can this be done at build time?"

Please resolve these items or create tracking issues for them.

Do you want me to open a new issue to track these tasks, or would you like suggestions on how to implement them?

25-26: Consider defensive checks for API response properties.

While the error handling catches network failures, missing or malformed properties in the JSON response (e.g., data.brand, data.ver, data.release) would display "undefined" in the UI.

Apply this diff to add defensive fallbacks:
-.then(data => {
-	document.querySelector('.installed-version').textContent = `${data.brand} ${data.ver} (${data.vid})`;
-	document.querySelector('.release-name').textContent = data.release;
+.then(data => {
+	const brand = data.brand || 'Unknown';
+	const ver = data.ver || 'Unknown';
+	const vid = data.vid || 'Unknown';
+	const release = data.release || 'Unknown';
+	document.querySelector('.installed-version').textContent = `${brand} ${ver} (${vid})`;
+	document.querySelector('.release-name').textContent = release;
pio-scripts/set_metadata.py (1)
69-77: Consider more specific exception handling.

The bare except Exception: clause at line 75 catches all exceptions, which can mask unexpected errors and make debugging difficult.

Consider catching specific exceptions or at least logging the error:
     except subprocess.CalledProcessError:
         # Git command failed (e.g., not a git repo, no remote, etc.)
         return None
-    except Exception:
+    except Exception as e:
         # Any other unexpected error
+        # Optionally log: print(f"Unexpected error in get_github_repo: {e}")
         return None
wled00/ota_update.h (1)

42-51: Correct the handleOTAData docs

Comment claims the function returns a bool/string pair, but the signature is void. Update the docblock to match the actual return type.
wled00/ota_update.cpp (5)
194-234: Make metadata windowing robust and bounded (avoid false negatives and dynamic growth)

Always accumulate a bounded window from start until METADATA_OFFSET+METADATA_SEARCH_RANGE before validating; then search that buffer once. This prevents edge cases where the first “crossing” chunk is large and doesn’t include bytes just before METADATA_OFFSET, and avoids repeated vector reallocs.

Example adjustment within this block:

Before validation: append incoming data while index+len <= METADATA_OFFSET+METADATA_SEARCH_RANGE (cap buffer to that size).

When buffer.size() >= METADATA_OFFSET+METADATA_SEARCH_RANGE, validate once using the buffer, then clear it.
This keeps memory bounded and improves reliability without increasing complexity.

236-243: Abort update immediately when validation never completes

Call abort to free resources ASAP when final chunk arrives without passing validation.

Apply this diff:
   if (isFinal && !context->releaseCheckPassed) {
     DEBUG_PRINTLN(F("OTA failed: Validation never completed"));
     // Don't write the last chunk to the updater: this will trip an error later
     context->errorMessage = F("Release check data never arrived?");
+    #if defined(ESP32)
+    Update.abort();
+    #endif
     return;
   }
145-146: Narrow the lambda capture

Use [request] instead of [=] to avoid capturing unrelated locals and reduce risk.

Apply this diff:
-    request->onDisconnect([=]() { endOTA(request); });  // ensures we restart on failure
+    request->onDisconnect([request]() { endOTA(request); });  // ensures we restart on failure
158-163: Docstring out of date with return type/semantics

Comment says “Returns pointer to error message…”, but function returns pair<bool, String>. Update for clarity.

Apply this diff:
-// Returns pointer to error message, or nullptr if OTA was successful.
+// Returns {done, message}. done=true when an HTTP response can be produced.
+// message is empty on success.
221-225: Error text should match UI wording

UI says “Ignore release name check”; message says “Ignore firmware validation”. Align for consistency.

Apply this diff:
-        context->errorMessage += F(" Enable 'Ignore firmware validation' to proceed anyway.");
+        context->errorMessage += F(" Enable 'Ignore release name check' to proceed anyway.");
wled00/wled_metadata.cpp (3)
21-24: Compile‑time length check can misbehave if RELEASE_NAME isn’t a literal

sizeof(WLED_RELEASE_NAME) only works for string literals. If it’s passed from build flags/macros that resolve to a non-literal, this may break. Consider a runtime assert/log or a static_assert on array length via a helper when literal is guaranteed.

64-72: Provide safe defaults for product/brand if integrator doesn’t define them

Prevent build breaks when WLED_PRODUCT_NAME/WLED_BRAND aren’t set.

Apply this diff:
+#ifndef WLED_PRODUCT_NAME
+#define WLED_PRODUCT_NAME "WLED"
+#endif
+#ifndef WLED_BRAND
+#define WLED_BRAND "WLED"
+#endif
120-128: Update the function doc to match the signature

The comment mentions binaryData/dataSize, but the function takes a descriptor. Fix the doc to avoid confusion.

Apply this diff:
- * @param binaryData Pointer to binary file data (not modified)
- * @param dataSize Size of binary data in bytes
- * @param errorMessage Buffer to store error message if validation fails 
+ * @param firmwareDescription Extracted firmware metadata to validate
+ * @param errorMessage Buffer to store error message if validation fails 

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7f1f986 and 5ca10f3.

📒 Files selected for processing (15)

pio-scripts/set_metadata.py (3 hunks)
pio-scripts/set_version.py (0 hunks)
platformio.ini (1 hunks)
tools/cdata.js (0 hunks)
wled00/data/update.htm (2 hunks)
wled00/dmx_input.cpp (1 hunks)
wled00/e131.cpp (1 hunks)
wled00/ota_update.cpp (1 hunks)
wled00/ota_update.h (1 hunks)
wled00/wled.cpp (2 hunks)
wled00/wled.h (1 hunks)
wled00/wled_metadata.cpp (1 hunks)
wled00/wled_metadata.h (1 hunks)
wled00/wled_server.cpp (2 hunks)
wled00/xml.cpp (0 hunks)

💤 Files with no reviewable changes (3)

pio-scripts/set_version.py
wled00/xml.cpp
tools/cdata.js

🧰 Additional context used

📓 Path-based instructions (5)

wled00/**/*.cpp