700 series controllers becoming unresponsive / jammed

[rohrsh]

Hello everybody. I'm struggling to get my network working reliably and would greatly appreciate any suggestions.

I started with a small Zwave-JS-UI network to test a few devices, and that was operating just fine for the last few months. Last week I then migrated the rest of my ~50 devices over from Fibaro Home Center 2.

Messages are slow to execute (if at all) and Home Assistant regularly reports the controller is Jammed

• Diagnostics on many nodes show that my controller can ping a node, but all return pings fail (See (b). Similar to Failed pings one way between nodes, not the other; help appreciated #5179
• Logs show some devices repeatedly updating the same value to the controller (See (a), which led me to this Super slow network updating the same value many times #6284

Situation:

• Zwave-JS-UI 9.14.3 docker on Raspberry Pi 3b. It was originally in a NUC docker, but I moved to the Pi temporarily so I could roam around to do exclude/includes
• Home Assistant websocket - but I'm running all my tests in Zwave-JS-UI
• Silicon Labs 700 stick. Firmware was originally 7.19.2 but I've since upgraded to 7.22.3 without any improvement. Could I just be suffering from the 700 series firmware bug?
• 2m USB extension cable
• I have rediscovered neighbours and re-interviewed many times
• Most of the devices are within 20m of the controller. The mesh seems very dense - some nodes have ~20 neighbours. Electrician often placed 6x pucks jammed into one plastic box. I see lots of red 9.6kb links (See (c)
• Devices are mostly Fibaro Dimmers and Relays, which are ~5 years old, I think 300 series protocol
• Value update reports set to reasonable thresholds: e.g. my dimmers only report data on 10% power change, or hourly
• I have read the troubleshooting docs here https://zwave-js.github.io/node-zwave-js/#/troubleshooting/index
• My devices did work last week on the Fibaro Home Center 2 controller

Logs attached (d). Towards the end of the log you'll see me fiddling with Node 20 (ok) and then it jams when adjusting Node 31.

Thank you for any help or suggestions.

(a)

(b)

(c)

(d)
zwavejs_2024-06-27.log

[rohrsh]

Some of the routing choices seem pretty odd: e.g. this node 45 is in a rather central location and has dozens of neighbours to choose from, I'm sure there's a better path than what has been chosen.

I wonder if the controller is restarting while testing routes, which delivers bad data, which leads to odd routing decisions.

[rohrsh]

[rohrsh]

Options I'm considering

• Setting priority/return routes
• Get a 500 series controller
• Split into two Z-wave networks

[Idaho947]

It a known bug on the firmware. Just have to wait. But silicon lab is on it for one year....

[AlCalzone]

See #6874, #6512, #6911 for the jammed/unresponsive issue.

FWIW, you're on firmware 7.21.3. From what I've seen, 7.21 is in a pretty bad state right now, even worse than 7.19.
7.22.x is limited to 800 series controllers, but it is supposed to include a few fixes for the lockup issues.

I'll take a look at the logs tomorrow to see if anything else sticks out, since you seem to have followed the typical troubleshooting steps already.
My personal experience is to avoid 700 series controllers though. Either 500 or 800 series IMO, 800 has much better range, but you're at the mercy of Silabs to fix the issues.

[webmogul1]

I am using the ZOOZ 700 stick with firmware v7.18.3 for quite some time without apparent issues.

[webmogul1]

I wonder if the controller is restarting while testing routes, which delivers bad data, which leads to odd routing decisions.

Without any expertise on my part, I would say that's the problem. At least, the same thing happened to me when I switched to an 800 stick and firmware 7.21.3

[AlCalzone]

Given how long we've seen complaints about bad routes (and the fact that I've seen the code responsible for choosing routes), I would say the two are not related. The routing algorithm is just terrible tbh.

[rohrsh]

The routing algorithm is just terrible tbh.

Any best practice guides for messing with priority/return routes? I saw this DrZwave guide. which seems to suggest only setting a priority route for nodes that you are certain can reach your controller.

[rohrsh]

My personal experience is to avoid 700 series controllers though. Either 500 or 800 series IMO, 800 has much better range, but you're at the mercy of Silabs to fix the issues.

Slim pickings here in Australia!

The 800 series Zooz ZST39 doesn't have an AU variant for sale. Could I buy the US model and simply set to AU region in ZwaveJS? Or will I need special firmware? AU-specific SAW filter? I don't need the LR functionality. Some Redditors here mentioned under-the-table AU firmware via thesmartesthouse.com

I dd try ordering a local Aeotec Gen5+, but the vendor helpfully delivered an 'upgraded' Z-Stick 700 instead...

Is the 500 series Z-Wave RaZberry Card V2 usable? I can see AU stock of that

[zackbcom]

You could check out this Silicon Labs Z-Wave 800 Series Development Kit. I have two devices. One flashed to be a controller and one flashed to be a zniffer. You can flash different country firmwares. Look into that. Also you can upgrade to any new firmware without having to wait for a vendor to roll their own out.

[AlCalzone]

Could I buy the US model and simply set to AU region in ZwaveJS? Or will I need special firmware? AU-specific SAW filter?

AFAIK (but I might be wrong), the hardware is the same for all regions, they just come with the firmware pre-set to the correct region. At least I am regularly switching my US ZST39 to EU frequencies.

Is the 500 series Z-Wave RaZberry Card V2 usable? I can see AU stock of that

It works, but not sure if the firmware supports SmartStart etc. Upgrades require using Z-Wave.me software.

[Botched1]

• Silicon Labs 700 stick

I also use a UZB-7 from SiLabs on my ~100 device network. 7.18.3 is the only firmware that I've tried that works pretty much 100% correctly. Occasional jammed status report, but it is VERY rare for me (<1/week).

[rohrsh]

Ordered a new Zooz 700 and it arrived with 7.19.2 firmware
Tried to downgrade my Silabs 700 via Windows Z-wave PC Controller, but the bootloader said no (Error 0x18)

So I've started a second network on my Zooz stick. Will migrate a few devices over each day and see if there happens to be some magic number of devices, or a specific device, that triggers the Jamming.

[rohrsh]

So I've been migrating my network over to a new 7.19.2 Zooz 700 stick, a dozen devices each day. Got to 24 nodes and looked ok - I was testing with the ping diagnostics and individual node control.

However, things turned bad when I tried to control multiple (3 - Edit: 4) lights at once (via Home Assistant.) Some lights would take 10+ seconds to implement the command.

Maybe I'm hitting this? #5471 ... although I run with no security.

Logs appear to show way too much traffic: here's a sample of me turning off and then on 3 lights simultaneously. They are Fibaro dimmers which report power changes.
zwavejs_2024-07-13--turnonoff.log

Here's my full day of logs for completeness also. The driver appears to crash at one point in my testing.
zwavejs_2024-07-13.log

Is this still the same 700 series firmware bug and I just have to wait?

[rohrsh]

Incidentally, two of the lights in this group of 4 are not currently directly connected to the controller. Could exacerbate the problem.

e.g. Living Alcove, Living Dining Downlights [top right of map]

[zwave-js-assistant]

This issue has not seen any recent activity and was marked as "stale 💤".
Closing for housekeeping purposes... 🧹

Feel free to reopen if the issue persists.

[rohrsh]

Please don't close until we see new firmware to fix the 700 series. The fixes out this week are only for 800 series for now.