What to do with migrated devices that have been set up strangely?

[AlCalzone]

Since the influx of new users I'm getting more and more reports of really strange device behavior. Some examples:

• Listening nodes that don't respond to RequestNodeInfo and send their sensor reports as broadcasts ([question] Migrated from OpenZwave - Difficulties with almost all devices #1689 (comment))
• Nodes that start spamming in response to single GET requests, first singlecast, then broadcast ([bug] Issue with multiple reports from specific devices #1677), see log:

Log

15:00:33.959 DRIVER » [Node 113] [REQ] [SendData]
                      │ transmit options: 0x25
                      │ callback id:      216
                      └─[ConfigurationCCGet]
                          parameter #: 7
15:00:39.705 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:39.707 SERIAL » [ACK]                                                                   (0x06)
15:00:39.708 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:41.002 SERIAL « 0x010e00a800017105700607010000a2fe                                  (16 bytes)
15:00:41.003 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:41.004 SERIAL » [ACK]                                                                   (0x06)
15:00:41.006 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:42.488 SERIAL « 0x010e00a800017105700607010000b9e5                                  (16 bytes)
15:00:42.489 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:42.499 SERIAL » [ACK]                                                                   (0x06)
15:00:42.500 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:42.594 SERIAL « 0x010e00a800017105700607010000b9e5                                  (16 bytes)
15:00:42.596 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:42.600 SERIAL » [ACK]                                                                   (0x06)
15:00:42.602 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:42.770 SERIAL « 0x010e00a800017105700607010000b9e5                                  (16 bytes)
15:00:42.771 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:42.776 SERIAL » [ACK]                                                                   (0x06)
15:00:42.778 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:43.318 SERIAL « 0x010e00a800017105700607010000a5f9                                  (16 bytes)
15:00:43.321 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:43.323 SERIAL » [ACK]                                                                   (0x06)
15:00:43.324 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:43.378 SERIAL « 0x010e00a800017105700607010000a5f9                                  (16 bytes)
15:00:43.379 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:43.381 SERIAL » [ACK]                                                                   (0x06)
15:00:43.383 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:43.470 SERIAL « 0x010e00a800017105700607010000a7fb                                  (16 bytes)
15:00:43.471 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:43.474 SERIAL » [ACK]                                                                   (0x06)
15:00:43.476 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:43.498 SERIAL « 0x010e00a800017105700607010000a5f9                                  (16 bytes)
15:00:43.498 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:43.501 SERIAL » [ACK]                                                                   (0x06)
15:00:43.502 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:43.525 SERIAL « 0x010e00a800017105700607010000a7fb                                  (16 bytes)
15:00:43.526 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:43.533 SERIAL » [ACK]                                                                   (0x06)
15:00:43.535 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:43.683 SERIAL « 0x010e00a800017105700607010000a3ff                                  (16 bytes)
15:00:43.685 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:43.688 SERIAL » [ACK]                                                                   (0x06)
15:00:43.689 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:44.041 SERIAL « 0x010e00a814017105700607010000d991                                  (16 bytes)
15:00:44.042 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:44.049 SERIAL » [ACK]                                                                   (0x06)
15:00:44.050 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:44.174 SERIAL « 0x010e00a814017105700607010000e8a0                                  (16 bytes)
15:00:44.176 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:44.179 SERIAL » [ACK]                                                                   (0x06)
15:00:44.182 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:44.191 SERIAL « 0x010e00a814017105700607010000da92                                  (16 bytes)
15:00:44.192 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:44.193 SERIAL » [ACK]                                                                   (0x06)
15:00:44.194 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:44.313 SERIAL « 0x010e00a814017105700607010000d49c                                  (16 bytes)
15:00:44.314 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:44.317 SERIAL » [ACK]                                                                   (0x06)
15:00:44.320 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:44.325 SERIAL « 0x010e00a814017105700607010000e9a1                                  (16 bytes)
15:00:44.326 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:44.327 SERIAL » [ACK]                                                                   (0x06)
15:00:44.328 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:44.611 SERIAL « 0x010e00a814017105700607010000e5ad                                  (16 bytes)
15:00:44.613 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:44.616 SERIAL » [ACK]                                                                   (0x06)
15:00:44.617 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:44.646 SERIAL « 0x010e00a814017105700607010000e9a1                                  (16 bytes)
15:00:44.647 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:44.650 SERIAL » [ACK]                                                                   (0x06)
15:00:44.651 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:44.667 SERIAL « 0x010e00a814017105700607010000d49c                                  (16 bytes)
15:00:44.668 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:44.670 SERIAL » [ACK]                                                                   (0x06)
15:00:44.671 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:44.684 SERIAL « 0x010e00a814017105700607010000d59d                                  (16 bytes)
15:00:44.685 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:44.687 SERIAL » [ACK]                                                                   (0x06)
15:00:44.688 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:00:44.772 SERIAL « 0x010e00a814017105700607010000e4ac                                  (16 bytes)
15:00:44.773 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:00:44.776 SERIAL » [ACK]                                                                   (0x06)
15:00:44.778 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:01.705 SERIAL « 0x011200a8000158097105000000ff07080000ad3c                          (20 bytes)
15:01:01.706 SERIAL » [ACK]                                                                   (0x06)
15:01:01.707 DRIVER « [Node 088] [REQ] [BridgeApplicationCommand]
                      └─[NotificationCCReport]
                          notification type:   Home Security
                          notification status: 255
                          notification state:  Motion detection
15:01:01.708 CNTRLR   [Node 088] [~] [Notification] Home Security[Motion sensor status] [Endpoint 0]
                      : 0 => 8
15:01:05.820 SERIAL « 0x010e00a800017105700607010000b9e5                                  (16 bytes)
15:01:05.821 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:05.823 SERIAL » [ACK]                                                                   (0x06)
15:01:05.824 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:07.049 SERIAL « 0x010e00a800017105700607010000bae6                                  (16 bytes)
15:01:07.050 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:07.054 SERIAL » [ACK]                                                                   (0x06)
15:01:07.055 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:08.898 SERIAL « 0x010e00a800017105700607010000a4f8                                  (16 bytes)
15:01:08.899 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:08.903 SERIAL » [ACK]                                                                   (0x06)
15:01:08.906 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.122 SERIAL « 0x010e00a814017105700607010000d39b                                  (16 bytes)
15:01:09.123 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.125 SERIAL » [ACK]                                                                   (0x06)
15:01:09.126 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.214 SERIAL « 0x010e00a814017105700607010000da92                                  (16 bytes)
15:01:09.215 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.217 SERIAL » [ACK]                                                                   (0x06)
15:01:09.217 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.225 SERIAL « 0x010e00a814017105700607010000e8a0                                  (16 bytes)
15:01:09.226 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.227 SERIAL » [ACK]                                                                   (0x06)
15:01:09.228 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.430 SERIAL « 0x010e00a814017105700607010000e9a1                                  (16 bytes)
15:01:09.431 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.433 SERIAL » [ACK]                                                                   (0x06)
15:01:09.433 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.460 SERIAL « 0x010e00a814017105700607010000d49c                                  (16 bytes)
15:01:09.461 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.463 SERIAL » [ACK]                                                                   (0x06)
15:01:09.464 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.475 SERIAL « 0x010e00a814017105700607010000d59d                                  (16 bytes)
15:01:09.476 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.477 SERIAL » [ACK]                                                                   (0x06)
15:01:09.478 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.515 SERIAL « 0x010e00a814017105700607010000e5ad                                  (16 bytes)
15:01:09.517 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.519 SERIAL » [ACK]                                                                   (0x06)
15:01:09.522 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.536 SERIAL « 0x010e00a814017105700607010000e9a1                                  (16 bytes)
15:01:09.537 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.538 SERIAL » [ACK]                                                                   (0x06)
15:01:09.539 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.631 SERIAL « 0x010e00a814017105700607010000d49c                                  (16 bytes)
15:01:09.632 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.634 SERIAL » [ACK]                                                                   (0x06)
15:01:09.636 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.659 SERIAL « 0x010e00a814017105700607010000dc94                                  (16 bytes)
15:01:09.660 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.662 SERIAL » [ACK]                                                                   (0x06)
15:01:09.663 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0
15:01:09.751 SERIAL « 0x010e00a814017105700607010000e4ac                                  (16 bytes)
15:01:09.753 CNTRLR   [Node 113] [~] [Configuration] 7: 0 => 0
15:01:09.756 SERIAL » [ACK]                                                                   (0x06)
15:01:09.757 DRIVER « [Node 113] [REQ] [BridgeApplicationCommand]
                      │ type:        broadcast
                      │ target node: 255
                      └─[ConfigurationCCReport]
                          parameter #: 7
                          value size:  1
                          value:       0

It also seems that excluding, resetting and re-including the devices with zwave-js often alleviates the problems. So I'm getting the feeling that properly bootstrapping the nodes (return route, wake up destination, Lifeline) was not always done before.

Which leads to the question: Which checks and corrections can automatically done when zwave-js gets control over a pre-existing network to avoid such weird behavior?
@hanskroner

[hanskroner]

Since the influx of new users I'm getting more and more reports of really strange device behavior. Some examples:

• Listening nodes that don't respond to RequestNodeInfo and send their sensor reports as broadcasts (#1689 (comment))
• Nodes that start spamming in response to single GET requests, first singlecast, then broadcast ([bug] Issue with multiple reports from specific devices #1677), see log:

Log
It also seems that excluding, resetting and re-including the devices with zwave-js often alleviates the problems. So I'm getting the feeling that properly bootstrapping the nodes (return route, wake up destination, Lifeline) was not always done before.

The Z-Wave protocol, and therefore the network, doesn't really care about what's on the host side of the serial interface. My bet would be that the spamming and broadcasting you mention happens when running under OZW as well, and that host considered it "normal." These things are becoming apparent now that there's a host that's trying to adhere to Certification requirements.

Which leads to the question: Which checks and corrections can automatically done when zwave-js gets control over a pre-existing network to avoid such weird behavior?

If you'd like the library to try to "fix" the situation with the network, the best procedure to follow would be to run the same process you do for a device after including it - that is, all the "interview" and setup process you do for the nodes. Start by asking every node in the network to look for neighbors - battery-powered devices will need to be woken up manually before you can send the command to them. Once that is done and the controller has an updated picture of the network, continue by (re-)setting up SUC return routes, return routes to nodes in slave's Associations table - again, battery-powered nodes will need to be manually woken up before commands are sent to them. Make sure to (re-)set the Lifeline for Z-Wave Plus devices while you're at it. For battery-powered devices, also (re-)setup their wake-up destination and interval to make sure they have one, and that it's not going to obliterate the device's battery by reporting way too often.

The only part you'll not be able to correct here would be Security - as that needs to happen immediately after inclusion. Users will need to exclude and re-include devices if they want S0 (or S2, when that gets supported).