[MQTT-Binding / Homie]: No communication possible with Homie-Thing after restart #6828
Comments
|
I was able to reproduce the issue:
Logging with DEBUG for MQTT Binding did not show anything other than in first post. |
euphi
changed the title
|
I've also been having this one. https://community.openhab.org/t/influxdb-openhab2-stops-storing-data-after-reboot/63123 Replication is even easier than already listed. Add a homie v3 device. Same responses as rebooting openhab. Glad to see others are at least having this, i've been pulling my hair out trying to work out what i've configured wrong.. |
|
Next try:
Only relevant loginfo after startup seems to be OK
|
|
Have tested this with Snapshot Version: 2.5.0-SNAPSHOT (#1493) Issue happens there too. |
|
Also backtested all the way to the first test release that had the mqtt 2 bindings 2.4.0~M4-1 issue exists there too. |
|
@davidgraeff fyi |
|
Same problem here. I'm working to update the miflora-mqtt-daemon (https://github.com/pavax/miflora-mqtt-daemon/blob/master/miflora-mqtt-daemon.py) to follow the homie 3.0 specification. The device is discovered and there is no problem adding the thing. Every time a value is updated in the MQTT Broker the corresponding channel receives it and updates the linked Item. But as it was mentioned after a restart that update behaviour is broken. With regards to the relevant behaviour of my script:
My observations:
My Env (openhabian-pi)
|
|
I've noticed something similar with trigger channels (which toggle a switch item). Every time a Homie device goes offline and back online again I observe one more toggle per trigger. If I restart OpenHab I still get Homie device updates but the trigger channel stays silent. Only tested on 2.4 https://community.openhab.org/t/mqtt-homie-trigger-channel-silent-after-reboot/63489 |
|
I have been pulling my hair out with this one, having just moved a couple of devices to Homie & the new MQTT 2.4 binding, and have been resetting, rebuilding and reflashing everything for a couple of days to find out where I messed up - so glad I am not the only one! I have a,
Homie devices are (with 3x Homie 3.0), I am seeing the same behaviour as above, when freshly added all devices work until OpenHab is restarted at which point all my temperature reports go to NaN and stay that way despite normal MQTT traffic. I have noticed however, that the "control" channels still work, so toggling switches in OpenHab sends out the required MQTT command, and the Homie devices respond - but my temperatures and configuration "readback" that the Homie devices transmit periodically are ignored. Again, removing and re-adding the devices cause them to start working again until OpenHab is restarted. I have trimmed down my logs during startup (removed the Tradfri lights 250~ item status updates) , I am a very out of practice dev, so I am going to do a little hunting around - but my first thought is that this might be related to the startup order, as I can see a couple of warnings popping up above before the broker comes online - although that could be completely normal for all i know ! |
|
@spafrost I can relate to your frustration - it took me also a couple of days and multiple re-installs to realise that the problem can not only be on my side but that there has to be some bug in openhab/eclipse-smarthome. Since we've established that removing and adding the thing again is one way to get around that problem it might be useful if someone else can confirm the second workaround that I've encountered.
#!/bin/sh
echo "cleaning " $1 " :: usage: cleanmqtt <host>"
mosquitto_sub -h $1 -t "#" -v | while read line _; do mosquitto_pub -h $1 -t "$line" -r -n; done
|
|
Could you guys please also try out to disable a Thing and re-enable it again? (Possible via the console / rest / Paper UI) I have no straight explanation for this behaviour to be honest. |
|
@davidgraeff I disabled and re-enabled the Thing using Paper-UI. ... then I published a new value: ... the values is still not changed for the MQTT-Home Thing (it is changed for the MQTT-Generic-Thing but I disabled it for this test-scenario) Log-Viewer output (disabling the generic-mqtt-thing, disabling the homie-thing, re-enabling the homie thing) |
|
Alright, thanks. I think I know where it fails then. |
|
@davidgraeff dunno if it helps you but what is weird that I never get to see the log-entry: HomieThingHandler.java @Override
public void accept(@Nullable List<Object> t) {
if (!device.isInitialized()) {
return;
}
List<Channel> channels = device.nodes().stream().flatMap(n -> n.properties.stream())
.map(prop -> prop.getChannel()).collect(Collectors.toList());
updateThing(editThing().withChannels(channels).build());
updateProperty(MqttBindingConstants.HOMIE_PROPERTY_VERSION, device.attributes.homie);
final MqttBrokerConnection connection = this.connection;
if (connection != null) {
device.startChannels(connection, scheduler, attributeReceiveTimeout, this).thenRun(() -> {
logger.trace("Homie device {} fully attached", device.attributes.name);
});
}
} |
|
Exactly. The culprit is in public void initialize(String baseTopic, String deviceID, List<Channel> channels) {
this.topic = baseTopic + "/" + deviceID;
this.deviceID = deviceID;
nodes.clear();
for (Channel channel : channels) {
final String nodeID = channel.getUID().getGroupId();
final String propertyID = channel.getUID().getIdWithoutGroup();
if (nodeID == null) {
continue;
}
Node node = nodes.get(nodeID);
if (node == null) {
node = createNode(nodeID);
node.nodeRestoredFromConfig();
nodes.put(nodeID, node);
}
// Restores the properties attribute object via the channels configuration.
Property property = node.createProperty(propertyID,
channel.getConfiguration().as(PropertyAttributes.class));
property.createChannelFromAttribute();
node.properties.put(propertyID, property);
}
}openHAB knows about the Things/channels from its loaded configuration. This method recreates the internal Node/Property structure. But the re-subscribing to the property values is not performed apparently. |
|
Is there a workaround available? |
|
hey! I have the same issue with my auto discovered homie compliant mqtt's. Any way to fix this or at least to make a workaround automatically executed at startup? |
|
There is unfortunately no workaround, at the same time I cannot work on mqtt for the next weeks. Busy with the Paper UI NG design study in my free time already. |
|
Hi David, I understand you're already sacrificing your free time to another project and I appreciate the work you have done so far. It's just a pity you promoted to switch to homie on Smart Home Day while it's not yet working. The auto-discovery was amazing, so I was a fan of this solution as soon as I discovered it, but we end up with a broken system at this point. I will investigate if I can fix this issue, but it will cost me a lot more time because I'm missing a log of background information. |
I worked perfectly. But some people contributed already and at some point it broke. My fault was that I didn't contribute a test case to catch that error in the first place, I think. It's tough times at the moment to contribute, because of all those package movements. |
|
Ok, David, sorry for blaming. I will see if I can find a reason why it's broken. |
|
@ddecockbe I've been playing around with that issue as well.... It took me quite a while to setup a openhab eclipse IDE in order to debug that issue. I narrowed the problem down the the following lines that I changed (see here: https://github.com/pavax/openhab2-addons/pull/1/files). PS: I'm aware that the openhab2-addons is a different repository (I haven't figured yet out how to clone the smarthome repository into the openhab-eclipse IDE and deploy the correct plugins) also I haven't yet run the unit tests |
|
Hi @pavax, thanks! I will try that and will keep you updated! |
|
@pavax That's a perfect fix. Please open a pull request to openhab2-addons (you are on the right repository, this one here will be soon discontinued). |
|
@davidgraeff Thx - I'll open a PR asap (it took me a bit longer since I stumbled across another bug+fix which I pushed as well to my branch) - I'd like to create some unit tests as well and ideally get the confirmation that these fixes work from someone else that patched the bundles. @ddecockbe If you find the time I'd appreciate if you could either checkout my branch and build the modules: org.openhab.binding.mqtt and org.openhab.binding.mqtt.generic or if you want (and trust me :) you can download the patched jars here: https://uploadfiles.io/f0g2v, https://uploadfiles.io/b32je How did I install the patched bundles on my openhabian-pi?
|
|
@pavax I took the risk to trust you ;). I installed the jars and the are working fine here. I restarted openhab and all my homie items came back as they should. Thanks a lot! I can now further 'homie'-fy my home-brew sensors. |
|
I also tested the jars. The one device that was configured as Homie Thing seems to reconnect the channels correctly after restart. However, some (not all) legacy MQTT devices DID NOT! So Items linked to their channels were not updated. After opening on channel of an affected Thing and Saving it (no change), all channels of this thing now work ok again. I don't have logs of startup, but some logs of the moment a channel was reconfigured: |
|
Found some logs/exceptions just after startup: The channels seem to be updated after this, but a little bit later the following happens and channels are no longer updated. The devices HomieRollershutter and HomieBME280 are offline (in another network), so this may be ok to stop them? (But why half an hour after restart?) I don't know which devices are "null"? |
You mean the Generic Thing?
Uih. That looks bad. I already noticed that I do not sanitize mqtt topics before using them as openHAB channels. So dashes, brackets and all kind of other non latin characters will cause problems atm. And yes the code is absolutely lost if no $name has been send in time, which might explain the
That depends on the heartbeat and last will publishing setting of your broker. |
Yes. These devices are all Homie devices, but some to version 2 of the convention, some to version 3.
All the $names have been stored as retained message, so they should be available. Maybe its a timing issue? My openhab instance is running on an old arm h7 dual-cpu. Startup of Openhab takes several minutes. |
|
Regarding the "no longer updated" channels. Most devices were affected, including all "homie convention 3.0.1" devices and some 2.0.0 devices: |
I know, it's time for a different input file format. XText based files (.thing/.item/.rule) are slow as hell. I don't know if you were part of that heated discussion in the OH forum.
I have a ridiculously short wait time only (200ms), I was surprised when I saw that recently. So that could be the problem. Really sad that we have no "done" signal in the MQTT protocol for a topic tree transmission. Oh and very important: Increase the limit of retained messages in mosquitto. A lot of users had problems. The default seem to be very low. |
strange.... I didn't change anything that should interfere with the Generic-Thing Handler :/ However since I used the master branch it could be that since the release of OH 2.4 other changes have been merged that could cause side-effects... Could it be, that the Generic-Thing Channel is not being updated when at the same time there is also a Homie-Thing Channel defined for the same Topic? With regards to the
|
My devices where all configured as "Generic MQTT Thing" only. The corresponding Homie Thing has been deleted from Inbox. |
|
I tried the latest jars from @pavax and the subsribing after restarting openhab works now. I would really apreciate if anybody could look into this. |
|
FYI: I do not plan on fixing any MQTT issues for the next weeks to come until the Eclipse Smarthome merge back has finished completely and openhab2-addons has been migrated to the pure maven build system. |
|
Totally understandable. I tried setting up the openHAB IDE, which works, but adding the mqqt addons to the launch runtime just gives errors and they dont show up on localhost. The total project is a mess and there is no tutorial how to properly set it up (seems like additional committers have to spend a week to get started, or just are not welcome for openHAB), but that has already been the case before the ESH project was closed. But i would suggest to remove the homie support from release notes to 2.4.0 and from the blog post announcement, because it is just broken and not usable. |
I can understand you as well, but if we really want to act like this then we should shutdown the entire homepage until the merge is done, because not much is working at all right now. But trust me there are people working on fixing and adapting the build system. And it is for the better in the end. |
|
Offtopic: @euphi , looking at your topic start your homie mqtt device project 'lab_thermo' holds code which could help me understand Arduino and the Homie library better and could boost my learning curve. Are you willing to share this code/project with me/public? |
|
I just want to give a thumbs up for the effort put into resolving this. I was in the process of analyzing the code to fix it myself when you posted the diff, so you saved me some time 👍 Works perfectly on my setup, with several Homie 3 devices. |
|
@BasvanH: You can have a look at https://github.com/euphi/ESP_HomieBME280Node or https://github.com/euphi/HomieNodeCollection/tree/develop-v3 |
|
Is this fix already in ubuntu packages ? I am having same/similar problems, so guess not. When it is planned to have release with this fix ? Also, I have noticed that added homie device stops refreshing values after some time (cca 1 day). Is that also caused by same bug ? |
|
This is the wrong repository. Eclipse smarthome is not used for openHAB anymore. There is a fix in the openHAB repository, yes. |
|
Ah, I am running 2.4.0 and just realized openhab updates are not so often. Even a 2.5.0.M1 milestone version is almost two months old and does not contain a fix. Only way to get fixed version is running snapshot version, hope fix will be released until I move all my devices to homie 3.0. |
|
BTW: The #develop-v3 branch of Homie-ESP8266 implementation now supports ESP32, too. It also seems quite stable, so please test and write bug reports in case of problems. |
@pavax would you mind reposting the compiled jars? Yours seem to be deleted.... |
|
|
|
Is there a release date? |
|
I've just started having this exact problem. Has it been solved? Is there a workaround? I'm running all the latest stable releases. Autodiscovery of Homie devices works and If I manually send them commands over console they respond and subscribing in console works too. It's really frustrating as just a couple days ago all my homie v3 devices were working perfectly. |
updating to the latest nightly helped me... |
My Homie-V3 test device was auto-discovered correctly and working fine (I manually linked the newly created items).
However, after restart, the device is "Online", but communication is ignored by Openhab and any attempt to change state of settable properties results in error:
Restart of the Homie device
No messages are sent to the MQTT broker by openhab, received messages are ignored.
Please note that the device is treated as online, as long as no command is sent to it.
(If it disconnects or is "lost" this is also correctly detected by Openhab)
Also, the channels are no longer configurable after Openhab was restarted once after the Thing and its channels were auto-discovered. Just after auto-discovery it was possible to edit the channel configuration:
Screenshot of PaperUI ("Edit-Pen" missing for channels (same behaviour if device is still seen as "online"):
Screenshot of PaperUI Control (no dropdown with valid Enum values for Loglevel):
PS: A manually added MQTT thing (Homie-2.0, so not auto-discoverable) works fine.
PPS: See also #6823 - this issue happend before the first retart of Openhab after the Homie-Thing was added.
PPPS: For completness, log of MQTT communication:
The text was updated successfully, but these errors were encountered: