Topic underscore are replaced by spaces

[madjack84]

Hey all,

i just found out that some of my topics changed since I use MQTT client instead of MQTT Broker/client.

Example:
in the objects following date with 2 underscores
mihome.0.devices.sensor_ht_158d000241bdaa.temperature

is offered in MQQT Client tool/option as
mihome/0/devices/sensor ht 158d000241bdaa/temperature
without underscores (replaced by spaces).

Maybe there is a wrong conversion :(
You are able to manually change it in the options, but maybe this bug could be fixed

I use following config on the raspian image
iobroker admin 3.6.0
npm 6.4.1
node v8.12.0
nodejs v8.12.0

just had a look into the code... and it seems to be a feature.... why??
//replace dots with slashes and underscores with spaces
topic = topic.replace(/./g, '/').replace(/_/g, ' ');

[IoTAntimon]

Would it be possible to add an option to switch on/off the replacement of underscores? I'd prefer underscores in MQTT topics, too...

[chrispi68]

Same problem here

[Pmant]

This was implemented due to ioBroker not allowing underscores in state ids.
However, as a matter of fact, ioBroker creates an example state beginning with 0_userdata.

We need some clarification on how to modify and validate ioBroker state ids.

[Apollon77]

Underscores are fully allowed characters for state ids (just not for adapternames) ... so I would not see any issue here? IF a state contains an "Invalid charccter" iobrokr even replace these by an underscore :-) SO have fun using underscores, but this might be a breaking change

[Pmant]

According to this underscores are not allowed, this documentation might be outdated tho.
MQTT allows all UTF-8 characters as topic, with '/' being the level seperator. I will look into this in try to come up with a more intuitive solution.

As you said this change might be breaking, this change should happen in a major release.

[Apollon77]

WTF, I will fix it. Infact the "reality" is a big different in the meantime
: https://github.com/ioBroker/ioBroker.js-controller/blob/master/packages/common/lib/common/tools.js#L44 (note it is a Negatove list!!)

As you said this change might be breaking, this change should happen in a major release.

or make a settings for users to switch with a note that it then is breaking

PS: Also / is basically allowed

[Pmant]

Replacing / with . and vise versa is mendatory since both are mostly equivalent (level seperator).
I would like to enforce this change since conversion is bugged right now anyway.
Since every UTF-8 string is a valid MQTT topic while ioBroker ids have some limitations, this can easily lead to collitions when two MQTT topics are converted to the same ioBroker id.
E.g.:

a/@b/c -> a._b.c
a/^b/c -> a._b.c

These cases have to be handled somehow.

[Apollon77]

I would like to enforce this change ...

Then do a major upgrade ... think best idea. If you like you could add a "message" to be shown in admin as soon as someone does such an upgrade??
https://github.com/ioBroker/ioBroker.knx/blob/master/io-package.json#L314-L371 ... but then also add the admin "globalDependencies" to have it shown to all users

[Pmant]

I will do a minor release before merging this so we have a working non breaking version.
Release script hung up on git push somehow, but I think I managed to finish it manually.

[DrJohnsn]

What is the current status ?
I currently have the same problem.
The HmIP adapter uses underscores for the states.

The MQTT state contains spaces!?

Thank you

[DoImant]

I accidentally posted this "bug" in another repo (ioBroker/ioBroker.mqtt#362).
You can work around it by manually re-entering the lost underscores of the automatically created topic in the mqtt-client settings. But it would be nicer if the automatism works correctly right away.

[mcm1957]

According to this underscores are not allowed, this documentation might be outdated tho. MQTT allows all UTF-8 characters as topic, with '/' being the level seperator. I will look into this in try to come up with a more intuitive solution.

As you said this change might be breaking, this change should happen in a major release.

NO, the mentioned regex starts with an ^, so it listes the allowed characters. An underscore is fully valid as part of an id.
SPACES should be avoided as part of an Id and are typically repelaced by an _