Frequent disconnects

[srudin]

Hi all

I'm completely lost and frustrated. I have searched in Mosquitto, Home Assistant and Ring-Mqtt discussions for a while and seen some similar issues, but no solutions. Please let me describe my problem here hoping that someone can get me on the right track - apologies if I choose the wrong place to start with.

I'm trying to get my Ring alarm system (base station, sensors, sirens) connected to my Home Assistant installation. I am using a Docker container installation of HA so no out-of-the-box addon support - what I did therefore is I installed another Docker container with Ring-Mqtt and one with Eclipse-Mosquitto (all latest version as of 2024-6-6).

For Mosquitto I have added 2 users to passwd_file ("home_assistant" and "ring_mqtt") and the following mosquitto.conf:

password_file /mosquitto/config/passwd_file
allow_anonymous false
listener 1883
socket_domain ipv4
use_username_as_clientid true
persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log

For Ring-Mqtt I added an oauth token and the following config.json:

{
"mqtt_url": "mqtt://ring_mqtt:mypwd@READ_BELOW:1883",
"mqtt_options": "",
"livestream_user": "",
"livestream_pass": "",
"disarm_code": "",
"enable_cameras": false,
"enable_modes": false,
"enable_panic": false,
"hass_topic": "homeassistant/status",
"ring_topic": "ring",
"location_ids": []
}

Then I configured a default MQTT integration in HA using the home_assistant user.

Now I have done 2 approaches:

First I have configured all containers to use the default bridge network and mapped port 1883 of the host machine to the Mosquitto container. In this case I used the ip address of the host in the json above ("READ_BELOW") as well as in the HA MQTT integration. After doing this I was able to see all my devices in HA. I was even able to change the alarm from Inactive to Away mode in the HA dashboard. However, events seem to transmit a bit slow and I got frequent disconnects for both users. This was shown in the Mosquitto log:

1717654094: New connection from 172.17.0.1:59640 on port 1883.
1717654094: New client connected from 172.17.0.1:59640 as home_assistant (p2, c1, k60, u'home_assistant').
1717654165: New connection from 172.17.0.1:59642 on port 1883.
1717654165: New client connected from 172.17.0.1:59642 as ring_mqtt (p2, c1, k60, u'ring_mqtt').
1717655117: Client ring_mqtt has exceeded timeout, disconnecting.
1717655120: New connection from 172.17.0.1:59644 on port 1883.
1717655120: New client connected from 172.17.0.1:59644 as ring_mqtt (p2, c1, k60, u'ring_mqtt').
1717655687: Client home_assistant has exceeded timeout, disconnecting.
1717655697: New connection from 172.17.0.1:59646 on port 1883.
1717655697: New client connected from 172.17.0.1:59646 as home_assistant (p2, c1, k60, u'home_assistant').
1717655735: Client ring_mqtt has exceeded timeout, disconnecting.
1717655753: New connection from 172.17.0.1:59648 on port 1883.
1717655753: New client connected from 172.17.0.1:59648 as ring_mqtt (p2, c1, k60, u'ring_mqtt').
1717655789: Client home_assistant has exceeded timeout, disconnecting.
1717655799: New connection from 172.17.0.1:59650 on port 1883.
1717655799: New client connected from 172.17.0.1:59650 as home_assistant (p2, c1, k60, u'home_assistant').
1717655892: Saving in-memory database to /mosquitto/data//mosquitto.db.
1717656275: Client ring_mqtt has exceeded timeout, disconnecting.
1717656277: New connection from 172.17.0.1:59652 on port 1883.
1717656277: New client connected from 172.17.0.1:59652 as ring_mqtt (p2, c1, k60, u'ring_mqtt').
1717656875: Client ring_mqtt has exceeded timeout, disconnecting.
1717656917: Client home_assistant has exceeded timeout, disconnecting.
1717656928: New connection from 172.17.0.1:59654 on port 1883.
1717656928: New client connected from 172.17.0.1:59654 as home_assistant (p2, c1, k60, u'home_assistant').
1717657019: Client home_assistant has exceeded timeout, disconnecting.
1717657072: New connection from 172.17.0.1:59664 on port 1883.
1717657072: New client connected from 172.17.0.1:59664 as home_assistant (p2, c1, k60, u'home_assistant').
1717657091: New connection from 172.17.0.1:59662 on port 1883.
1717657091: Client home_assistant already connected, closing old connection.
1717657091: New client connected from 172.17.0.1:59662 as home_assistant (p2, c1, k60, u'home_assistant').
1717657091: Client home_assistant closed its connection.
1717657103: New connection from 172.17.0.1:59666 on port 1883.
1717657103: New client connected from 172.17.0.1:59666 as ring_mqtt (p2, c1, k60, u'ring_mqtt').
1717657103: New connection from 172.17.0.1:59658 on port 1883.
1717657103: New client connected from 172.17.0.1:59658 as home_assistant (p2, c1, k60, u'home_assistant').
1717657103: Client home_assistant closed its connection.
1717657104: New connection from 172.17.0.1:59668 on port 1883.
1717657104: New client connected from 172.17.0.1:59668 as home_assistant (p2, c1, k60, u'home_assistant').
1717657259: Client home_assistant has exceeded timeout, disconnecting.
1717657279: New connection from 172.17.0.1:59672 on port 1883.
1717657279: New client connected from 172.17.0.1:59672 as home_assistant (p2, c1, k60, u'home_assistant').
1717657693: Saving in-memory database to /mosquitto/data//mosquitto.db.
1717657895: Client ring_mqtt has exceeded timeout, disconnecting.

And also in the HA history where all entities had frequent gaps causing HA to report a lot of events every time the connection was lost and re-established.

So I went for another approach and configured all containers to use a custom bridge network and removed all port mappings of the host machine. In this case I used the container name in the json above ("READ_BELOW") as well as in the HA MQTT integration. The HA connection succeeded but RingMqtt failed to connect so I opened a console on all three machines and started to ping each other. Every machine was able to reolve the others IP and ping worked for HA<->Mosquitto and HA<->RingMqtt but not for Mosquitto<->RingMqtt. I see absolutely no reason why this should not work but it ain't.

Questions:
What is the appropriate setup to get this to work? Custom network? Default network?
What could be causing the permanent disconnects and how to get rid of them?
Why is a direct connection Mosquitto<->RingMqtt not possible?

[srudin]

Seems like most questions in here remain unanswered... too bad.

Anyway, I was able to find and solve my problem(s). For the record:

1. For whatever reason both the Mosquitto and the RingMqtt container had the same mac address assigned which is why it wasn't possible to communicate from one to the other. This probably happened when moving them from the default bridged to the custom bridged network but not sure if Docker itself or Portainer (which I use for managing my containers) is responsible for that mistake. After setting them to a different value everything worked fine again.
2. After everything worked fine again I was able to get the connection working - this time directly over the custom Docker network and not through mapped ports. With this setup everything is running nicely for 12h+ already so I assume the disconnects/interruptions were related to the unnecessary routing in the previous network setup.

Learning 1: Always check all settings of your Docker containers.
Learning 2: When connecting IoT devices always use the most simple and direct network possible.

[tsightler]

Seems like most questions in here remain unanswered... too bad.

I'm not sure why you would say this, I personally respond to 95% of the posts here, and a good 70+% have actual resolved answers which is higher than the vast majority of free projects. I especially do my best to answer those that are actually related to ring-mqtt issues vs some unique environmental issue like the one you posted.

Unfortunately, I'm just one person with an actual real job and family that takes 98% of my time and this project, which I just stupidly threw out there expecting only a few geeks to use, now has 10,000+ users of which 95% seem to not understand anything about docker, mqtt, networking, etc, and I somehow don't have enough time to help every person running some unique setup to troubleshoot their specific environment, but quite honestly, I find having that expectations to be somewhat crazy.

I would have eventually responded to this post and tried to help, but I was travelling for work last week when you posted, and I had to address the other major issue caused by HA publishing breaking changes without notifications.

I'm glad you got it working and thanks for sharing your result.

[srudin]

Sorry - I didn't mean to offend you!

But looking at the discussions page right now I can see 15 out of 25 issues "Unanswered". Mine also was there without an answer for 4 days. That's why I said "Seems like most questions in here remain unanswered..." And I said "Too bad" because I think this is actually a cool project and I'd like people to overcome their issues and use it.

However, I fully understand your point and I know the problems of voluntary work getting over your head. From my opinion giving priority to the family and (after that) to the job is definitely a must! I am very thankful that you created and published this project and I hope you find the time to maintain it. (Actually I think you should get paid by Ring for this but of course this is never going to happen.) So again my apologies if I offended you - guess I just mixed my general frustration with lacking support for paid products with the situation here, which of course is not fair.

[tsightler]

If you are just looking at the flags of whether there is an accepted answer, well, that's pretty optimistic that people will actually follow-up enough to get an answer, even if the problem is resolved. If you look at the posts, pretty much every post has at least one response, sometimes the response is a simple as, what do the logs say, because people post useless things like "My sensors are offline", like somehow that can possibly be useful. Even then, I try to respond, ask for logs, and basic stuff, but about 75% of users post one thing and never respond again, thus of course those will not have an accepted answer. Even many that do resolve their problem either never post, or most only that they fixed it and never select "accepted answer". People are imperfect.

Yes, you posted 4 days ago, but I would have eventually responded, there's indeed no SLA, but if you feel unsatisfied I'm happy to give you your money back! :-)