Integration causes Home Assistant to lock up for 30-40 seconds when generating map (vacuum running)

[joq3]

Describe the bug
When using v2.0.0b16 on Home Assistant 2024.8.1 the integration causes Home Assistant to lock up and become unresponsive for a while (30-40 seconds) before coming back again. This seems consistent to when the map updates, it does only happen when the vacuum is moving and the map is updating, not when it is docked.
Disabling the integration gives me no issues with slowdowns on Home Assistant.

I have tried redownloading and installing the integration again. Restarted Home Assistant through UI and the Docker container.
I have tried with the low resolution map without any change.

Running Home Assistant in docker on a dedicated NAS with i5-4670K and 24GB RAM.

To Reproduce
Activate Dreame integration v2.0.0b16 and start cleaning with vacuum. When the map is being created Home Assistant slows down and locks up for about 30-40 seconds before being responsive again until the next time the map updates.

Expected behavior
The integration to not cause Home Assistant to lock up.

Debug log
home-assistant_dreame_vacuum_2024-08-16T09-44-19.366Z.log

Additional Information (please complete the following information)

• Integration version: v2.0.0b16
• Model Name [e.g. dreame.vacuum.p2028]: dreame.vacuum.r2228o
• Firmware Version [e.g. 1156]: 4.3.9_3204
• Home Assistant Version: 2024.8.1
• Configuration Type [With or without map support]: Map support with Xiaomi Vacuum Map Card
• Errors or warnings shown in the HA logs (if applicable):

[Tasshack]

Your 11 year old CPU cannot keep up with the rendering.
If you think about it, the CPU you are using is actually older than Home Assistant itself.

[joq3]

Your 11 year old CPU cannot keep up with the rendering. If you think about it, the CPU you are using is actually older than Home Assistant itself.

I have had no issues at all with previous versions of this integration. No performance issues at all.

[Tasshack]

Your 11 year old CPU cannot keep up with the rendering. If you think about it, the CPU you are using is actually older than Home Assistant itself.

I have had no issues at all with previous versions of this integration. No performance issues at all.

Can you downgrade and try again?

[Tasshack]

I have revised the changes between v2.0.0b15 and v2.0.0b16 but cannot see a modification that will cause this kind of performance loss.
I can see in the logs that rendering a single map frame took more 5 seconds consistently, that may mean something else are using the CPU.

[Tasshack]

I am running 10 different vacuums at the same time on my HA on a Windows WSL virtual machine (which is notoriously much more slower than a normal linux virtual machine) without an issue like you have mentioned.

[Tasshack]

It may be because of a map data corruption in a way that the integration does not expect. That kind of bug would be very hard to find and fix so if that was the case, restoring the map to a previous state should resolve your issue.

[joq3]

It may be because of a map data corruption in a way that the integration does not expect. That bug will be very hard to find and fix so if that was the case, restoring the map to a previous state should resolve your issue.

How do I restore the map?
I have removed my vacuum completely, and added it again. Same thing happened. I will try to downgrade to an earlier beta to check.
When looking at my servers CPU usage I can see that one of the CPU cores is at 99-100% when this occurs. However the overall load is about 20-30% on the whole CPU. Is it possible to use more cores, or is this a limitation in Home Assistant with Docker maybe?

Weird that this started happening, as I have done no changes to my server, not running anything else at the moment.

Would it be possible to render an even lower quality map that is easier on the CPU?

[joq3]

In this comment you are talking about a RPi 3 being able to render the map:
#52 (comment)

Surely my old i5 is more powerful than that?

[Tasshack]

How do I restore the map?

Check the Map Management from the app.

Is it possible to use more cores, or is this a limitation in Home Assistant with Docker maybe?

I am not familiar with the HA Docker setup.

Would it be possible to render an even lower quality map that is easier on the CPU?

Normally Low resolution map option should fix your issue if not that means it is not the rendering that causes the issue. You can confirm this by disabling the Current Map and Saved Map entities from HA manualy. If you disable the camera entities from HA, there won't be any rendering at all.

[joq3]

How do I restore the map?

Check the Map Management from the app.

Is it possible to use more cores, or is this a limitation in Home Assistant with Docker maybe?

I am not familiar with the HA Docker setup.

Would it be possible to render an even lower quality map that is easier on the CPU?

Normally Low resolution map option should fix your issue if not that means it is not the rendering that causes the issue. You can confirm this by disabling the Current Map and Saved Map entities from HA manualy. If you disable the camera entities from HA, there won't be any rendering at all.

I see no difference in map quality when enabling Low resolution map, is there any way to confirm this setting really takes action?

[Tasshack]

I see no difference in map quality when enabling Low resolution map, is there any way to confirm this setting really takes action?

Right click and select open image in new tab, you should see half sized image than before when it is enabled.

[Tasshack]

In this comment you are talking about a RPi 3 being able to render the map: #52 (comment)

Surely my old i5 is more powerful than that?

It is not just pure power when it comes to the old CPUs. For instance; my old AMD laptop cannot encode videos becuse it does not have a built in hardware decoder like a RPI3 but that does not mean it is slower than a RPI.
Architechure and other HW features builtin within the cpu affects a lot of performance when it comes to the software can run that HW feature or not. Python can be very picky about the hardware optimization options about rendering digitized images.

[joq3]

In this comment you are talking about a RPi 3 being able to render the map: #52 (comment)
Surely my old i5 is more powerful than that?

It is not just pure power when it comes to the old CPUs. For instance; my old AMD laptop cannot encode videos becuse it does not have a built in hardware decoder like a RPI3 but that does not mean it is slower than a RPI. Architechure and other HW features builtin within the cpu affects a lot of performance when it comes to the software can run that HW feature or not. Python can be very picky about the hardware optimization options about rendering digitized images.

I understand, is there any CPU I could upgrade to which uses the same socket but will perform better for this application? I don't want to upgrade the whole motherboard, ram, cpu etc.
Or do you have any recommendation on a good separate Home Assistant mini pc/RPI or alternative?

Checked the low resolution map, and I get a smaller image when it is activated.

[Tasshack]

I understand, is there any CPU I could upgrade to which uses the same socket but will perform better for this application? I don't want to upgrade the whole motherboard, ram, cpu etc.

First you should confirm that this is actually an hardware issue that I sugested earlier that cannot be solved by software. Installing a fresh HA on a different virtual machine can be used to confirm that.

Or do you have any recommendation on a good separate Home Assistant mini pc/RPI or alternative?

I really don't recommend an ARM cpu for this application so any RPI or even HA Yellow is off the list but any X64 CPU not much older than 5 years can be good so you can buy an old thinclient PC or a new mini PC with an Intel CPU (not much actually expensive than RPI4).

[joq3]

Is there a good option for removing the map, but still get some information of the progress. Like how far it has gotten (percentage), or current location (room name)?

[Tasshack]

Is there a good option for removing the map, but still get some information of the progress. Like how far it has gotten (percentage), or current location (room name)?

If you are not using a card to select rooms for room cleaning then you really dont need a map image. You will still get the Current Room sensor and other map entities without the camera entity because integration still be parsing the map data but just not rendering it.
L10s does not have information about the job progress, only newer vacuums have that specific information can be read by the integration.

[Tasshack]

how far it has gotten (percentage)

You can calculate this on your HA using Cleaned Area sensor.

[somenewaccountthen]

I have 800mb mem free , when i start the integration it fills it and the system hangs.
I had the SSH open and was looking at memory usage. As soon as the integration starts the memory starts filling.
Some seconds later the connection was lost.

I see you saying enable Low resolution map.
What is "Low resolution map" where and how can i enable it?

[Tasshack]

I have 800mb mem free , when i start the integration it fills it and the system hangs. I had the SSH open and was looking at memory usage. As soon as the integration starts the memory starts filling. Some seconds later the connection was lost.

I see you saying enable Low resolution map. What is "Low resolution map" where and how can i enable it?

#52
#122

[Tasshack]

I have 800mb mem free

App on your phone actually uses more than that.

[somenewaccountthen]

Thanks for your reply.

I installed your latest beta and chose low resolution map.
It works now and i can see the map.

But guess what!
The map it gets from the vacuum includes a part that doesn't exist. it is 4x larger.
So no wonder i suddenly started running out of memory.

I think for some people It may be as simple as to reset your map in the xiaomi/dreame app.

[Tasshack]

@somenewaccountthen
Normally integration should crop empty pixels from the map but I am not sure what is going on.
Can you share a debug log so I can reproduce and fix it?

[Tasshack]

@somenewaccountthen I have suggested this before

It may be because of a map data corruption in a way that the integration does not expect. That kind of bug would be very hard to find and fix so if that was the case, restoring the map to a previous state should resolve your issue.

[somenewaccountthen]

@somenewaccountthen Normally integration should crop empty pixels from the map but I am not sure what is going on. Can you share a debug log so I can reproduce and fix it?

Not really i have no log, i don't think it's relevant either.
The map was just bigger (4x) than the house.
This is the bad map.

I rolled back to an older map in the Xiaomi app and that works fine.
That's all it was in the end.

[somenewaccountthen]

Anyway.. crisis averted.
And me and my dad are now on the v2 beta.

On his pi 2G he now has 30% mem free again.
On mine Mem is not an issue :P

If you tell me where the log is i can send it.
But like i said i think the image was just too big for his pi to handle.
Nothing to do with your code.

[Tasshack]

But like i said i think the image was just too big for his pi to handle.
Nothing to do with your code.

You are right, image looks correctly rendered, there is nothing wrong with it or some kind of corruption.