Map not working on newer bots (Deebot X1, T10, ...)

[xXDjMikeXx]

Checks

• I have searched the existing issues and no issue is describing my issue
• I have checked the FAQ
• I have checked the documentation
• I have installed the latest version

The problem

Map is not shown properly with Deebot X1 (last Model 950 worked without issues)

Diagnostics information

deebot-0fbf64dfdbed3ff9aedda7dc22e10a23-Sigbert-84ec494accf775ef946082380388d0e4.json.txt

Anything in the logs that might be useful for us?

No response

Additional information

No response

[edenhaus]

I have only the 950 and as student I cannot afford to buy me the new X1. Probably they have changed the command, how you can retrieve the map data.

If you have the technical skills to intercept and analyse the traffic please feel free to do it. Each contribution is welcome :)
Also you can buy me some beers on https://www.buymeacoffee.com/edenhaus, so I can buy me the X1 in the future and the support for it :)

[xXDjMikeXx]

Hmm okay, I wish I could help but I don't know how to do this.... I do not have skills for stuff like that haha

[Sildenafil99]

Same issue with Ecovacs T10 model

[Nismonx]

yes, same on my X1..... don't think this is a HA issue as valetudo on my other vacuum is displaying well.

if pointed in the right direction, I don't mind snifing the traffic...

tia

[xrh0905]

Same on my X1 OMNI.
I can snifing the traffic if you want, I just need to know the right place to look at.

[edenhaus]

Capture the traffic between the robot and the app.
Then filter for map commands and analyse them.
Probably your models are using V2 commands, which are currently not implemented.

[Nismonx]

Capture the traffic between the robot and the app. Then filter for map commands and analyse them. Probably your models are using V2 commands, which are currently not implemented.

I understand what you are saying, just not sure what tools to use to do so.

Also, not sure if this is relevant at this point, but I believe there are currently issues with either the deebot app or robot itself when it comes to the map integrity. My self and other have been experiencing issues where the maps appear to have a mind of its own in the official app! In the last 2 weeks I had to re-do quick map several times. Rooms often merge together other time the actual rooms changes size. Don't even get me started on the 3D Maps lol.

Again, I'm just speculating based on my experience and the comments of others in the official app in Google play.

[Marow]

Thats what i mean in my issue long time ago. But i dont know what i can do to help to fix that.
#247

[TeCHiScy]

I recently analyzed the data of minor_map command on my X1 robot, the discovery is:

1. on X1 robot, value in a piece (100x100 array) is not only 0x01 (floor), 0x02 (wall), 0x03 (carpet). value = 0x02 forms only part of the house outline.
2. the lower 5 bit seems to represent the "area index". I divided the map into 32+ areas in the ecovacs app, and draw a gray-scale picture of lower 5 bit values, it shows same area divides as in the app.
3. the higher 3 bit seems to be always 011, which forms a union set (i.e, complete map) of all the areas.

[edenhaus]

@TeCHiScy Please feel free to open a PR to make the map working on the X1

[TeCHiScy]

@TeCHiScy Please feel free to open a PR to make the map working on the X1

Yes, I'm willing to do that. Can u share a picture of the map that works fine on your robot? I'd like to do some comparasion, to verify the results.

[edenhaus]

Maybe you need only to set the color for your new values here.
Otherwise you need to update probably the GetMinorMap command.

My map look similar to https://deebot.readthedocs.io/assets/images/custom_vacuum_card.jpg (it's a old image but nothing really changed)

[MetroAffe]

Same problem here.
For comparison I've made screenshots of both maps.
In the App are two different types of "floor". Only the dark grey areas and normal walls are rendered by this Integration.
It looks like the developer of Deebot added a second layer with Room-floors and AI generated walls.

[edenhaus]

I don't have a X1 and as I have nothing to do with ecovacs, I also don't get support from them. I cannot afford to buy each model. You can buy me always a coffee at https://www.buymeacoffee.com/edenhaus, so I can buy me new models.

Otherwise someone with a X1 model needs to analyze the traffic :)

Without a X1 it is hard to guess, which commands have changed and therefore currently I cannot fix this issue and need help for you guys, which have the X1.
I know about the problem and the only solution is to analyze the traffic or buy for me the X1 😄

[MetroAffe]

Okay well I will try to do this. But I need some help.
Which packages are important? How to provoke the commands? Is the connection not encrypted?

It would be no problem to send you all details or the whole traffic which are needed if you tell me some more Information.
Please contact me preferably by email and in maybe in German (TU Vienna? :D)

[Nismonx]

Okay well I will try to do this. But I need some help. Which packages are important? How to provoke the commands? Is the connection not encrypted?

It would be no problem to send you all details or the whole traffic which are needed if you tell me some more Information. Please contact me preferably by email and in maybe in German (TU Vienna? :D)

You may need something like "packet capture" app or "wireshark" in a laptop. This is as far as I can help sorry!

[mrbungle64]

@MetroAffe @Nismonx @edenhaus

I'll try to find the time this weekend to analyze the traffic.

[MetroAffe]

Yes, of course. I already captured the traffic between the App and Ecovacs/Deebot Server. But I think its encrypted - not surprising …. So I have no idea how to get the Information which commands to check for the Map. And I dont have the time to read into the API of Ecovacs/Deebot

[edenhaus]

Yes, of course. I already captured the traffic between the App and Ecovacs/Deebot Server. But I think its encrypted - not surprising …. So I have no idea how to get the Information which commands to check for the Map. And I dont have the time to read into the API of Ecovacs/Deebot

If you for example use mitmproxy and setup it as proxy of the device you want to capture the traffic, you will see the traffic unencrypted.

@MetroAffe I prefer to stay here and write in english so everyone can follow and understand it

[MetroAffe]

Okay did it. And its working. I‘ve made a screenshot - more as proof that its running. Unfortunately I had to black out a lot because I don't like to do it publicly.

On which packets would you like to look closely at? If you would like to have the whole capture you know how to contact me. Its hard work to blure all private keys to share it in public environment…..

[MetroAffe]

Maybe I figured out some new commands and in my opinion this could be the Problem:
getMapInfo_V2
getMapSet_V2
getPos_V2

These got triggered too:
getSpeed
getBattery
getMinorMap
getMapTrace
getInfo
getCachedMapInfo
getMapInfo
getMajorMap

[edenhaus]

All currently implemented map commands can be found here.
For sure all V2 commands are missing, but you need to check also the other map related commands, if there was added/changed some properties.

[MetroAffe]

Response of getCachedMapInfo starting with index 3 even it is empty:
{
"backupId": "",
"built": 0,
"index": 3,
"isFastBuilding": 0,
"mid": ""
"name": ""
"status": 1
"using": 0
},
{
"backupId": "123456789",
"built": 1,
"index": 0,
"isFastBuilding": 2,
"mid": "988764321"
"name": "Zuhause"
"status": 0
"using": 1
},
{
"backupId": "",
"built": 0,
"index": 2,
"isFastBuilding": 0,
"mid": ""
"name": ""
"status": 1
"using": 0
},
{
"backupId": "",
"built": 0,
"index": 2,
"isFastBuilding": 0,
"mid": ""
"name": ""
"status": 1
"using": 0
}

[edenhaus]

Please give a look at the current implementation to understand how the commands interact in between. Explaining here each command and how they interact is to much. Some commands are also documented here but still work in progress...
Also all docs is a guess from our side as ecovacs is not sharing any information

[MetroAffe]

That’s impossible. I already took a look at both. It’s not enough to understand a few lines of the code. For troubleshooting its needed to overview the whole code.

For now I’m done.
To everyone else: Take a look at following:
I think the Problem could also be „getMapSubSet“. This Command got never triggered and „getMapSet_V2“ includes subsets as one String now.
If that could be the problem I'm at the end of my abilities

[edenhaus]

Than please share information about the new getMapSet_V2 command.

In short the map works:

• Call getCachedMapInfo to see which map is currently used. Storing mid (map id) for further processing
• Call getMapSet which the mid and all available getMapSet Maybe the X1 has a new Type. Please analyse getMapSet_V2
• Call getMapSubSet for all msid (map subset id) returned by getMapSet. getMapSubSet returns the map in the property value. Probably getMapSet_V2 has different properties, which need to analyzed.

Please also note that analyzing the traffic is not a 5 minutes job. You need to the check some commands and verify what has changed. Maybe also some reverse engineering of the android app could be necessary to understand how the new data must be processed. And for all of this you need a lot of time

[MetroAffe]

getMapSet_V2 type ar:

{
   "ret": "ok",
   "resp": {
      "header": {
         "pri": 1,
         "tzm": 120,
         "ts": "1681634062023",
         "ver": "0.0.1",
         "fwVer": "1.5.9",
         "hwVer": "0.1.1",
         "wkVer": "0.1.54"
      },
      "body": {
         "code": 0,
         "msg": "ok",
         "data": {
            "type": "ar",
            "mid": "451359723",
            "msid": "2012069423",
            "batid": "dbjhli",
            "serial": 1,
            "index": 1,
            "subsets": "XQAABACdAQAAAC2WwEHwYhHYQTGecueIxnXIJ1eT5msc+tUDcwyVN/qN4VpHCZICfcK0XSbLbHa2rDVyXa6W83QaBll7wDe0DbcHD7V0w73VqcenLcwRGTtLfhGyQAZckNp/W8R97OF03u2I1EVf3L/KtrKY+H8yXgcH9C7o2I4GrQaStxFptirT8uekcfwBeOTGNzwAe/TSzNO3H2IynQomiQgiADy+9AMs02A=",
            "infoSize": 413
         }
      }
   },
   "id": "7fGU",
   "payloadType": "j"
}

getMapSet_V2 type vw:

{
   "ret": "ok",
   "resp": {
      "header": {
         "pri": 1,
         "tzm": 120,
         "ts": "1681634069523",
         "ver": "0.0.1",
         "fwVer": "1.5.9",
         "hwVer": "0.1.1",
         "wkVer": "0.1.54"
      },
      "body": {
         "code": 0,
         "msg": "ok",
         "data": {
            "type": "vw",
            "mid": "451359723",
            "batid": "cpipli",
            "serial": 1,
            "index": 1,
            "subsets": "XQAABAACAAAAAC2XPAAAAA==",
            "infoSize": 2
         }
      }
   },
   "id": "HSoS",
   "payloadType": "j"
}

getMapSet_V2 type mw:

{
   "ret": "ok",
   "resp": {
      "header": {
         "pri": 1,
         "tzm": 120,
         "ts": "1681634069523",
         "ver": "0.0.1",
         "fwVer": "1.5.9",
         "hwVer": "0.1.1",
         "wkVer": "0.1.54"
      },
      "body": {
         "code": 0,
         "msg": "ok",
         "data": {
            "type": "mw",
            "mid": "451359723",
            "batid": "cpipli",
            "serial": 1,
            "index": 1,
            "subsets": "XQAABAACAAAAAC2XPAAAAA==",
            "infoSize": 2
         }
      }
   },
   "id": "zR84",
   "payloadType": "j"
}

[MetroAffe]

Well its much easier to get a solution. The guys from iobroker have had the same problem - and maybe already a solution!
They are working on a Version which implemented the new V2 commands:
getMapImage:
mrbungle64/ioBroker.ecovacs-deebot@master...feature/v2/GetMapImage

[Nismonx]

@MetroAffe the commits are over a year old!

I'm afraid the last time Sascha stepped in recent conversations here, he confirmed it still a work in progress.

[mrbungle64]

@MetroAffe

Well its much easier to get a solution. The guys from iobroker have had the same problem - and maybe already a solution!

Not really 😉

They are working on a Version which implemented the new V2 commands

I already implemented some of the V2 commands, but I still have the same problem with the map generation.
You posted a link to an outdated feature branch. I'll delete this branch now 😉

On Friday I already posted that I'll try to find the time this weekend to analyze the traffic:
#268 (comment)

I'll let you know when I've found a solution.

[MetroAffe]

uh ok oops. I actually just overlooked the fact that it was from you and posted not the correct link. I read about it in this thread:
https://forum.iobroker.net/topic/55189/ecovacs-deebot-adapter-informationen-und-praxistipps/42?_=1681653819687&lang=de

If its not working yet ... How is it this possible:
https://forum.iobroker.net/assets/uploads/files/1677221745333-ecovacs.png

But I think in that case the problem is in the best hands! :D
If I could help you in any way with my newbie skills don't hesitate to ask.
Thanks for your work!

[mrbungle64]

@MetroAffe

If its not working yet ... How is it this possible:
https://forum.iobroker.net/assets/uploads/files/1677221745333-ecovacs.png

This is the image of the last cleaning from the cleaning log 😉

[MacrosorcH]

Hi everybody, any progress at all with fixing the map for X1? @mrbungle64 Did you ever find a solution?
Thank you all for the great work!

[mrbungle64]

@MacrosorcH

@mrbungle64 Did you ever find a solution?

Yes and no. The solution is mostly only a theoretical one so far...

I think the only thing missing from the map is the border lines (boundaries).
I already had a short conversation with @edenhaus about it some month ago.

Some of the results I've already got looked very similar to the map in the Ecovacs Home app.
But the boundary data comes from a different dataset, so I have to refactor my code for it, because drawing in the lines for the boundaries doesn't work reliably yet. Unfortunately I haven't had the time to do this yet...

[edenhaus]

Here is a short update on my side. I will refactor the library to allow different commands for different models.
After that significant refactoring, we can implement V2 commands and activate them for some models.

Only with your help can I implement the required changes, as I don't have your model.
And as always, I can give no ETA.

[StefanoGiu]

I have a deebot X1 Omni, if you need help...

[TruantRiccardo]

Hi, I have a t10 plus and I am experiencing the same problem, so if you need information also from this model I can get it for you, if you point me to what.

[maoz82]

I can also support in case you need help, I've a Deebot X1 omni.

[edenhaus]

The new ecovacs models are using different commands for the map, which are currently not implemented.
I currently have no time to analyze these new commands and fix the map. Feel free to capture all commands and analyze them and open a PR for adding support. Please see also https://github.com/MVladislav/bumper/blob/dev/docs/dev/Ubuntu_Android_Sniffing.md#extend-mitmproy, which should help to capture commands but I haven't tested it yet.

[lotablet]

Any update about this?
I have Deebot X1e Omni version, and this is exacty my problem too.

I don't have many skills, but I can provide any information you need. Just teach me a little bit 😁

[TeCHiScy]

@MacrosorcH

@mrbungle64 Did you ever find a solution?

Yes and no. The solution is mostly only a theoretical one so far...

I think the only thing missing from the map is the border lines (boundaries). I already had a short conversation with @edenhaus about it some month ago.

Some of the results I've already got looked very similar to the map in the Ecovacs Home app. But the boundary data comes from a different dataset, so I have to refactor my code for it, because drawing in the lines for the boundaries doesn't work reliably yet. Unfortunately I haven't had the time to do this yet...

Same as me, still wonder how to get the boundaries as shown in the Ecovacs app.

[edenhaus]

DeebotUniverse/client.py#372 tires to fix the map issue but it still needs some work before it can be merged. Please be patient