Client goes crazy after receiving group message

[Lodifice]

After I receive a group message, my client sents the following message

<<< Contact ***REDACTED*** requested a group sync. >>>

which is immediately replied as

<<< Group creation and membership information was set/sent by the Group Admin: ***REDACTED*** >>>

This quickly repeats for about 100 times and also continues after disconnect/connect.
It appears in a separate chat window which does not have a name.
The group itself does not appear in the contacts list, but all group members made an appearance.

The log output has the following form:

[2020-04-17 12:29:26.045] [main] [info] Sending MESSAGE_SUCESS for message ID <> send to group <> owned by ID <>.

(of course I have replaced the IDs with <>).

Now, is this a bug?
Do I have to add the group explicitly somehow (I started from my plain Threema ID, see #60 – the first message of a contact just added the contact and opened a chat window containing the message, so I'd expect the same for groups)?

[Lodifice]

Quick correction: the group appears as a clickable item in the group list, its name is just the empty string.

[blizzard4591]

The scenario is the following:

• The client does not know group A.
• You connect and receive a stack of messages from group A that were sent while you were offline.
• Since the client does not know group A, it requests a group sync from the group admin.

Somewhere in there, something goes wrong. Looking at the code, my guess is that with the client being online, the sync request is sent and then the client does not "see" the next time that a request was already sent (but of course, the reply is last in the queue of incoming messages).

So, I see two possible solutions:

1. Just wait until all previously un-received messages have been received, once this is done, it should be back to normal.
2. Enable the "Trust others" option in the options. This will just accept group messages from unknown groups for the time being and wait for the sync to "complete the picture".

I am also looking into fixing the situation altogether, but that is not as simple as I thought.

[Lodifice]

Just for my understanding: how are unseen messages determined? The sync request targets the group admin who directly responds, as it seems. But only with the member list.

There were four messages send to the group since I switched to openMittsu, I received nr. 2 and 4. So my client is basically waiting for the other two? They won't arrive no matter how long I wait, the group just fills up with sync requests and responses. Sometimes there are many minutes between new messages of this kind, and then it starts going crazy again.

I tried "trust others" which lead to an explosion in CPU consumption and a freeze of my laptop for several minutes. Then, the group name appeared and also other types of sync responses (containing the group name). Interestingly, the group name vanishes and re-appears as new messages are sent/received.

Now, with over 5000 sync messages in the group, I decided to disconnect. Seeing that you're working on the issue right now, just let me know when I should test something.

[blizzard4591]

This is extremely weird behavior. I will add some debugging statements for this case and then we can investigate.

There is a difference between unseen and unreceived.
Unseen means that the UID of this message has not been seen before, i.e. it is not in the database.
Unreceived means, when connecting to the server, the server will try to deliver this message and waits for an "ack" from the client. As long as we do not send this ACK, the server will try sending this message everytime we connect (but only once per connection), for up to 14 days.

Since your client does not know about the other messages, its not waiting for them.

[Lodifice]

Have you made any progress on this, e.g. adding the debug statements you mentioned? A few weeks ago I started using the latest master.

It still sends these requests, however, the frequency is much lower and it does not escalate anymore.
I have also noticed that the group name only appears when I "trust others". Then sending sync requests stops.

What's about these sync messages in the database? Can I savely delete them? They pollute the message log quite a bit, especially in the group that has 6000 of them.

[blizzard4591]

I am sorry, I was/am swarmed with work. I added a bunch of debug statements, so please try again and report back what you see!

[Lodifice]

No worries, as you see, it took me some time to respond as well.
I have posted a log here (with anonimized Threema IDs).

It is divided into three blocks which somehow correlate with my observations.
First, the option to "accept messages from users whose group membership has not (yet) been confirmed" was disabled. Someone sent a group message, but it did not show and the group title did was not set.

Then (second block) someone else in the group sent a message which was displayed.

Later (third block) I enabled the option from above. Now I saw the first message I talked about as well as the group title (also some other messages arrived in that group). Unfortunately the third block is polluted with some private messages as it appears.

Fun fact: there is another group owned by me. The sender of the first message I talked about also sent a message to that group, which didn't show up. After enabling the accept option it did (I think you can see it in the log). The name of that group still is not set, but I guess this makes sense since I'm the owner and my client does not know the name.

If you need more specific testing, just let me know. Maybe we can also setup a synthetic group for that.

[pingberlin]

Can be closed.

I also had this issue with large groups. The exponential backoff has fixed this for me.

FYI, I'm running this version:

Version: 0.9.15 (+17 commits) build from revision g18235ef (DIRTY).