History keeping for drop messages

[thechauffeur]

We have to define a way to keep history for drop messages. This is module dependent but we should help modules with this.

This is an issue because we will not use FFsync anymore and hence do not have a sync protocol yet.

Idea: Use Qabel storage to hold a history file for specific modules and provide functionality to ease the history keeping process for module implementations.

[ghost]

I already implement this for saving the chat messages for every user. Maybe we find some ideas there.

But I do not believe that to save it on the server is a good idea because ...
... the size of the history is one problem.
... we have to implement a lock because every device of a user can write to this history.
... different device can have different settings i.e. shares

I think a local history shall be the better solution.

[aBothe]

Ich würde vorschlagen, Messages clientseitig á la Mail-Inbox zu behalten und diese im Bedarfsfall im Namen des Servers an andere Clients/"Peers" zu schicken.
So wird das Bottleneck Server umgangen, auch wenn es vielleicht schwierig wird, etwas von einem Smartphone per Socket auf ein anderes zu schicken -- weil es die ISPs u.U. nicht zulassen, einen Socket mobil zu binden.
Dennoch, einen History-Blob im Storage vorzuhalten klingt vernünftig. Oder noch besser, mehrere History Blobs, wo jeder client das individuelle Log hineinschreibt und für ein späteres Nachsehen mehrere Logs gemerged werden können.
Wäre dies dann jedoch nicht zu high-leveled, d.h. zu modulseitig?

[f-porter]

Why not just provide the functionality in the drop message module to create a module specific history and to save messages there, locally? Thus there will only be messages stored which are determined to be stored.
For synchronization with other devices:
-- we implement a functionality to request the history of the other clients of this module via a drop message to the same identity and the other client sends it by Qabel Storage (similar to @aBothe suggestion, but with the use of the Qabel protocols). After that initial sync, new (sent) messages which are stored to history will be sent to the own identity's drop box.

The synchronization with other devices could be done via the Qabel Storage or by sending drop messages to his own identity/contact and the drop module puts them to history.

@k-c13:
-- The locking problem is also a problem for the file-module (several user change a shared file and sync it with the server) and we have to solve it anyway
-- The different settings of a device do not affect the message history as they are part of the configuration (or am i wrong?)

@aBothe:
Sending the messages from peer to peer would be another complex part to implement (another protocol) just for this history and therefore IMO not really reasonable.

[thechauffeur]

+1 for providing history via drop and storage functionality. I'm unsure however if this should be a new "history module" or if it should be part of the core.

[thechauffeur]

Oh and BTW, I'm OK with keeping things locally as much as possible - at least for the BETA.

[ghost]

I think that the history shall be a part of every module and shall save locally.

The drop messages are on the drop server and so the history of every module can do from that message. And every module have to decide which drop messages are so important that it have to save to the own module history. An example is the chat module.

BTW: I will add the last update item to the local setting to know when the last messages readed from the drop servers.

[f-porter]

What do you mean with "the history shall be a part of every module"?
That every module shall implement a history functionality on its own?

"The drop messages are on the drop server and so the history of every module can do from that message"?

[f-porter]

And I would call the "last_update" item something like "last_drop_update". It could be possible that there will be more last update timestamps in the settings.
Or put the "last_update" in the module specific local settings of the drop module.

[ghost]

@f-porter:
last_update
I agree with you because the local settings change to less sub directories as I start with. I will change it
history
Any module shall have its own history from the drop message forward from the core to the module. Example: If you start a chat again the chat module will know which sent data will shown. So the chat shall handle the chat history. The core module shall also have a small history but only with information which is important for the core. For the core the module message are not interesting. The only forward it to the module.

[thechauffeur]

We have to write that the core will handle this (in the drop message documentation). This is needed for @malte-z.

[malte-z]

@Qabel/internal-write-access: I added some infos for the drop history https://github.com/Qabel/intern-doc/wiki/Qabel-Client-Drop
Is everybody ok with that?

[ghost]

I agree

[thechauffeur]

OK.

[thechauffeur]

@Qabel/internal-write-access we need input on this.

[thechauffeur]

Or we decide this is post BETA.

[L-Henke]

Maybe a modified concept of a collapse key might work for us. Like: http://developer.android.com/google/gcm/adv.html
(This is just an idea and might not work at all)

[thechauffeur]

@Gottox you self-assigned this one. If you still have an idea or something to contribute just tell us. You can formulate this "something" later. I just wondered about the assignee.

[aBothe]

See https://github.com/Qabel/qabel-sync-module/wiki/Components-Sync
I thought myself that "modules should 'convert' their specific drop histories (e.g. chat logs, mail archives etc.) that shall be synced into one single JSONizable history, meaning the sync module will not preserve a drop history but only e.g. a JSONized chat log. This will likely save time on each side of the implementation."
No collisions, no timing attacks or other things. Instead each module handles its own things - the respective API for persisting things might be provided by Sync though.

[thechauffeur]

This is POST BETA now.

[thechauffeur]

Not only is this vulnerable to a replay attack but also it would help modules if the core would handle not to handle already handled drop messages.

@L-Henke you did a work around for that in the android branch. Let us talk about this.