Waku Store v3 protocol

[jm-clius]

Waku requires an improved version of the Store protocol, marking a crucial increment towards a Store synchronisation protocol.

Amongst other things, this new protocol must:

• introduce the concept of deterministic message hashes to index messages
• consider the Store as a key-value store
• allow for querying a list of keys (message hashes) from the Store
• allow for querying for the full message content (values) of a set of keys from the Store
• keep all previous value-based filtering (e.g. content topic, timestamp) in place

The proposed PR to simplify the Store protocol and use message hashes as index/cursor, can be used as a starting point.

[chaitanyaprem]

• introduce the concept of deterministic message hashes to index messages

Just to clarify the existing Waku-message hashing algo is already deterministic right, so no change is required or expected to be done wrt it. Only the concept of mapping message hash to message is being newly introduced.

[jm-clius]

the existing Waku-message hashing algo is already deterministic right

Yes, it is. If an application crafts a message with a clashing hash, it MUST use the meta field to disambiguate.

One related note is that we cannot currently reconstruct the hash from the stored fields in the archive, because we don't store meta - in other words, if we believe it necessary to in future verify that the stored entry matches its key/hash, we will have to expand our DB scheme to include meta.

[chaitanyaprem]

One related note is that we cannot currently reconstruct the hash from the stored fields in the archive, because we don't store meta - in other words, if we believe it necessary to in future verify that the stored entry matches its key/hash, we will have to expand our DB scheme to include meta.

Interesting point, maybe we should start storing this.
This will also address a point made by @kaichaosun in some discussion about validating the contents of a message returned by StoreNode by the requestor.