Proxy messaging

[muhamadazmy]

Introduction

Currently we use RMB for the general client<->node communication. There are some special cases when the client can't run his own instance of RMB then he uses a proxy to do it for you. But for the simplicity of this document we will not discuss the current proxy scenario and instead focus on the general RMB to RMB communication.

But why RMB?

Rmb initially was intended to have P2P communication without no single point of failure, but then it requires that all RMBs need to have public IPs in a sense that all remote RMBs need to be reachable from all other RMB instances. Hence the yggdrasil overlay network was used.

As a concept, RMB is fine, but the problem is we rely mainly on a 3rd party overlay network ygg to actually establish the communication. The problem is yggdrasil proved unreliable and randomly introduces downtime and huge latency.

Alternate design

It's obvious without having a public IP for each single entity on the network, a real P2P protocol won't be possible. Hence we have to go back and rely on NATting. In other words instead of waiting for messaging an entity need to initialize and maintain a connection to "chat" or "proxy" service that acts like a central hub for messages. Think telegram or matrix.

The "central" in the name doesn't mean there is a single instance, we hope with proper design we can have a service that can scale both horizontally and vertically.

I already looked into matrix and i totally don't believe it's the right direction although we can use few ideas

Suggesting #1

• A Websocket API (WSS)
  • On intializing a connection, client identity is verified by the means of digital signatures, so only users with valid tfchain accounts can verify and login, hence their twin id is know.
  • Twin ID is the main identity of the user on the messaging bus, hence all messages distend to this user must uses his user ID.
• In the back the API does use the powerful redis streams to queue and deliver messages. The idea goes like this:
  • when a message is received the message is pushed to the destination queue, the queue can has a max length of 10000 message in case client is dead and never coming back (value can be changed later)
  • for each receiver a special consumer group is created, hence a receiver can track what last message has been processed and or received.
  • On message received the client must acknowledged the reception of the message (same as mark message as read on chat services)
    • Failure to mark the message as received will not pop the message from the user queue hence it will be redelivered later
• E2E encryption can be done easily since each twin can also can set his public key on tfchain. so to have a secure communication between 2 peers, each must have his public key set correctly.
• So while client-server communication is already secure with tls (wss) a message can also be either encrypted or in plain text based on if the client have his curve26619 public key set or not. a client can then decide to refuse to answer a plain-text message.
• Scaling of this can happen in 3 dimensions as follows:
  • Redis cluster for redundancy and fault tolerance
  • Multiple instances of the API can work against the same cluster instance for load balancing, since the API itself is stateless
  • home servers (how matrix does it) but we can make regional server
    • europe region can get europe.grid.tf domain.
    • All clients in europe need then to connect to europe.grid.tf but then the domain becomes part of the user id
    • A twin address then becomes <ID>:eruope.grid.tf. When a europe client wants to send a message to another twin in euope the server will do routing to the right queue locally
    • If a message is intended to a twin outside this domain, the message is then forwarded (by the proxy) to the remote domain over a special federation port/protocol.
    • This means other people can install their own proxies as long they are publicly reachable, and then they can still communicate with the global network.
    • Big farms for example GE can have their own proxy server running say proxy.greenedge.com and that is configured as the proxy for that farm. Means that all nodes in this farm will connection to proxy.greenedge.com hence the full address for each node 123 (in this farm) is 123:proxy.greenedge.com.
    • If my twin is 666 and my twin address is 666:europe.grid.tf for example can send a message to 123 by sending the message to MY proxy europe.grid.tf over the open web-socket. then the proxy will federate the message to proxy.greenedge.com, which will hold the message and deliver when possible to 123. Return messages takes the same path.

[muhamadazmy]

Of course this is just an initial idea with no actual technical specs. Details like message format, multi-part messages, encryption algorithms, and files upload support, and other details can be detailed once initial idea is approved.

[robvanmieghem]

An alternative might be waku

[robvanmieghem]

E2E encryption using public/private keypair is pretty resource expensive, a keypair handshake exchanging symmetric keys is preferable especially when exchanging files for example.