diff --git a/README.md b/README.md index 88fa1f1..95f57c5 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,19 @@ [![Rust](https://github.com/threefoldtech/rmb-rs/actions/workflows/rust.yaml/badge.svg)](https://github.com/threefoldtech/rmb-rs/actions/workflows/rust.yaml) # Reliable Message Bus + Reliable message bus is a secure communication panel that allows `bots` to communicate together in a `chat` like way. It makes it very easy to host a service or a set of functions to be used by anyone, even if your service is running behind NAT. Out of the box RMB provides the following: + - Guarantee authenticity of the messages. You are always sure that the received message is from whoever is pretending to be - End to End encryption - Support for 3rd party hosted relays. Anyone can host a relay and people can use it safely since there is no way messages can be inspected while using e2e. That's similar to `home` servers by `matrix` ## Why + RMB is developed by ThreefoldTech to create a global network of nodes that are available to host capacity. Each node will act like a single bot where you can ask to host your capacity. This enforced a unique set of requirements: + - Communication needed to be reliable - Minimize and completely eliminate message loss - Reduce downtime @@ -18,33 +22,42 @@ RMB is developed by ThreefoldTech to create a global network of nodes that are a - Fast request response time Starting from this we came up with a more detailed requirements: + - User (or rather bots) need their identity maintained by `tfchain` (a blockchain) hence each bot needs an account on tfchain to be able to use `rmb` - Then each message then can be signed by the `bot` keys, hence make it easy to verify the identity of the sender of a message. This is done both ways. - To support federation (using 3rd party relays) we needed to add e2e encryption to make sure messages that are surfing the public internet can't be sniffed - e2e encryption is done by deriving an encryption key from the same identity seed, and share the public key on `tfchain` hence it's available to everyone to use ## Specification + For details about protocol itself please check the [docs](docs/readme.md) ## How to use + There are many ways to use `rmb` because it was built for `bots` and software to communicate. Hence, there is no mobile app for it for example, but instead a set of libraries where you can use to connect to the network, make chitchats with other bots then exit. Or you can keep the connection forever to answer other bots requests if you are providing a service. ### If there is a library in your preferred language + Then you are in luck, follow the library documentations to implement a service bot, or to make requests to other bots. #### known libraries -- Golang [rmb-sdk-go](https://github.com/threefoldtech/rmb-sdk-go) -- Typescript [rmb-sdk-ts](https://github.com/threefoldtech/rmb-sdk-ts) + +- Golang [rmb-sdk-go](https://github.com/threefoldtech/tfgrid-sdk-go/tree/development/rmb-sdk-go) +- Typescript [rmb-sdk-ts](https://github.com/threefoldtech/tfgrid-sdk-ts) ### Well, I am not that lucky + In that case: + - Implement a library in your preferred language - If it's too much to do all the signing, verification, e2e in your language then use `rmb-peer` ## What is rmb-peer + think of `rmb-peer` as a gateway that stands between you and the `relay`. `rmb-peer` uses your mnemonics (your identity secret key) to assume your identity and it connects to the relay on your behalf, it maintains the connection forever and takes care of + - reconnecting if connection was lost - verifying received messages - decrypting received messages @@ -55,9 +68,11 @@ Then it provide a simple (plain-text) api over `redis`. means to send messages ( > More details about the structure of the messages are also in the [docs](docs/readme.md) page ## Download + Please check the latest [releases](https://github.com/threefoldtech/rmb-rs/releases) normally you only need the `rmb-peer` binary, unless you want to host your own relay. ## Build from source + ### Perquisites - download Rustup and install Rust, run the following in your terminal, then follow the on-screen instructions: @@ -93,7 +108,7 @@ This allows you to compile Rust programs that can run on Linux systems that do n - Redis: - Install Redis server using the apt package manager: - + ```bash sudo apt update sudo apt install redis-server @@ -101,35 +116,46 @@ This allows you to compile Rust programs that can run on Linux systems that do n - Configure Redis server by editing the `/etc/redis/redis.conf` file (optional) - Enable Redis server to start automatically on boot: - + ```bash sudo systemctl enable redis-server ``` - Start Redis server using systemd: - + ```bash sudo systemctl start redis-server ``` ### Building + ```bash git clone git@github.com:threefoldtech/rmb-rs.git cd rmb-rs cargo build --release --target=x86_64-unknown-linux-musl ``` + ### Troubleshooting - If you encounter an error like the one below, it is likely that the `protoc` version installed by your package manager is too old. + ```bash --- stderr types.proto: This file contains proto3 optional fields, but --experimental_allow_proto3_optional was not set. codegen failed: parse and typecheck ``` + **Solution**: The best way to ensure that you’re using the latest release of `protoc` is installing from pre-compiled binaries. See perquisites. - + +- A peer must use a unique `mnemonic` or keys. It's not correct if multiple peers uses the same mnemonic this will make it impossible to route messages correctly. It's possible for the same peer to make multiple connections to the same `relay` given that it uses different `session ids`. A session id identify the connection and hence make routing messages possible. +- A single peer on the other hand can make multiple connections to multiple relays for redundancy given that his data on the tfchain must reflect that + +> RUNNING MULTIPLE PEERS WITH THE SAME MNEMONIC MUST BE AVOIDED UNLESS FOLLOWING THE GUIDE LINES ABOVE + ### Running tests + While inside the repository + ```bash cargo test ``` diff --git a/docs/code-guide.md b/docs/code-guide.md index 1095734..e252920 100644 --- a/docs/code-guide.md +++ b/docs/code-guide.md @@ -1,18 +1,24 @@ +# Coding guides + > This document will always be `work in progress` and will get updated every time something comes up -# Coding guides This documents will maintain the coding guide for this project. We will try to keep it up-to-date with what we find more useful and cleaner. ## module structure + in rust there are multiple ways to create a (sub)module in your crate + - `.rs` A single file module. can be imported in `main.rs` or `lib.rs` with the keyword `mod` - `/mod.rs` A directory module. Uses mod.rs as the module `entrypoint` always. Other sub-modules can be created next to mod.rs and can be made available by using the `mod` keyword again in the `mod.rs` file. We will agree to use the 2nd way (directory) but with the following restrictions: + - `mod.rs` will have all `traits` and `concrete types` used by the traits. - `.rs` file next to `mod.rs` that can include implementation for the module trait. z + ### Example + Following is an example of `animal` module. ``` @@ -25,6 +31,7 @@ animal/ > File names are always in `snake_case` but avoid the `_` as much as possible because they basically look ugly in file tree. For example we prefer the name `dog.rs` over `dog_animal.rs` because we already can tell from the module name that it's a `dog` __animal__. Hence in the identity module for example the name `ed25519.rs` is preferred over `ed25519_identity.rs` because that's already inferred from the module name. The `mod.rs` file then can contain + ```rust pub mod dog; pub mod cat; @@ -44,6 +51,7 @@ where ``` The `dog.rs` file then can contain + ```rust use super::{Animal, Food}; @@ -64,10 +72,13 @@ impl Animal for Dog { ``` A user of the module now can do + ``` use animal::dog::{Dog, DogFood}; ``` + For common implementation that are usually used in your modules, a `pub use` can be added in `mod.rs` to make it easier to import your type. For example + ```rust // dog is brought directly from animal crate use animal::Dog; @@ -76,16 +87,19 @@ use animal::cat::Cat; ``` ## naming conventions + following the rust guide lines for name + - `file names` are short snake case. avoid `_` if otherwise name will not be descriptive. Check note about file names above. - `trait`, `struct`, `enum` names are all `CamelCase` - `fn`, `variables` names are snake case -Note, names of functions and variables need to be `descriptive` but **short** at the same time. Also avoid the `_` until absolutely necessary. A variable with a single `word` name is better if it doesn't cause confusion with other variables in the same context. +Note, names of functions and variables need to be `descriptive` but __short__ at the same time. Also avoid the `_` until absolutely necessary. A variable with a single `word` name is better if it doesn't cause confusion with other variables in the same context. The name of the variable should never include the `type`. ## `error` Handling + We agreed to use `anyhow` crate in this project. Please read the docs for [`anyhow`](https://docs.rs/anyhow/1.0.57/anyhow/) To unify the practice by default we import both `Result` and `Context` from `anyhow` @@ -112,6 +126,7 @@ fn might_fail2() -> Result<()> { ``` ## `logs` + logging is important to trace the errors that cannot be propagated and also for debug messages that can help spotting a problem. We always gonna use `log` crate. as ```rust @@ -119,20 +134,25 @@ log::debug!(); // for debug messages log::info!(); // info messages ``` -Note only `errors` that can **NOT** be propagated are logged. +Note only `errors` that can __NOT__ be propagated are logged. > NOTE: All log messages start with lowercase. +> ## function signatures + For function inputs (arguments) `generic` types are preferred if available over concrete types. This most obvious with `string` types. depending on the function behavior ### Examples + This is bad + ```rust fn call1(key: String); fn call2(key: &str); ``` and preferred to use + ```rust // in case function will need to take ownership of the string. fn call1>(k: K); diff --git a/docs/readme.md b/docs/readme.md index 9329986..3f7498f 100644 --- a/docs/readme.md +++ b/docs/readme.md @@ -1,17 +1,21 @@ # RMB -RMB is (reliable message bus) is a set of tools (client and daemon) that aims to abstract inter-process communication between multiple processes running over multiple nodes. + +RMB is (reliable message bus) is a set of protocols and a `relay` server that aims to abstract inter-process communication between multiple processes running over multiple nodes. The point behind using RMB is to allow the clients to not know much about the other process, or where it lives (client doesn't know network addresses, or identity). Unlike HTTP(S) where the caller must know exact address (or dns-name) and endpoints of the calls. Instead RMB requires you to only know about + - Twin ID (numeric ID) of where the service can be found - Command (string) is simply the function to call - The request "body" which is binary blob that is passed to the command as is - implementation of the command need then to interpret this data as intended (out of scope of rmb) Twins are stored on tfchain. hence identity of twins is granted not to be spoofed, or phished. When a twin is created he needs to define 2 things: + - RMB RELAYS - His Elliptic Curve public key (we use secp256k1 (K-256) elliptic curve) Once all twins has their data set correctly on the chain. Any 2 twins can communicate with full end-to-end encryption as follows: + - A twin establish a WS connection to his relays - A twin create an `envelope` as defined by the protobuf schema - Twin fill end all envelope information (more about this later) @@ -25,14 +29,17 @@ Any new messages that is designated to this twin, is pushed over the websocket t This rmb-peer tool makes it possible to run multiple services behind this twin and push replies back to their initiators ## Overview of the operation of RMB relay + ![relay](png/relay.png) ### Connections + The relay can maintain **MULTIPLE** connections per peer given that each connection has a unique **SID** (session id). But for each (twin-id, session-id) combo there can be only one connection. if a new connection with the same (twin-id, session-id) is created, the older connection is dropped. The `rmb-peer` process reserved the `None` sid. It connection with No session id, hence you can only run one `rmb-peer` per `twin` (identity). But the same twin (identity) can make other connection with other rmb clients (for example rmb-sdk-go direct client) to establish more connections with unique session ids. ### Federations + Starting from version 1.1.0, the federation field has been deprecated, and the logic of federation has moved to happen in the relays. Relay is now responsible for determining whether to deliver the message to one of its directly connected clients or to a remote relay. Relay now has an in-memory ranking system to rank its known relays according to their known mean failure rate over a recent configured period of time (time window). @@ -44,12 +51,15 @@ The rank of a relay will heal over time because the system will only consider fa The ranker time window can be configured when starting the relay by specifying the period in seconds after the `--ranker_period` option. If the option is omitted, the default value of one hour will be used. Example: + ``` rmb-relay --substrate wss://tfchain.dev.grid.tf:443 --domain r1.3x0.me --ranker-period 1800 ``` ### Peer + Any language or code that can open `WebSocket` connection to the relay can work as a peer. A peer need to do the following: + - Authenticate with the relay. This is by providing a `JWT` that is signed by the twin key (more on that later) - Handle received binary mesasge - Send binary messages @@ -57,6 +67,7 @@ Any language or code that can open `WebSocket` connection to the relay can work Each message is an object of type `Envelope` serialized as with protobuf. Type definition can be found under `proto/types.proto` ### Peer implementation + This project already have a peer implementation that works as local peer gateway. By running this peer instance it allows you to run multiple services (and clients) behind that gateway and they appear to the world as a single twin. @@ -69,14 +80,18 @@ run multiple services (and clients) behind that gateway and they appear to the w ![peer](png/peer.png) #### `rmb-peer` message types + To make it easy for apps to work behind an `rmb-peer`, we use JSON message for communication between the local process and the rmb-peer. the rmb-peer still maintains a fully binary communication with the relay. A request message is defined as follows + ##### Output requests + This is created by a client who wants to request make a request to a remote service > this message is pushed to `msgbus.system.local` to be picked up by the peer + ```rust #[derive(Serialize, Deserialize, Clone, Debug)] pub struct JsonOutgoingRequest { @@ -104,9 +119,11 @@ pub struct JsonOutgoingRequest { ``` ##### Incoming Response + A response message is defined as follows this is what is received as a response by a client in response to his outgoing request. > this response is what is pushed to `$ret` queue defined by the outgoing request, hence the client need to wait on this queue until the response is received or it times out + ```rust #[derive(Serialize, Deserialize, Clone, Debug)] pub struct JsonError { @@ -134,8 +151,10 @@ pub struct JsonIncomingResponse { ``` ##### Incoming Request + An incoming request is a modified version of the request that is received by a service running behind RMB peer > this request is received on `msgbus.${request.cmd}` (always prefixed with `msgbus`) + ```rust #[derive(Serialize, Deserialize, Clone, Debug)] pub struct JsonIncomingRequest { @@ -164,14 +183,15 @@ pub struct JsonIncomingRequest { Services that receive this needs to make sure their responses `destination` to have the same value as the incoming request `source` - ##### Outgoing Response + A response message is defined as follows this is what is sent as a response by a service in response to an incoming request. Your bot (server) need to make sure to set `destination` to the same value as the incoming request `source` The > this response is what is pushed to `msgbus.system.reply` + ```rust #[derive(Serialize, Deserialize, Clone, Debug)] pub struct JsonOutgoingResponse { @@ -193,6 +213,7 @@ pub struct JsonOutgoingResponse { ``` # End2End Encryption + Relay is totally opaque to the messages. Our implementation of the relay does not poke into messages except for the routing attributes (source, and destinations addresses, and federation information). But since the relay is designed to be hosted by other 3rd parties (hence federation) you should not fully trust the relay or whoever is hosting it. Hence e2e was needed @@ -202,17 +223,17 @@ As you already understand e2e is completely up to the peers to implement, and ev - On start, if the key is not already set on the twin object, the key is updated. - If a peer A is trying to send a message to peer B. but peer B does not has his `pk` set, peer A will send the message in plain-text format (please check the protobuf envelope type for details) - If peer B has public key set, peer A will prefer e2e encryption and will does the following: - - Drive a shared secret point with `ecdh` algorithm, the key is the `sha256` of that point - - `shared = ecdh(A.sk, B.pk)` - - create a 12 bytes random nonce - - encrypt data as `encrypted = aes-gcm.encrypt(shared-key, nonce, plain-data)` - - create cipher as `cipher nonce + encrypted` - - fill `envelope.cipher = cipher` +- Drive a shared secret point with `ecdh` algorithm, the key is the `sha256` of that point +- `shared = ecdh(A.sk, B.pk)` +- create a 12 bytes random nonce +- encrypt data as `encrypted = aes-gcm.encrypt(shared-key, nonce, plain-data)` +- create cipher as `cipher nonce + encrypted` +- fill `envelope.cipher = cipher` - on receiving a message peer B does the same in the opposite direction - - split data and nonce (nonce is always first 12 bytes) - - derive the same shared key - - `shared = ecdh(B.sk, A.pk)` - - `plain-data = aes-gcm.decrypt(shared-key, nonce, encrypted)` +- split data and nonce (nonce is always first 12 bytes) +- derive the same shared key +- `shared = ecdh(B.sk, A.pk)` +- `plain-data = aes-gcm.decrypt(shared-key, nonce, encrypted)` ## Rate Limiting @@ -240,6 +261,7 @@ This way, if a substrate connection failed, other urls are used to try to connec The client uses iterates between urls in a Round Robin fashion, and tries to reconnect. If a specified number of trials is done (currently 2x the number of urls) and none of them was successful, the client fails and returns and error. ## redundancy and failover + Starting from version 1.1.0, RMB has integrated redundancy and failover into the system to achieve high availability. This is done by allowing RMB-peer to set more than one relay domain for a twin on-chain and establish connections with several redundant relays at the same time. diff --git a/src/peer/mod.rs b/src/peer/mod.rs index a708baf..2d22ebd 100644 --- a/src/peer/mod.rs +++ b/src/peer/mod.rs @@ -250,6 +250,8 @@ where Some(bl) => bl, None => { log::warn!("received reply of an expired message"); + log::debug!("envelope {:#?}", envelope); + return Ok(()); } };