Feature Proposal: Secure Inter-Node Communication

[martinsumner]

Background

Inter-node communication, with the exception of handoffs, uses the standard Erlang TCP Distribution in Riak. this means that anything passed between nodes is un-encrypted and inter-node communication is unsecured.

To provide necessary, Riak users generally either secure the environment at another layer (i.e. use private environments, AWS nitro or network-based VPNs), or use their own scripts to run Riak with Erlang set to use the Erlang Distribution over TLS.

Going forward, it may be preferable to use Erlang distribution more for communications (e.g. handoffs) that previously bypassed the internal communications due to historic problems with the distribution protocol.

It would be preferable, to have a clearly documented and tested process for TLS-enablement of all Riak's intra-cluster communication.

Proposal

Add to the riak.schema cuttlefish options to:

• change the distribution protocol;
• add certificate authority, certificate and private key files.

Write a basic getting started guide to cover use.

Design

n/a

Alternative Design Ideas

It may have been possible to provide security at a different level, for example:

• a new riak_object:to_binary format that would encrypt the object;
• add security in the riak_core_send_msg module, using credentials distributed via the ring.

Any such option would be too likely have holes or weaknesses that may not be expected by the end user.

Some thought was given to try and automate the setup of TLS in Erlang further, with some built in generation of certificates as part of the join process. However, other erlang-based products with a much broader customer-base take the simple approach - so that should guide us that this answer is sufficient.

Testing

To what extent riak_test needs to be modified to test with TLS enabled is an open question.

Caveats

As part of this, some consolidation may be required on the configuration of SSL credentials. The handoff protocol should either be refactored to use Erlang Distribution, or the configuration combined. nextgenrepl (which uses standard Riak client -> API for all communications) and riak_repl (which manages its own connections, security) - also have their own configuration requirements.

Pull Requests

An early draft was done some years ago - basho/riak_test#1342.

Planned Release for Inclusion

tba