My experience with riak_core

[JesseStimpson]

Hello, at some point last year, being unaware of the OpenRiak effort, I became interested in seeing whether or not riak_core could be used to build a general purpose crdt library for Elixir. I did make some progress, although not nearly enough to make any releases.

After seeing the HN post today, I figured if anyone out there may be interested, they would be here. 😄 I haven't touched this work in over a year, so there's a lot I've forgotten, but I'll do my best to remember what I did.

My experimental riak_core branch has some aggressive changes in it. IIRC I spent the bulk of my time removing the leveldb dependency. I was pretty ruthless about ripping out code that was inconvenient at the time, so you'll find a lot missing, and I'm sure I broke some important things.

However, I was successful in getting a multinode integration test with a basic in-memory key-value vnode.

Then, I began playing around with an Elixir API on the riak_core CRDTs. My interest in the project fizzled, and I haven't looked at it since.

Here's a trivial example that demonstrates the Elixir API. I admit it's way too pipe heavy, but I hope it gets the point across.

:ok =
  Crdt.for(:incr_api)
  |> Crdt.at(:latest)
  |> Crdt.increment(:like)
  |> Crdt.apply()

{:ok, %{latest: %{like: 1}}} = Crdt.get(:incr_api)

:ok =
  Crdt.for(:incr_api)
  |> Crdt.at(:latest)
  |> Crdt.decrement(:like)
  |> Crdt.apply()

{:ok, %{latest: %{like: 0}}} = Crdt.get(:incr_api)

Anyway, I just wanted to introduce myself since I've been on a team that has had success in using riak at scale in production, and I believe there lives some excellent engineering work inside the riak repos that is begging for a practical modern application.

Please let me know if I can be any help to the project. In particular, is there any small work in riak_core that I can get my feet wet on?

Jesse

[martinsumner]

Hello!

I don't know if you came across riak-core-lite which had similar goals.

There is some work which needs doing on riak_core, but at the moment it isn't high up the priority list.

• There are a couple of PRs which are stalled, partly due to relative importance, but also as we're not sure this is the right approach to achieving their goals.
• There's a general need to strip under-utilised things out, as you've discovered.
• At some stage we want to remove the bespoke connection setup for handoff (it is a legacy of network issues in older OTP versions).
• Also In the long-term project list is further optimisation of coverage planning, to extend the previous work to get sub 1ms planning a larger ring sizes.

Away from riak_core, @adkron was thinking of looking at the Elixir client for Riak, which hasn't had much attention in the past few years, it may be worth connecting with Amos.

CRDTs and riak_dt is another thing that we've struggled to give sufficient attention to in recent years.

Help is welcome. If you're an erlef member there's an open-riak channel in the EEF slack. I'll let everyone there know you're here.

[adkron]

I really need to prioritize some time to set up a stack and do this. I'd also like to work on a gleam driver.