Move to rebar3 [JIRA: CLIENTS-530]

[sdebnath]

Rest of Erlang world is moving to rebar3 to take advantage of the latest developer friendly features. Basho's riak client currently does not build with rebar3 because of protobuffs library (and potentially others). This is a problem for developers and customers as we are stuck with having to continue using older versions of the rest of the dependencies to make sure they can continue to use Basho's software. Which means missing out on bug and security fixes.

Rebar2 is also not actively developed any longer. Rebar3 is actively developed, has more repeatable builds (so they don't randomly break over time), support for packages, is more newcomer-friendly (on-boarding new employees with rebar 2.x is always a poor experience and always ends up in Makefiles to get things to work). There's better support for environment-specific or purpose-specific configuration as well, thanks to profiles.

It's time to get the erlang client up to 2015 :)

[mbbroberg]

FYI @cuyler @zeeshanlakhani @macintux.

@sdebnath - we were talking about this internally and there are some concerns to discuss on this thread. I'd like to understand what branch is best to work on before anyone puts in too much effort.

[danieldreier]

@mjbrender any updates on this? I've got a new project that would be a good candidate to experiment with this, but I'm quite hesitant to lock myself into it when it means using an older version of the tool chain.

[mbbroberg]

@danieldreier the scope is yet to be defined and that's the main concern. I can't offer an idea to getting this done until that is done. Sorry I don't have better news just yet - this code is an intimate part of core product code. // @sdebnath

[cuyler]

I've done some preliminary work on it in the last couple of days. There are some challenges with protobufs that we're ironing out. Not sure how long it will take but I'll drop some notes here as we progress. Thank you for your patience!

[sdebnath]

Appreciate it guys, this will be much welcomed by the community!

[danieldreier]

thanks @cuyler! I appreciate the update.

[cuyler]

Just a quick update here; We're in the process of writing some custom plug-ins for rebar3, due to the complex nature of how riak_pb works. We've run into a few issues with rebar3 itself, including this issue: erlang/rebar3#731

We'll update as we make more progress.

[danieldreier]

I really appreciate the update and the work you're doing on this. I've been holding off on integrating with any of the basho erlang tools primarily due to wanting to use rebar3, so it's very encouraging to see periodic updates.

[cuyler]

OK! More to update. We've completed building a new rebar3 plugin, to help with the generation of protobuf definitions and now we're working on ( what we hope! ) is the final step. A new rebar3 protobuf compiler plugin. Unfortunately, rebar3 does not include a protobuf compiler by default ( as rebar does ) so we have to implement a new plugin to help us along.

We have a new repo here where you can track progress, should you desire. Right now, it's just a skeleton of a plugin, but we're working on porting/reshaping the protobuf code from rebar into it.

Apologies for this taking so long... It seems that this was much more of pandora's box than we realized... All work we needed ( and wanted ) to do, so... onward!

We'll update you again as we get closer to the compiler plugin working. Thanks for your patience!

[mbbroberg]

Hey @cuyler - curious if you could add an update when time permits! Thanks for your work and thanks to @sdebnath for keeping this dependency a priority!

[sata]

Any specific reason for going with Rebar3 instead of erlang.mk for the riak erlang client library?

What are the requirements for riak_pb that forces one to choose Rebar3 over erlang.mk?

I think it's worth understanding what Riak customers are using for build tools in their products before making any assumptions. Not everyone want to adopt Rebar3 nor ever will.