Making the broker configurable

[matthew-healy]

Hi folks! I'm working on a project which is using rails_event_store for event persistence & publishing. However rather than sending & receiving events in-process, our dispatched events first make a round trip through an external queue (in order to, e.g., subscribe to events in other services).

To do this we have to prevent the RubyEventStore::Broker from dispatching events directly to subscribers. We've achieved this currently by implementing our own Broker and replacing the RailsEventStore::Client's @broker instance variable, however this is naturally brittle as the broker interface is an internal detail of the library & its configuration is not officially supported.

To resolve this, I was wondering if you would be open to making the RailsEventStore::Client's (and thus also the RubyEventStore::Client's) broker configurable via the initializer? I appreciate this is a slightly niche use-case, so if that's not something you want to support I totally understand, but I figured it couldn't hurt to ask. 😁

[mostlyobvious]

Thank you for reporting this. I agree than current Broker implementation is a bit too tied to in-process event handling, which at the moment prevents dispatching without any subscriber. I believe this the main problem.

Making Broker interchangeable is one thing, ensuring that passing dispatcher and subscriptions still makes sense is another.

I'm not sure yet what is the best long-term solution. Happy to see your proposal.

[mostlyobvious]

One workaround, until the issue is addressed, is to leverage subscribe_to_all_events.

Either by having an empty subscriber of ->{} to fool Broker and make it route all published events through dispatcher. This has the advantage of dispatcher having access to both event object and serialized record, more suitable to pass over the network.

Or having the dispatch-to-external-queue there in all-events-subscriber. With the drawback of acting only on event objects as subscriptions do.

[matthew-healy]

Thank you so much for replying, and apologies for the long delay in getting back to you.

I think your suggestion of using subscribe_to_all_events makes a lot of sense. I'm also now thinking that another way around this would be for us to run two event stores side-by-side: one which has a subscription to all events, persists them & forwards them to the external queue, and a second which has all of our "actual" subscribers, doesn't persist anything, and routes events from the external queue.

I think that'd work for us without requiring us to touch the broker at all, though I haven't actually had the time to try it out yet.