feat(rpc): pipelined aggregator setup

[doscortados]

Closes #460

[mmtftr]

Thanks a lot for the changes.

I believe the original issue also wants the try_join_alls to happen in background tasks so that they're happening in parallel (you can check aggregator for details)

We're also going to be working on moving the business logic out of the gRPC handlers, and moving parsing/validation logic out into trait impls so that they're more easily tested and the code has less clutter.

[doscortados]

Thank you for the comments @mmtftr , and sorry for my late reply!
This is rather initial version and significant re-work will follow in further commits.

At this point I see that mpsc channel isn't working out for multiplexing N-over-M streams, and I am trying out mpmc one to avoid pulling all the params in memory before streaming it to consumers.

[ceyhunsen]

Overall, it looks good. But apparently there is a deadlock or something like that: CI/CD tests are not finishing.

[doscortados]

Thank you for your comments @mmtftr @ceyhunsen , I will address them once I get the tests pass.
Right now I figured out a "quick & dirty" way to avoid deadlock when broadcasting operator/watchtower params, but now when deadlock is no more, two tests still fail.

[doscortados]

At this time the deadlock and the failing tests are fixed, I will make one more round of cleanups and mark it ready for the review.

[ceyhunsen]

It looks unnecessarily complex to me in general tbh

[doscortados]

It looks unnecessarily complex to me in general tbh

I totally agree @ceyhunsen ! But it also is non-trivial multiplexing M-over-N streams in a concurrent fashion. Also it is necessary to re-stream from broadcast consumer to mpsc consumer due to limitation of the client API.

Compared with existing non-concurrent implementation the only overhead is actually only two extra broadcast channels (one for setting operator params, another one for watchtowers).

UPDATE

If you know how to simplify it, I am open to suggestions by all means!

We can probably also approach the verifier cliet API to accept broadcast channel into-stream wrappers, but I didn't want to go there in scope of this PR just to avoid bloating it.

[mmtftr]

Thanks a lot for the refactor, just some small comments