VPP host stack

[unicomp21]

Is there any merit in the idea of using VPP host stack (ie the LD_PRELOAD) to get comparable docker performance to bare-metal seastar when using boost.beast/asio?

https://wiki.fd.io/view/VPP/HostStack

[emaxerrno]

turning this into a discussion

[emaxerrno]

@unicomp21 - good thinking!

there are so many low hanging fruit, and as of today, we are the lowest latency impl, that we are just still working on the low hanging fruit. It would be great to test that idea!!!

[unicomp21]

The reason I ask. If performance of beast websocket running in a container on vpp is fast enough, when compared to sea star, then there is no need for websocket support in sea star. I guess this begs the question of a tcp performance test which can run on seastar and beast?

[emaxerrno]

you are missing the point of thread per core architecture and the memory model. saturating the network is relatively straight forward.

[unicomp21]

If not using kernel bypass, and packets are all MTU, totally agree.

[unicomp21]

What I’m unclear on is mux overhead of 1k tcp sockets, where packets are tiny, when comparing seastar to vpp/beast. From the k8s perspective, I think vpp is interesting here too?

[unicomp21]

I see measurements like this:
http://seastar.io/http-performance/

and it makes me think seastar is the way to go. But, sense I might be missing something.

[vinniefalco]

See: https://stackoverflow.com/questions/44446984/can-boost-asio-be-used-to-build-low-latency-applications

[unicomp21]

https://github.com/boostorg/beast/discussions/2272#discussioncomment-938867

[unicomp21]

@senior7515, trying to get my head straight. Generally speaking, when should one use Seastar vs Beast? What are the trade-offs?

[emaxerrno]

seastar is really good for IO intensive apps. But it requires a new mental model.

1. there is no virtual memory (so you will have to reimplement a bunch of allocation pools, etc)
2. single thread per core
3. explicit message passing - no implicit concurrency

So if you have a problem that can be sharded (by core) and has relatively minimal coordination (or you can change isuch that's the case like we did for redpanda, then is a good fit.

is really onerous as a dev in many ways, so only if lower latency is a big thing for the app.

I just gave a talk bout it here https://vectorized.io/blog/redpanda-talk-at-hydra-2021/

[unicomp21]

But pandaproxy isn't being done w/ seastar, right? Not sure I understand why. I believe seastar supports http?

[unicomp21]

Nice talk!

[BenPope]

But pandaproxy isn't being done w/ seastar, right? Not sure I understand why. I believe seastar supports http?

Pandaproxy is implemented with seastar, and is embedded in the Redpanda binary.

[unicomp21]

Got it. Apologies for the confusion. Now I understand. Beast/asio is only being used for primitives.

[BenPope]

And essentially protocol framing logic.