Tricks to reduce resource usage in a development kubernetes cluster

[fcbry]

I'm looking to demonstrate the benefits of redpanda vs kafka in our development enviroment. Long story short, we have hundreds of self managed namespaces, each of which has got a kafka instance, and each of which is doing absolutely nothing 99% of the time, and instance would be lucky to process a hundred messages per-day.

I wonder if there's a way to configure seastar to request less startup memory, loosen all the safety stuff, and just run it with the minimal required to provide read/write Kafka API.

So speed/latency is less important than low CPU/memory usage in this case.

Perhaps someone has been here before and can provide some sweet, sweet, tips-n-tricks?

[fcbry]

Maybe this is even a product request - :micro-panda: for memory contstrained enviroments. Kafka has a thing called Kafka embedded where you can host an instance of Kafka in your (Java) application, but that sounds clunky.

[rockwotj]

If you run Redpanda with --help-seastar you can see a bunch of options related to resources. Specifically I'd suggest running with --smp=1 to run redpanda on a single CPU core and --memory=1G to specify the memory usage. Note that giving Redpanda a limited amount of memory restricts how many partitions you can create, but I often run Redpanda with a single core and one GiB of RAM for testing. Let me know if that doesn't work - I'm happy to help further.