cancelling a waiter does not free all associated memory until a slot is returned

[problame]

Problem

Assume all slots are in use by submitted operations.

If we now want to perform another tokio-epoll-uring operation, we enqueue in SlotsInnerState::Open::waiters here:

https://github.com/neondatabase/tokio-epoll-uring/blob/9cf400e9086cea7b2c06960d66276a61fea775a4/tokio-epoll-uring/src/system/slots.rs#L419-L427

If we now drop the returned rx, the tx remains stored in waiters until return_slot gets called and we run

https://github.com/neondatabase/tokio-epoll-uring/blob/9cf400e9086cea7b2c06960d66276a61fea775a4/tokio-epoll-uring/src/system/slots.rs#L219-L236

That happens only if one of the submitted operations completes.

In the meantime, the txes pile up in waiters.

Impact

Memory exhaustion and hence OOM kill in pathological cases.

Unlikely, but, could happen if more and more ops are requested and filesystem hangs completely.

The OOM could make it harder to understand the root cause in such cases.

Solution

• Proactive cleanup of the waiters array as part of Drop impl?
• Wrap waiters in a semaphore?
• Follow long-standing TODO to replace waiters with a bounded channel (cap=SLOT_SIZE) that in idle state stores all the slot indices?

Context

I found this bug while

• investigate: slot leak? #57