[Get By Key] get by key with limits

[PranavVadrevu]

No description provided.

[chrismccord]

Can we get a rundown on the rationale and needs for this feature? The issue with opening up the lookup interfaces is we are rather tied to what we can do inside ets in the future. It may be fine, but it's something we need to be careful with, so I'd like to know more about what specifically you are needing. Thanks!

[PranavVadrevu]

Can we get a rundown on the rationale and needs for this feature? The issue with opening up the lookup interfaces is we are rather tied to what we can do inside ets in the future. It may be fine, but it's something we need to be careful with, so I'd like to know more about what specifically you are needing. Thanks!

Hey @chrismccord, I'm sorry - I meant to push to our custom fork of the repo, didn't mean to get this PR in the main Phoenix Framework, sorry about that 😅

[indrekj]

@PranavVadrevu may be useful for you: #127 We've been using this for a long time now.

[PranavVadrevu]

@PranavVadrevu may be useful for you: #127 We've been using this for a long time now.

Thanks a lot @indrekj, it looks like it should definitely help!

[PranavVadrevu]

Tiny question @indrekj, did you do a performance comparison between dirty_list and your solution? If the main optimization to be made is to remove the Genserver blocking call (which dirty_list also does), wondering if this solution adds more efficiency (if we don't care whether the list is dirty or not)

[indrekj]

I don't remember exactly anymore. It's been almost 5 years :). I think our main goal was to improve performance while keeping the existing guarantees and interface. The GenServer part was the major blocker. An extra call to down_replicas ets which is generally very small is likely a very minor optimization.

[PranavVadrevu]

I see, that's great to know, thank you!

[jaybe78]

Not sure if it's related but I've been experiencing the same issue as @indrekj with Presence.list timing out for channels with less than 70 users tracked, while 50K users are joining at the same time

I just wrote a small test where 50K users join a channel and get subscribed to a specific room, if there's still some slots available (<= 70 slots).
When that room is found, I push Presence.list(room) which times out at around 30K users joining.

Since my channels are pretty small I did not expect that.

#127 (comment)

I suspect that when a large a mount of users join at the same time, Presence is too busy broadcasting diff + handling tracking of new users, so it cannot handle such thing as get_by_key or list within 5000 ms