fix: enhance connection rate limit #196

zmstone · 2024-11-09T11:13:02Z

Prior to this change, if connection rate is limited, the acceptors will enter suspending state and stop accepting the sockets leaving the sockets in the system backlog.

If the acceptor backlog (default=1024) is filled up, for long enough time to cause the majority of the clients to have closed socket from their end and try to reconnect aggressively, the acceptor may never be able to get a normal socket again.

The fix is: in suspending state, accept the sockets and immediately close them to free up the backlog.
The close triggers TCP-RST to cut the TCP graceful close overheads.

Prior to this change, if connection rate is limited, the acceptors will enter suspending state and stop accepting the sockets leaving the sockets in the system backlog. If the acceptor backlog (default=1024) is filled up, for long enough time to cause the majority of the clients to have closed socket from their end and try to reconnect aggressively, the acceptor may never be able to get a normal socket again. The fix is: in suspending state, accept the sockets and immediately cose them to free up the backlog. The close triggers TCP-RST to cut the TCP graceful close overheads.

src/esockd_acceptor.erl

qzhuyan · 2024-11-11T19:50:51Z

src/esockd_acceptor.erl

    case eval_tune_socket_fun(TuneFun, Sock) of
-        {ok, Sock} ->
-            case esockd_connection_sup:start_connection(ConnSup, Sock, UpgradeFuns) of
+        {ok, NewSock} ->


why the socket is changed?

the socket does not change most of the time, but the API is designed to allow it to change.

qzhuyan · 2024-11-11T19:54:37Z

src/esockd_acceptor.erl

                {ok, _Pid} ->
+                    %% Inc accepted stats.
+                    inc_stats(State, accepted),


why move to here? I think the old code looks fine. the socket is indeed accepted isn't it?

there was previously no clear definition what accepted means.
and there was no counters for error cases.

now it means: accepted from the listener backlog, and successfully started an owner process.
and this counter is disjoint from error cases.

src/esockd_acceptor.erl

src/esockd_server.erl

test/esockd_acceptor_SUITE.erl

src/esockd_acceptor.erl

src/esockd_acceptor_sup.erl

src/esockd_acceptor.erl

src/esockd_acceptor_sup.erl

esslaccept error reason has been deleted since OTP 16

when system limit is reached, it does not make sense to continue accepting new connections aggressively

src/esockd_acceptor.erl

zmstone changed the title ~~feat: enhance connection rate limit~~ fix: enhance connection rate limit Nov 9, 2024

zmstone force-pushed the 241109-accept-but-close-when-conn-rate-limit-reached branch from e0fe870 to d3e15b9 Compare November 9, 2024 11:16

ci: fix ci

cf15a53

zmstone force-pushed the 241109-accept-but-close-when-conn-rate-limit-reached branch 3 times, most recently from 2de8851 to 1a86898 Compare November 11, 2024 17:30

zmstone added 4 commits November 11, 2024 19:08

ci: only trigger github actions on pull-request

e704f70

fix: maybe -> 'maybe'

f500702

chore: remove unused dependency

35040c0

zmstone force-pushed the 241109-accept-but-close-when-conn-rate-limit-reached branch from 85445bd to 4f1a472 Compare November 11, 2024 18:09

zmstone added 2 commits November 11, 2024 19:14

fix: dialyzer on otp 27

9510e4c

refactor: delete coveralls

f66c1aa

zmstone force-pushed the 241109-accept-but-close-when-conn-rate-limit-reached branch 2 times, most recently from 910c448 to 0031552 Compare November 11, 2024 18:34