Why does a TLS prefork server worker accept more than configured # of client connections and then die for lack of heartbeat?

[fpertl]

Starting prefork server with TLS and then loading it up with request seems to repeatedly kill the worker. I could be wrong but I think the worker is accepting more than the configured # of client connections (only with -c 1) and attempts to service them all which results in a heartbeat timeout. Is this expected behavior or is there something going wrong with the event loop when using TLS?

test.pl

#!/usr/bin/env perl
use Mojolicious::Lite;
app->log->level('debug');
get '/test/' => sub {
    my $c=shift;
    my $sleepSec = $c->req->params->to_hash->{'sleep'};
    print ".";
    select(undef, undef, undef, $sleepSec);
    $c->render(json => {'sleep'=>$sleepSec});
   };
app->start;

test.sh

export MOJO_IOLOOP_DEBUG=1
./test.pl prefork -a 0 -c 1 -G 1 -I 3 -H 10 -i 0 -w 1 -l 'https://*:443?cert=./pki/localhost.crt&key=./pki/localhost.key&ca=./pki/ca.crt'

running the test.sh and then loading it with

ab -c 15 -n 200 https://127.0.0.1/test?sleep=1

results in the following output which seems to indicate the worker connected with all 15 clients rather than one at a time. When using regular http (no TLS) one connection is serviced at a time without heartbeat timeouts.

$ ./test.sh
-- Reactor initialized (Mojo::Reactor::EV)
[2023-06-09 21:44:54.58290] [34] [info] Listening at "https://*:443?cert=.%2Fpki%2Flocalhost.crt&key=.%2Fpki%2Flocalhost.key&ca=.%2Fpki%2Fca.crt"
Web application available at https://127.0.0.1:443
[2023-06-09 21:44:54.58302] [34] [info] Manager 34 started
[2023-06-09 21:44:54.58516] [34] [info] Creating process id file "/tmp/prefork.pid"
[2023-06-09 21:44:54.58516] [35] [info] Worker 35 started
-- e84e9447f68995f9e168549b0e22fc8e >>> 35 (1:0)
-- e84e9447f68995f9e168549b0e22fc8e <<< 35 (0:0)
-- 0e9734cc4cecd31bbce46c244eaec62c >>> 35 (1:0)
-- 30dcfa98aed100e42d37577f539fe455 >>> 35 (2:0)
-- 0e42c7743dd4c245fbada4836cdda359 >>> 35 (3:0)
-- d1a6bd070ce82402cd897b493e638773 >>> 35 (4:0)
-- 3431efb4dcfb5004d71e25da48f44475 >>> 35 (5:0)
-- d61af15d363d9cc729da877c59bb8782 >>> 35 (6:0)
-- e39afb47348b70c63ec487d38cfd9f62 >>> 35 (7:0)
-- 780b3419f6735a049b4267383462479b >>> 35 (8:0)
-- 8297a09a8fa0fe8debd7d4181e4644f6 >>> 35 (9:0)
-- 9e65ed7549c84505b8bd41e034a1fc93 >>> 35 (10:0)
-- daef6fb93c8cc7260eec5e5781e96232 >>> 35 (11:0)
-- 64f5ddcc0cd97c079e16766e2188e01e >>> 35 (12:0)
-- be0f021c081c6b0cd0f4dc91a0a57b3f >>> 35 (13:0)
-- 575bef490ceae7376fa8457a52512eda >>> 35 (14:0)
-- 10d90b630ceb49674e955a0ea78fa9c1 >>> 35 (15:0)
[2023-06-09 21:45:29.36437] [34] [error] Worker 35 has no heartbeat (10 seconds), restarting
[2023-06-09 21:45:29.36450] [34] [info] Stopping worker 35 gracefully (1 seconds)
[2023-06-09 21:45:29.36671] [36] [info] Worker 36 started
[2023-06-09 21:45:30.36797] [34] [warn] Stopping worker 35 immediately
[2023-06-09 21:45:30.37029] [34] [info] Worker 35 stopped
-- fa1aaf2ee98712b1461baa255e98733e >>> 36 (1:0)
-- 4845e58338c0cf3f8b1ff60cd10bbfcd >>> 36 (2:0)
-- 11d21bdced11694cd3ebb8cf543f562f >>> 36 (3:0)
-- d6817785ec62fcb75d5c6f936d41bf32 >>> 36 (4:0)
-- 12190da37ff15804042dd81be6d3161e >>> 36 (5:0)
-- b3252d769213a08e6832d3b5ba245342 >>> 36 (6:0)
-- 9ff681cc49242c840b03f7de1128e997 >>> 36 (7:0)
-- 219b14c333fd5c3c25d38759f9be8043 >>> 36 (8:0)
-- ba798f44fcfd88f895d8930c6f33ed27 >>> 36 (9:0)
-- 99f6e83b659c78fbbed5f41b284c9d39 >>> 36 (10:0)
-- f02ed253daf217e8e9af2627c5f4b50d >>> 36 (11:0)
-- 32061139d403c5aae7472f7de13d1f22 >>> 36 (12:0)
-- c69196aa72a3445bc7c6739f1a523fde >>> 36 (13:0)
-- 3af8076f6bb478f65b4ed23e90393c16 >>> 36 (14:0)
-- 03a014896d6aecce345f453a074d0181 >>> 36 (15:0)
[2023-06-09 21:45:42.38802] [34] [error] Worker 36 has no heartbeat (10 seconds), restarting
[2023-06-09 21:45:42.38816] [34] [info] Stopping worker 36 gracefully (1 seconds)
[2023-06-09 21:45:42.39042] [37] [info] Worker 37 started
[2023-06-09 21:45:43.39262] [34] [warn] Stopping worker 36 immediately
[2023-06-09 21:45:43.39488] [34] [info] Worker 36 stopped
...

$ mojo version
CORE
  Perl        (v5.36.1, linux)
  Mojolicious (9.32, Waffle)

OPTIONAL
  Cpanel::JSON::XS 4.09+   (n/a)
  EV 4.32+                 (4.33)
  IO::Socket::Socks 0.64+  (n/a)
  IO::Socket::SSL 2.009+   (2.083)
  Net::DNS::Native 0.15+   (n/a)
  Role::Tiny 2.000001+     (2.002004)
  Future::AsyncAwait 0.52+ (n/a)

This version is up to date, have fun!

OS version (running in a docker container)

$ cat /etc/*release
3.18.0
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.18.0
PRETTY_NAME="Alpine Linux v3.18"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

[fpertl]

The above behavior of workers getting killed seems to persist when client and server are running in the same instance on WSL using TLS on 3443 with just released Mojolicious 9.33 and using wrk as the client e.g. wrk -d 60s -c 15 -t 15 --timeout 200s http://127.0.0.1:3443/test?sleep=1