[erts][socket] Add {otp, scheduler_polling} option

[potatosalad]

Building on top of #10323 (as it relates to #10322), this adds a new option to socket:

% disables scheduler pollset migration for this socket
ok = socket:setopt(Socket, {otp, scheduler_polling}, false).

% enables scheduler pollset migration for this socket
ok = socket:setopt(Socket, {otp, scheduler_polling}, true).

This also adds a new field to the struct returned by enif_system_info() named scheduler_polling_support so that prim_socket_nif.c doesn't need to depend directly on erl_poll.h.

[github-actions]

CT Test Results

    4 files    196 suites   1h 53m 59s ⏱️
3 223 tests 2 825 ✅ 397 💤 1 ❌
4 119 runs  3 643 ✅ 475 💤 1 ❌

For more details on these failures, see this check.

Results for commit 39c0f96.

♻️ This comment has been updated with latest results.

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

• Complete CT logs (Download Logs)
• HTML Documentation (Download HTML Docs)
• Windows Installer

// Erlang/OTP Github Action Bot

[michalmuskala]

If we already have a config, I wonder if it would make more sense to make this config the count, rather than true/false with an "arbitrary" count of 10

[nickva]

How would users know when to enable this option?

Some docs or guidance for it could help, something like, "if you have these many sockets of this type, you might benefit from it" or "if you enable this you'll increase your latency but improve throughput" etc.

Maybe there is a way to automatically set this behind the scenes, based on some heuristic (socket is accepting vs receiving moslty)?

[potatosalad]

How would users know when to enable this option?

Some docs or guidance for it could help, something like, "if you have these many sockets of this type, you might benefit from it" or "if you enable this you'll increase your latency but improve throughput" etc.

Maybe there is way to automatically set this behind the scenes, based on some heuristic (socket is accepting vs receiving moslty)?

I think this option only made sense at the time due to my then misunderstanding of how scheduler pollset worked (specifically: I thought there was a pollset per scheduler, not a singular global scheduler pollset).

Thanks to @sverker's clarification, I don't think this particular option actually makes sense on its own.

As in:

1. If you need low-latency at the cost of more syscalls (higher CPU), then just disable scheduler pollset globally with erl +IOs false
2. If you need lower CPU or less syscalls at the cost of higher-latency, then leave scheduler pollset enabled globally.

[nickva]

1. If you need low-latency at the cost of more syscalls (higher CPU), then just disable scheduler pollset globally with erl +IOs false
2. If you need lower CPU or less syscalls at the cost of higher-latency, then leave scheduler pollset enabled globally.

Thanks for explaining, Andrew. That makes sense.