Selecting records with more than one "tag" element

[digitalcora]

The Erlang socket module has a "nowait" mode for some functions which, instead of waiting for something to happen (like receiving data), returns immediately and later sends a message letting the caller know when they should try again. The format of this message is {'$socket', Socket, T, X}, where T is an atom indicating the type of message and X a type-specific payload.

If you want a Gleam selector that selects all such "socket messages", that is easy enough:

let assert Ok(tag) = atom.from_string("$socket")
process.new_selector()
|> process.selecting_record4(tag, fn(a, b, c) { ... })

But I don't think there is currently a way to select only messages related to a specific socket, as it would require treating the first two elements as fixed-value "tags", and selecting_recordX only uses the first element.

I will say, I don't have a concrete use case for this yet — a process handling multiple sockets would probably want to receive messages for all of them, and a process handling a single socket could just panic if the Socket is not what it expects. But it seems like a notable limitation in general, since there is no lower-level API that lets you build arbitrarily-"shaped" selectors. Is that something you've considered? Are there other cases where it would be useful? (selecting_trapped_exits might be one)

[lpil]

It's tricky- adding another clause for the second element would result in a performance hit, and selectors are already slower than Erlang receives. Selectors will be used in the hot path for Gleam OTP applications so performance is important here. Do you have thoughts on how it might be implemented?

More specifically, could you expand upon the use case and how you'd like to use this new selector? There could be another solution for working with sockets.

[digitalcora]

More specifically, could you expand upon the use case and how you'd like to use this new selector?

I will say, I don't have a concrete use case for this yet

So in other words, it's all speculative 😄 I'm not yet sure why exactly one would want to selectively receive messages about specific sockets (other than defensive programming, and that can be done in other ways). It just is a thing you technically can do with the socket module, and I wasn't sure how I would translate the same feature into Gleam if I wanted to, e.g. implementing a selecting_messages_for(Socket) function.

The context is that I'm writing myself a low-level socket FFI layer, which may or may not be released for others to use at some point, but I'm treating it as though it could be. So I'm thinking about things beyond just my own use cases. But, this may not be very helpful for you in determining whether to make some change to gleam_erlang, so I understand if the answer is "come back when you have something more concrete"! I just wanted to raise it for consideration, since it seemed like the kind of thing that could come up elsewhere in Erlang FFI and might be worth having a generic solution for.

As far as the shape of that solution, I'm not really sure. It feels like there should be an "escape hatch" (even if you have to write some Erlang) for when you want to receive some non-Gleam messages with potentially any shape, but still take advantage of Actors and Subjects. Maybe this is possible already and I just haven't figured it out yet.

But, as I said, the use cases are speculative anyway.

[lpil]

It feels like there should be an "escape hatch" (even if you have to write some Erlang) for when you want to receive some non-Gleam messages with potentially any shape, but still take advantage of Actors and Subjects.

I'd love to be able to do this, but I've not worked out how this might be implemented yet unfortunately.

[digitalcora]

Yeah, it seems tricky when there's no way to directly write the receive clause yourself, and the "patterns" of the receive aren't a first-class value. If we had metaprogramming where Gleam code could generate Erlang code, then maybe.

[lpil]

It would be a lot more complex, but we could generate an Erlang module at runtime that has a receive function. It would also make creation of a selector a more expensive operation.

This library is a good reference for how this could be done: https://github.com/rabbitmq/horus/

[digitalcora]

I've come across another case where the current "selector" functions don't do well, namely socket:monitor/1, which monitors a socket and delivers a message when it is closed:

{'DOWN', MonitorRef, socket, Object, Info}

Even if a selecting_record5 existed, it would only allow selecting all messages where the first element is 'DOWN', which would also incorrectly select e.g. process monitor messages (gleam_erlang currently does support those via a special case).

[lpil]

We have a special case specifically for process down messages, we could expand it for sockets also.

        {'DOWN', Ref, Kind, Downed, Reason} when is_map_key({Kind, Ref}, Handlers) ->
            Fn = maps:get({Kind, Ref}, Handlers),
            {ok, Fn(Downed, Reason)};

[digitalcora]

Yeah, though I suppose there would need to be an API for specifying the Kind and Ref directly rather than the layer of abstraction selecting_process_down currently has (not sure how much of the current opaque-ness you'd want to preserve?).

[lpil]

I'd stick with the existing design and have a function for each kind of down message. I don't think there's much in common between ports and processes so we don't need to abstract over them.

[digitalcora]

Hm, so you think this library should have a selecting_socket_down also? What would be the signature for that? The current selecting_process_down accepts a ProcessMonitor which is an opaque wrapper for a ref, so the special case for 'DOWN' messages can match on the specific ref. How would this work for sockets, assuming gleam_erlang doesn't also include the function for monitoring sockets (and the Socket type that would be needed to represent them)? Or would this function just select all socket down messages, instead of those from a specific monitor?

[lpil]

We'll include a socket module with a definition of a socket type and a monitor function for that.

We originally had this for processes and ports, though only the processes one survived the redesign of the Erlang and OTP packages. I should probably bring that back to this package also.

[digitalcora]

Ah, I see. To me it feels a bit odd to include an external type (socket) in the library that no function in the library actually produces; and also as a socket library implementer, that a tiny part of that library (monitors) would live in gleam_erlang instead. Is a generic "selecting down messages" function out of the question? Something like

selecting_down(kind: Atom, monitor: Reference, transform: fn(Dynamic, Dynamic) -> a)

That way gleam_erlang could stay unconcerned with ports/sockets/etc. if it doesn't otherwise need to be. (Though it occurs to me that if this library were concerned with sockets, it could also implement a special case for selecting $socket messages from a specific socket. How many special cases is too many? 😅)

Either way, I would be able to implement this, so let me know if you would like it 🙂

[lpil]

This library is intended to provide type definitions for all of the core Erlang data types, so it would include sockets. Slightly odd, but that's just the nature of Erlang and how embedded networked stuff is into the language itself.

I don't think that function design would be ideal as there's only 3 types of down, but that API suggests there is an unlimited amount. We want to avoid dynamically typed APIs where practical, and I think we can cover 3 happily. We'd need functions to convert the references even with that API so it would produce more functions rather than fewer, and the call sites would have more boilerplate to work with the untyped data.

Do you have a use case for selecting from a specific socket? Generally anything in core should be added in response to real world requirements as we may make incorrect assumptions if predicting usage.

[digitalcora]

Makes sense to me.

Do you have a use case for selecting from a specific socket?

Nothing that's otherwise impossible, so I will save it for when my library is more developed (assuming it gets there) 🙂