writing a library that can be reused across many runtimes

[nikomatsakis]

• Character: Barbara
• Brief summary: Barbara tries to write SLOW in a way that it can be used across runtimes; she tries various approaches, none of which are fully satisfactory
• Key points or morals (if known):
  • Writing a library that is generic across libraries is often possible but difficult
  • Feature flags is one option, traits are another
  • Wants to find solutions that are zero-cost
  • Most common features needed are async-read, async-write traits, timers, spawning, opening UDP/TCP sockets
• Conversations:
  • recent Zulip conversation

[AldaronLau]

@nikomatsakis Though it's not necessarily "zero-cost", I think the "crossterm" crate with async enabled has an okay approach that should be mentioned here. It essentially starts a background thread (the not zero-cost part) that sleeps most of the time, which lets the futures know when they're ready. After the thread has started, I believe the costs are relatively low, and acceptable for most projects.

[nikomatsakis]

@AldaronLau that's great, thanks! Another approach worth citing (though it has limitations) is the agnostik crate. From what I can tell, it supports spawning tasks primarily. It is a facade model, where feature flags are used to select which runtime it will use, and the dispatch is done via impl Trait.

I don't know if there are people with real-life experience using the crate to talk about.

Some of the pros I can see theoretically are:

• Code against a single interface
• Lets application crates pick the executor they want without any kind of dynamic dispatch

Some of the cons I can see are:

• Doesn't describe timers, nor the ability to open sockets, I haven't looked at the interfaces to know how hard that would be to include
• Effectively imposes a kind of "global runtime". So if I as a user code my library against the agnostik crate, people can retarget it by specifying feature flags in their application crate, but if they want to use my library with both tokio and async-std, or with some runtime that is not supported by agnostik, they cannot.
• Your clients have to know to specify the feature flag in their Cargo.toml if they care. This doesn't seem so bad.

[AldaronLau]

@nikomatsakis cool, I didn't know about that crate! Now I'm wondering if similar to agnostik, there could be something like std::alloc::GlobalAlloc, called something like std::executor::GlobalExecutor.

The docs say it:

can be used with every executor provided by agnostik, or you can integrate your own executor with Agnostik.

which makes me think it's already pretty close to working like std::alloc::GlobalAlloc.

[ibraheemdev]

There have been proposals for a global_executor in the past:

• https://without.boats/blog/global-executors/
• https://internals.rust-lang.org/t/global-executors/11295

There has also been discussion about an extension to Context that would allow futures to interact with their environment:

• Custom Future contexts rfcs#2900 (comment)
• RFC: Generic member access for dyn Error trait objects rfcs#2895 (comment)

Both of these would make writing runtime-agonstic libraries much easier, but there haven't been any conrete proposals that I know of recently.

[nikomatsakis]

Collecting links is good, but I'd like to focus for now on the challenges -- what is difficult now (and what works well now)? I'd like to hold off on exploring solutions because that quickly gets into the nitty gritty.

One thing that could be really useful though is to look through for the counterarguments to a global allocator and make sure we spell those out. We want to be describing the constraints and considerations for whatever solution we wind up with. (My personal opinion is that a global allocator may be part of that, but I'm not sure, and we will want to be careful whatever we do that we leave room for services that have multiple runtimes interacting.)

[dlight]

The story I would like to see for async Rust is not just to be possible to write cross-runtime libraries, but for it to be the easiest, preferred route: if everybody has access to APIs that abstracts away most differences between runtimes, people will have little reason to directly or indirectly depend on tokio if they are writing a library. It adds more time to the build, for one.

There's another, more philosophical reason to dislike special-casing executors in libraries: perhaps the one we will end up using isn't written yet. So even a library like agnostik that abstracts away N executors (currently, N = 4) isn't enough: I hope we write forward-compatible libraries, so they will stay fresh as the ecosystem evolves.

[erickt]

For a real-world example - On Fuchsia, we support a single executor and runtime fuchsia_async library. However, in order to more easily share libraries across host and target, we've partially implemented a compatibility layer to fuchsia-async that's built upon async-std. Unfortunately, some of the decisions we made in our API makes it difficult to share across runtimes.

The specific problem I ran into was trying to make a portable version of fuchsia_async::net::fuchsia::tcp. One of the APIs to get a stream of accepted sockets is fn TcpListener::accept_stream(self) -> Acceptor. Unfortunately async-std made a different choice, and uses fn TcpListener::incoming(&self) -> Incoming`.

I could probably change fuchsia-async to match async_std's interface, but it looks like tokio's equivalent accept stream takes ownership over the listener with it's TcpListenerStream::new(TcpListener), so there really isn't a standard API in the ecosystem.

[Stupremee]

Hey 👋 ,
I'm the maintainer of the agnostik crate and just stumbled across this thread.
The crate was originally developed to only allow agnostic spawning of tasks, however, we never really started using it which
is the reason it got kinda stuck and really inactive. It's also in general not really well designed could definitely be overhauled.

I would love to rewrite it, also including abstractions for other stuff like sockets, if there's interest. It would probably make a good "playground" for testing executor agnostic ideas. However, I was, and still am, stuck on a good design to have an agnostic runtime. Global spawn (and friends) methods are really helpful, but then you'd only be able to have one active runtime.
Passing a Runtime struct everywhere would also be really annoying to use I think.

So I'm open for any suggestions on how to make the User API and design better.

[nikomatsakis]

@Stupremee thanks! I think that brainstorming what such designs might look like is exactly what I hope to do during the "shiny future" period that is coming up. Maybe we can schedule a time to talk about this.

[Stupremee]

That sounds good. You can ping me when the time has come.

[nikomatsakis]

Reproducing this comment by @erickt from another issue:

To add on to this - sometimes it's impossible to use these libraries. For example, on Fuchsia we haven't implemented support for tokio, async-std, and etc. We'd love to someday support them, but we haven't stabilized a lot of the interfaces executors need, so we'd end up needing to impose a lot of upstream churn.

So instead, we've been trying to leverage "executor-less" flags (like https://github.com/hyperium/hyper, where we plug into it with https://cs.opensource.google/fuchsia/fuchsia/+/master:src/lib/fuchsia-hyper/). This works for us, but it can also be difficult for upstream projects to be pluggable like this. So for hyper, we ended up having to reimplement a lot of https://github.com/hyperium/hyper/tree/master/src/client because they didn't have a mechanism for abstract dns resolution.

[rylev]

We did a writing session on this and came up with an almost complete outline of a story for this. We will likely meet next week to finish this outline and turn it into an actual story.

[fpagliughi]

Thanks for this topic. Just wanted to add a +1 on the need for guidance and best practices for writing async libs that don't need full I/O, but just some basic functionality. I'm on my 3rd library that just needs:

• Spawning a few tasks on a simple executor
• Async channels for communicating between them
• Futures timers/timeouts

It just seems a bit more difficult and confusing than it needs to be for this fairly minimal use case.

[rylev]

FYI: we met again, and finished the outline. @zeenix is working on turning the outline into a full story.

[realcr]

I think that a major obstacle for writing a runtime agnostic library is that runtimes currently do not have traits that represent them. We do have traits for Futures, and so it seems like all libraries agree about what a Future is.

As far as I know runtimes only have a trait for spawning, and from my experience even that trait is not being respected by most libraries. Without a runtime standard, rust async runtimes slowly diverge, introducing their own incompatible async channels and incompatible traits. Hence writing a runtime-agnostic library becomes more and more difficult.

Issues with embedding runtimes in libraries

I see a few issues with the current way things work:

1. Fragmentation. Every library has to choose its camp of runtimes, and it becomes somewhat difficult to have a reasonable application that uses a few different libraries from different runtime camps. From my experience this also makes it very difficult for newcomers to understand how to approach async Rust. Very often I see questions like: "How do I do async in Rust? It Tokio Rust's async library? What is async-std then?".

2. Inefficiency: If a library includes a runtime inside of it, it forces me to use this runtime. If for example I want to use two different libraries that use different runtimes, I am forced to have the code of both runtimes inside my final binary.

3. Testing: A library that depends on internal runtime becomes opaque for testing. This requires some explanation. Async code often manages some kind of a state machine that is more complex than the classic send-block-receive. Such code will usually require careful testing. To prove that a certain state machine is correct in all cases, one needs to have delicate control over the passage of states. When a library uses its own "sleep" method, redirected to some internal runtime, it becomes impossible for me to test the passage of time in a deterministic way. Another example is when a library uses its own "spawn" method. It then becomes impossible for me to track the creation of new tasks in a deterministic way.

Async in Offset

My main experience with async was working on Offset, a decentralized protocol (library)

One of the things that was a huge productivity improvement for me is the ability to create my own test executor, and use it to do things like having detailed control over the passage of time. Think about how to test a timer that shoots only once in 10 minutes. Instead of writing a test that waits for 10 minutes, I prefer to have some system that simulates the passage of time.

Another thing that my test executor provided me with is a fully deterministic execution with my own executor's task management. This kind of fine grained control also allows to detect async deadlocks and reproduce them in a deterministic manner. In the world of opaque inner runtimes, detecting such bugs is virtually impossible.

One thing I discovered is that if I chose to use any runtime depended library, it would have infected my whole library, making testing using my test executor impossible. I think that the same is true for the case of having a global executor.

How to be a runtime agnostic library?

I developed a few async projects recently, some of which are libraries (Offset). All of my libraries are 100% runtime agnostic, and my applications are runtime agnostic up to the code of the main() function. Here are some patterns that are shared between my projects:

• Only use future's runtime agnostic channels.
• Represent time by ticks flowing through a stream (possibly a runtime agnostic channel). This stream will be given to the main function of the library, provided by a runtime chosen by the user. Use the incoming stream of ticks the only source of time.
• Use futures::task::Spawn trait to spawn new tasks. Take a spawner: impl Spawn argument explicitly on all methods or constructors that might require spawning of tasks.
• If relying on an extra internal executor (For example, to spawn blocking sqlite3 tasks) always provide the user the ability to provide the executor himself. This way the user will be able to test the whole setup deterministically, possibly with a single executor, and be able to detect deadlocks. Detecting deadlocks requires having control over all the executors/runtimes in use.
• If any network communication is being used, represent it as runtime agnostic channels / future's Streams / future's Sinks, and take those as arguments from the user (Or possibly as glue logic code that will be written by the user).
• File operations should be done in a synchronous manner, wrapped in a blocking task that will be spawned in a separate, user chosen executor.

For me it is always disheartening to see a good library having a path like library_name::runtime::tokio::spawn shows up in its documentation. It feels like cancer to me. Once you depend on this library, you have to do the same, and you lose all the benefits of being runtime agnostic.

When having to use runtime-dependent dependencies, much of my development time is dedicated to doing the acrobatics of integrating the dependency as late as possible in the development (As close as possible to main()), to keep maintaining my testing freedoms as long as I can.