Callback streams for use as library

[bakaq]

This adds a new StreamConfig option that allows both to asynchronously react to the Machine writing to stdout or stderr, and to programatically send data to its stdin, all in-memory.

This is an incredible feature for many reasons, like being able to get stdout in parts and not just all at once, but the application I have in mind is running the same toplevel we use for CLI inside Wasm in the Scryer Playground, which will be a great improvement in usability!

[Skgland]

Rather than having one value in StreamConfig that configures stdin/stdout/stderr at once would it maybe make sense to have individual configs per stream like what I currently have in 6d0ddc1 as part of #2786.

Also (unrelated to this PR) having in_memory configure the stdin to be a null stream, stdout to be buffered in memory and stderr to still be stderr is a bit confusing.

[bakaq]

I guess it also makes sense to make OutputStreamConfig and InputStreamConfig public and allow some builder style configuration in StreamConfig (like with_stdin, with_stdout and with_stderr methods, although I think these particular names are confusing), and also fix the issue you pointed out with in_memory, but I'm not sure this should happen in this PR.

[jjtolton]

👀 what's the high level of this? configurable input/output/error streams? If so, I'm all for it! If not... probably still all for it, just not sure what I'm all for yet!

[bakaq]

Yes, this lets you handle stdin, stdout and stderr programatically, instead of it leaking to the real stdin, stdout or stderr.

[jjtolton]

Yes, this lets you handle stdin, stdout and stderr programatically, instead of it leaking to the real stdin, stdout or stderr.

I assume this does not apply to shell/N? different PR, probably?

[bakaq]

Yeah, shell/N would be Scryer controlling another process. This PR is concerned with the embedding API, and specifically with being able to control Scryer itself, so almost the exact opposite.

[bakaq]

I removed the way to make input null streams too. Now the default is an empty memory buffer. I also made some more tests, and in the process added support for converting streams to Terms, something that was missing from the work on #2475. I believe null streams should eventually be changed back to the default as soon as they have sane behavior (at least not crashing or panicking on use).

Something funny I noticed is that write/1 flushes the output but nl/0 doesn't. I would expect the exact opposite.