Skip to content

Conversation

@ggmichaelgo
Copy link

Types of Changes

  • Bug fix.
  • New feature.
  • Performance improvement.

PR Summary

Instead of using JSON string for communicating between supervisor and workers, use MessagePack instead for better performance and String safety.

Async::Container::Supervisor::MessageWrapper has been newly introduced to this gem.
The MessageWrapper normalizes the message object by invoking the message object's as_json function.

Connection uses the new MessageWrapper and for now I have configured it to have 4GiB. (maximum value from unsigned 32-bit int / may be unsigned 16-bit int with 65KB will be enough 🤔)

Contribution

@samuel-williams-shopify
Copy link
Contributor

samuel-williams-shopify commented Nov 12, 2025

We don't need to add our own framing:

require "msgpack"
require "socket"
require "async"

factory = MessagePack::Factory.new
sockets = Socket.pair(:UNIX, :STREAM)

packer = factory.packer(sockets[0])
unpacker = factory.unpacker(sockets[1])

Async do
	Async do
		while true
			puts unpacker.read
		end
	end

	Async do
		while true
			packer.write("Hello, World #{Time.now}!")
			packer.flush
			sleep rand
		end
	end
end

MessagePack already knows how long each message is internally. (flush is not strictly needed, it's just to ensure the message is sent across the wire at that time - it does not impact framing).

@ggmichaelgo ggmichaelgo marked this pull request as draft November 13, 2025 20:52
@ggmichaelgo ggmichaelgo marked this pull request as ready for review November 14, 2025 17:49
@ggmichaelgo ggmichaelgo marked this pull request as draft November 14, 2025 18:01
@ggmichaelgo ggmichaelgo marked this pull request as ready for review November 14, 2025 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants