Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for message delimiting #13

Open
ghost opened this issue Oct 3, 2016 · 2 comments
Open

Add support for message delimiting #13

ghost opened this issue Oct 3, 2016 · 2 comments

Comments

@ghost
Copy link

ghost commented Oct 3, 2016

Google's Java implementation of protobuf provides the writeDelimitedTo/parseDelimitedFrom methods, basically just prefixing individual messages with their length and thus allowing multiple messages to be combined in one output stream.

I've been trying to cobble this together myself, but I think it would be nice to have as part of the library as well. This would make API creation using your library much easier, since you might wanna support requests querying multiple resources/messages of one kind.

If you're not likely to implement this, could you maybe point me in the right direction for doing this myself? It's no problem for small messages (where the length fits into 8 bytes)

pack("c*", strlen($message->toStream()));

but for larger messages, packing with v* or V* produces errors on the client's side, because their length needs to be converted to a varint.

@Mararok
Copy link

Mararok commented Oct 4, 2016

Hey you don`t need do 'strlen' on Stream. You can use Stream::getSize() and prefix message with that value.
If your message is very tiny you need only 1 or 2 bytes for encode size.

Write:

$stream = $message->toStream();
$packedSize = pack('n', $stream->getSize());
$data = $packedSize.$stream->getContents();
fwrite($file, $data);

Read:

$packedSize = fread($file, 2);
$size = unpack('n' $packedSize)[1];
$messageData = fread($file, $size);
$message = new YourMessage(Stream::fromString($messageData));

And its good idea to add that functionality to lib.

@FabioBatSilva
Copy link
Member

You can probably use YourMessage#writeTo() and YourMessage#serializedSize() to write multiple messages to the same Stream.

Implementing writeDelimitedTo/parseDelimitedFrom would require changes to protobuf-php/protobuf-plugin to generate the new methods or if possible implementing it in AbstractMessage

But given a good PR I would be happy to reconsider it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants