pass more advanced objects between processes

[notslang]

The HTML5 web-workers spec specifies a way to pass dates, regexps, buffers, and other advanced datatypes between processes. There's an implementation of that here: https://www.npmjs.org/package/structured-clone

We could also use JASON, but that's a bit more complicated, and isn't a part of a W3C standard.

[rvagg]

Protocol Buffers might be the way to go with this, fast and efficient: https://github.com/mafintosh/protocol-buffers

[notslang]

oh wow... faster than native JSON sterilization? that's pretty insane... definitely worth looking into

[kwiatkk1]

But a drawback of Protocol Buffers is that it requires a schema to work. I think that the BSON from mongodb looks like a better option to use in this particular case as it's schema-less.

[guybedford]

I've been wanting to integrate cbor for data passing here using https://www.npmjs.com/package/cbor (http://hildjj.github.io/node-cbor/), but it seems like passing buffers isn't supported. Perhaps this project could natively use cbor with possible streaming for transport if it turns out faster than JSON encode / decode for bandwidth savings or streaming possibilities? Could be worth some experimenting...

[rvagg]

if #90 gets merged and we get worker thread support then native structured clone will be a thing, considering this stale for now.

[JakeHamix]

I've been wanting to integrate cbor for data passing here using https://www.npmjs.com/package/cbor (http://hildjj.github.io/node-cbor/), but it seems like passing buffers isn't supported. Perhaps this project could natively use cbor with possible streaming for transport if it turns out faster than JSON encode / decode for bandwidth savings or streaming possibilities? Could be worth some experimenting...

@guybedford Actually, it seems that you can pass the buffers just alright right away, no need to implement cbor here - not sure about the other advanced objects. I've just deployed this package as a part of our mailing system, where I pass mail attachments to nodemailer as buffers. Sure, they got malformed a little bit on the way, but parsing it again did the trick.

const nodeMailer = require('nodemailer');

module.exports.sendMail = (mailOptions, transporterOptions, callback) => {

    mailOptions.attachments = mailOptions.attachments.map(item => {
        item.content = Buffer.from(item.content.data);
        return item;
    });

    const transporter = nodeMailer.createTransport(transporterOptions);
    return transporter.sendMail(mailOptions, (error, info) => {
        callback(error || null, info);
    });
};