Merge the read and write buffers

[anarthal]

All operations work in a half-duplex fashion, so they don't need separate buffers. Pipelines have their own buffer, so they don't need this, either. This can reduce memory consumption and make #279 easier.