You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Let's say I want to read a zip file. I open an std::fs::File to pass it into zip::ZipArchive::new . Should I wrap my file in a BufReader first?
Later I get a zip::read::ZipFile that I want to read from the archive. ZipFile implements Read. Should I wrap it in BufReader?
The same questions hold for the writing side. You can read the std docs [1] [2] for why generally these adapters make sense. The question is whether the structs in this crate already do buffering internally. In this case adding another layer of buffering is detrimental. The documentation should talk about this.
There is no explicit buffering by the zip crate on the writer side.
The bzip2::write::BzEncoder and flate2::write::DeflateEncoderdo not do any implicit buffering here.
Empirically, for writing with zstd compression I saw great speedups by wrapping the ZipWriter in a BufWrite for each file like so:
Overall I think this question is not that simple to answer and document without doing extensive benchmarks for all supported compression backends.
It's not just a question of reducing file I/O, some compression algorithms might benefit by being given larger chunks even when doing memory-to-memory.
Maybe the zip crate should be itself adding buffering where it makes sense?
Let's say I want to read a zip file. I open an std::fs::File to pass it into zip::ZipArchive::new . Should I wrap my file in a BufReader first?
Later I get a zip::read::ZipFile that I want to read from the archive. ZipFile implements Read. Should I wrap it in BufReader?
The same questions hold for the writing side. You can read the std docs [1] [2] for why generally these adapters make sense. The question is whether the structs in this crate already do buffering internally. In this case adding another layer of buffering is detrimental. The documentation should talk about this.
[1] https://doc.rust-lang.org/std/io/struct.BufReader.html
[2] https://doc.rust-lang.org/std/io/struct.BufWriter.html
The text was updated successfully, but these errors were encountered: