Generic block record readers and performance improvements
·
87 commits
to master
since this release
This release includes new Generic block record readers for Lzo compressed protobuf data. It also contains a change to make minimum indexable file size configurable for Lzo output and performance improvements for reading Lzo indexes and splits.
Note: BinaryConverter now throws DecodeException if deserializing a record fails, instead of returning null.
Change log:
Issue 440. LzoGenericBlockOutputFormat (Ruban Monu)
Issue 439. Adds generic block record readers (Ruban Monu)
Issue 435. Faster working with LzoBinary data (Ian O'Connell)
Issue 434. Speed up getSplits by reusing FileStatus'es from the very first listStatus (Gera Shegalov)
Issue 430. Configurable minimum indexable file size (Gera Shegalov)