Skip to content

Latest commit

 

History

History
84 lines (62 loc) · 4 KB

README.md

File metadata and controls

84 lines (62 loc) · 4 KB
Raw

Apache Parquet Official Native Rust Implementation

crates.io docs.rs

This crate contains the official Native Rust implementation of Apache Parquet, which is part of the Apache Arrow project.

See the API documentation for examples and the full API.

The API documentation for most recent, unreleased code is available here.

Rust Version Compatibility

This crate is tested with the latest stable version of Rust. We do not currently test against other, older versions of the Rust compiler.

Versioning / Releases

The arrow crate follows the SemVer standard defined by Cargo and works well within the Rust crate ecosystem.

However, for historical reasons, this crate uses versions with major numbers greater than 0.x (e.g. 19.0.0), unlike many other crates in the Rust ecosystem which spend extended time releasing versions 0.x to signal planned ongoing API changes. Minor arrow releases contain only compatible changes, while major releases may contain breaking API changes.

Feature Flags

The parquet crate provides the following features which may be enabled in your Cargo.toml:

  • arrow (default) - support for reading / writing arrow arrays to / from parquet
  • async - support async APIs for reading parquet
  • json - support for reading / writing json data to / from parquet
  • brotli (default) - support for parquet using brotli compression
  • flate2 (default) - support for parquet using gzip compression
  • lz4 (default) - support for parquet using lz4 compression
  • zstd (default) - support for parquet using zstd compression
  • snap (default) - support for parquet using snappy compression
  • cli - parquet CLI tools
  • experimental - Experimental APIs which may change, even between minor releases

Parquet Feature Status

  • All encodings supported
  • All compression codecs supported
  • Read support
    • Primitive column value readers
    • Row record reader
    • Arrow record reader
    • Async support (to Arrow)
  • Statistics support
  • Write support
    • Primitive column value writers
    • Row record writer
    • Arrow record writer
    • Async support
  • Predicate pushdown
  • Parquet format 4.0.0 support

Support for wasm32-unknown-unknown target

It's possible to build parquet for the wasm32-unknown-unknown target, however not all the compression features are currently unsupported due to issues with the upstream crates. In particular, the zstd and lz4 features may have compilation issues. See issue #180.

cargo build -p parquet --target wasm32-unknown-unknown --no-default-features --features cli,snap,flate2,brotli

License

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0.