Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use slice16 as default impl on all width but u128 and document the tradeoffs in the Readme #91

Merged
merged 1 commit into from
Feb 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 78 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,20 +40,90 @@ let crc = Crc::<u16>::new(&CUSTOM_ALG);
let mut digest = crc.digest();
digest.update(b"123456789");
assert_eq!(digest.finalize(), 0xaee7);
```
```

### Lookup table flavors

This crate offers three flavors of lookup tables providing a tradeoff between computation speed and used memory.
See the benchmark section for hints, but do benchmarks on your target hardware to decide if the tradeoff is worth it to you.

1. `NoTable` provides an implementation that uses no additional memory
2. `Bytewise` provides an implementation that uses a lookup table that uses 256 entries of the used width (e.g. for u32 thats 256 * 4 bytes)
3. `Slice16` provides an implementation that uses a lookup table that uses 16 * 256 entries of the used width (e.g. for u32 thats 16 * 256 * 4 bytes)

These can be used by substituting `Crc<uxxx>` with e.g. `Crc<Slice16<uxxx>>`. If you use `Crc<uxxx>` the default implementation is used which are as follows:

* u8 -> Slice16
* u16 -> Slice16
* u32 -> Slice16
* u64 -> Slice16
* u128 -> Bytewise

Note that these tables can bloat your binary size if you precalculate them at compiletime (this happens in `Crc::new`).
Choosing a crate like oncecell or lazystatic to compute them once at runtime may be preferable where binary size is a concern.

## Benchmark

`cargo bench` with 2.6 GHz Intel Core i7. [Comparison](http://create.stephan-brumme.com/crc32/)
`cargo bench` with AMD Ryzen 7 3800X. [Comparison](http://create.stephan-brumme.com/crc32/)

### With no lookup table

```
crc8/nolookup time: [138.00 µs 138.17 µs 138.35 µs]
thrpt: [112.93 MiB/s 113.08 MiB/s 113.22 MiB/s]
crc16/nolookup time: [147.89 µs 148.00 µs 148.12 µs]
thrpt: [105.49 MiB/s 105.57 MiB/s 105.65 MiB/s]
crc32/nolookup time: [140.10 µs 140.37 µs 140.62 µs]
thrpt: [111.11 MiB/s 111.31 MiB/s 111.52 MiB/s]
crc64/nolookup time: [112.02 µs 112.06 µs 112.10 µs]
thrpt: [139.39 MiB/s 139.44 MiB/s 139.49 MiB/s]
crc82/nolookup time: [171.19 µs 171.50 µs 171.84 µs]
thrpt: [90.929 MiB/s 91.109 MiB/s 91.270 MiB/s]
```

### With 256 entry lookup table

```
crc16 time: [2.0082 ms 2.0206 ms 2.0367 ms]
thrpt: [468.25 MiB/s 471.96 MiB/s 474.89 MiB/s]
crc8/bytewise time: [26.670 µs 26.699 µs 26.732 µs]
thrpt: [584.51 MiB/s 585.22 MiB/s 585.87 MiB/s]
crc16/bytewise time: [32.303 µs 32.320 µs 32.338 µs]
thrpt: [483.17 MiB/s 483.45 MiB/s 483.70 MiB/s]
crc32/bytewise time: [30.284 µs 30.309 µs 30.339 µs]
thrpt: [515.02 MiB/s 515.52 MiB/s 515.95 MiB/s]
crc64/bytewise time: [30.218 µs 30.223 µs 30.227 µs]
thrpt: [516.92 MiB/s 517.00 MiB/s 517.07 MiB/s]
crc82/bytewise time: [35.603 µs 35.670 µs 35.724 µs]
thrpt: [437.39 MiB/s 438.05 MiB/s 438.87 MiB/s]
```

## With 16 x 256 entry lookup table

```
crc8/slice16 time: [4.8891 µs 4.9057 µs 4.9250 µs]
thrpt: [3.0982 GiB/s 3.1104 GiB/s 3.1210 GiB/s]
crc16/slice16 time: [4.7201 µs 4.7235 µs 4.7277 µs]
thrpt: [3.2276 GiB/s 3.2304 GiB/s 3.2327 GiB/s]
crc32/slice16 time: [4.6134 µs 4.6217 µs 4.6302 µs]
thrpt: [3.2955 GiB/s 3.3015 GiB/s 3.3075 GiB/s]
crc32 time: [1.7659 ms 1.7793 ms 1.7952 ms]
thrpt: [531.25 MiB/s 535.98 MiB/s 540.05 MiB/s]
crc64/slice16 time: [5.2283 µs 5.2303 µs 5.2324 µs]
thrpt: [2.9162 GiB/s 2.9174 GiB/s 2.9185 GiB/s]
crc64 time: [2.0655 ms 2.0803 ms 2.0973 ms]
thrpt: [454.71 MiB/s 458.43 MiB/s 461.72 MiB/s]
crc82/slice16 time: [25.055 µs 25.061 µs 25.067 µs]
thrpt: [623.32 MiB/s 623.47 MiB/s 623.64 MiB/s]
```

## License
Expand Down
10 changes: 6 additions & 4 deletions src/crc16/default.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
use crate::crc16::{finalize, init, update_bytewise};
use crate::table::crc16_table;
use crate::crc16::{finalize, init};
use crate::table::crc16_table_slice_16;
use crate::{Algorithm, Crc, Digest};

use super::update_slice16;

impl Crc<u16> {
pub const fn new(algorithm: &'static Algorithm<u16>) -> Self {
let table = crc16_table(algorithm.width, algorithm.poly, algorithm.refin);
let table = crc16_table_slice_16(algorithm.width, algorithm.poly, algorithm.refin);
Self { algorithm, table }
}

Expand All @@ -15,7 +17,7 @@ impl Crc<u16> {
}

const fn update(&self, crc: u16, bytes: &[u8]) -> u16 {
update_bytewise(crc, self.algorithm.refin, &self.table, bytes)
update_slice16(crc, self.algorithm.refin, &self.table, bytes)
}

pub const fn digest(&self) -> Digest<u16> {
Expand Down
8 changes: 4 additions & 4 deletions src/crc32/default.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use crate::table::crc32_table;
use crate::table::crc32_table_slice_16;
use crate::{Algorithm, Crc, Digest};

use super::{finalize, init, update_bytewise};
use super::{finalize, init, update_slice16};

impl Crc<u32> {
pub const fn new(algorithm: &'static Algorithm<u32>) -> Self {
let table = crc32_table(algorithm.width, algorithm.poly, algorithm.refin);
let table = crc32_table_slice_16(algorithm.width, algorithm.poly, algorithm.refin);
Self { algorithm, table }
}

Expand All @@ -16,7 +16,7 @@ impl Crc<u32> {
}

const fn update(&self, crc: u32, bytes: &[u8]) -> u32 {
update_bytewise(crc, self.algorithm.refin, &self.table, bytes)
update_slice16(crc, self.algorithm.refin, &self.table, bytes)
}

pub const fn digest(&self) -> Digest<u32> {
Expand Down
8 changes: 4 additions & 4 deletions src/crc64/default.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use crate::table::crc64_table;
use crate::table::crc64_table_slice_16;
use crate::{Algorithm, Crc, Digest};

use super::{finalize, init, update_bytewise};
use super::{finalize, init, update_slice16};

impl Crc<u64> {
pub const fn new(algorithm: &'static Algorithm<u64>) -> Self {
let table = crc64_table(algorithm.width, algorithm.poly, algorithm.refin);
let table = crc64_table_slice_16(algorithm.width, algorithm.poly, algorithm.refin);
Self { algorithm, table }
}

Expand All @@ -16,7 +16,7 @@ impl Crc<u64> {
}

const fn update(&self, crc: u64, bytes: &[u8]) -> u64 {
update_bytewise(crc, self.algorithm.refin, &self.table, bytes)
update_slice16(crc, self.algorithm.refin, &self.table, bytes)
}

pub const fn digest(&self) -> Digest<u64> {
Expand Down
10 changes: 6 additions & 4 deletions src/crc8/default.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
use crate::crc8::{finalize, init, update_bytewise};
use crate::table::crc8_table;
use crate::crc8::{finalize, init};
use crate::table::crc8_table_slice_16;
use crate::{Algorithm, Crc, Digest};

use super::update_slice16;

impl Crc<u8> {
pub const fn new(algorithm: &'static Algorithm<u8>) -> Self {
let table = crc8_table(algorithm.width, algorithm.poly, algorithm.refin);
let table = crc8_table_slice_16(algorithm.width, algorithm.poly, algorithm.refin);
Self { algorithm, table }
}

Expand All @@ -15,7 +17,7 @@ impl Crc<u8> {
}

const fn update(&self, crc: u8, bytes: &[u8]) -> u8 {
update_bytewise(crc, &self.table, bytes)
update_slice16(crc, &self.table, bytes)
}

pub const fn digest(&self) -> Digest<u8> {
Expand Down
28 changes: 24 additions & 4 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
//! digest.update(b"123456789");
//! assert_eq!(digest.finalize(), 0xaee7);
//! ```
//#![no_std]
#![no_std]
#![forbid(unsafe_code)]

pub use crc_catalog::*;
Expand Down Expand Up @@ -79,9 +79,29 @@ pub trait Implementation: private::Sealed {
type Table;
}

impl<W: Width> Implementation for W {
type Width = W;
type Table = [W; 256];
impl Implementation for u8 {
type Width = u8;
type Table = [[u8; 256]; 16];
}

impl Implementation for u16 {
type Width = u16;
type Table = [[u16; 256]; 16];
}

impl Implementation for u32 {
type Width = u32;
type Table = [[u32; 256]; 16];
}

impl Implementation for u64 {
type Width = u64;
type Table = [[u64; 256]; 16];
}

impl Implementation for u128 {
type Width = u128;
type Table = [u128; 256];
}

/// Crc with pluggable implementations ([Nolookup], [Bytewise], [Slice16]).
Expand Down