Skip to content

go-btfs-chunker provides Splitter implementations for data before being ingested to BTFS

License

Notifications You must be signed in to change notification settings

bittorrent/go-btfs-chunker

 
 

Repository files navigation

go-btfs-chunker

go-btfs-chunker implements data Splitters for go-btfs.

go-btfs-chunker provides the Splitter interface. BTFS splitters read data from a reader an create "chunks". These chunks are used to build the BTFS DAGs (Merkle Tree) and are the base unit to obtain the sums that BTFS uses to address content.

The package provides a SizeSplitter which creates chunks of equal size and it is used by default in most cases, and a rabin fingerprint chunker. This chunker will attempt to split data in a way that the resulting blocks are the same when the data has repetitive patterns, thus optimizing the resulting DAGs.

Table of Contents

Install

go-btfs-chunker works like a regular Go module:

> go get github.com/bittorrent/go-btfs-chunker

Usage

import "github.com/bittorrent/go-btfs-chunker"

Contribute

PRs accepted.

Small note: If editing the README, please conform to the standard-readme specification.

License

MIT © Bittorrent.

About

go-btfs-chunker provides Splitter implementations for data before being ingested to BTFS

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 100.0%