Data Chunking Tests

Windows

Type	Format	Chunk Size	Memory Peak	Time
raw	plain	NONE	283MB	<1s
raw	plain	100,000	116MB	5s
raw	plain	500,000	158MB	2.5s
raw	plain	200,000	144MB	3.5s
raw	plain	50,000
raw	plain	25,000	103MB	18s
raw	plain	15,000	94MB
raw	plain	10,000	92MB
raw	plain	5,000	87MB

macOS

Type	Format	Chunk Size	Memory Peak	Time
raw	plain	NONE	479MB	<1s
raw	plain	50,000	306MB	6.5s
raw	plain	100,000	309MB	3.5s
raw	plain	500,000	431MB	2.3s
raw	plain	25,000	308MB	10s

• Memory start of 273MB

Linux

Type	Format	Chunk Size	Memory Peak	Time
raw	plain	NONE	355MB	<1s
raw	plain	50,000	298MB	9s
raw	plain	100,000	306MB	6s
raw	plain	500,000	307MB	2.8s
raw	plain	25,000	309MB	17s

• Memory start of 286MB