Implement individual clip streaming support

[nfrechette]

The current database streaming works in bulk. All clips within stream together at the desired quality level. This can make it quite hard to have fined grained control over quality. It forces runtimes to group few clips together and have many databases. In turn, this harms the ability of the algorithm to globally optimize for quality when targeting a specific memory footprint.

Streaming has been implemented in bulk to optimize for older disk based hardware where seeks and reads are expensive and must thus be minimized. However, with flash memory on mobile and SSD/NVMe on PC and consoles, seeks and reads are dramatically cheaper. As such, it makes sense to extend the ability to stream individual clips within a database for those platforms.

Bulk streaming is nice because it requires little metadata. All chunks are read from disk linearly and they contain the necessary metadata. Streaming individual clips will require extra metadata. We need to know where in the bulk buffer the clip lives for each quality level. This metadata will need to be present in memory with the rest of the database runtime metadata.

Streaming a clip would then first look up which offset to read from disk for the desired quality level. A read from disk would be performed, and we'd update the runtime metadata.

Each quality level would perform an individual seek and read. We can later optimize this by swizzling the data such that data for all quality levels of a clip live contiguously on disk. This will allow us to perform a single read for all quality levels. Optimizing the data in this way will make it much slower to stream/unstream the whole database at a specified quality level.

Streaming out no longer makes sense if we manipulate individual clips. Streaming out a single clip would not allow us to reduce the memory footprint as we can't allocate memory for individual clips.

A different paradigm is necessary when managing memory at the clip level. We will need to allocate memory in chunks of a fixed size and clips will populate them in the order we stream them in. Because our runtime metadata stores 32bit offsets, we need to be careful where things live and how we manage it. We could reserve virtual memory up front in a single buffer large enough to accommodate multiple times the size of the database to account for fragmentation. When we need to stream out, we'd mark the allocation as free in our chunk. Later, a garbage compaction phase would kick off and reclaim unused space by compacting things into existing/new chunks. This can be done entirely async and in a thread safe way since we atomically update our offsets. Reserving virtual memory could also be avoided if we split out 32bit offset into two parts: a chunk index (8bit) and an offset into the chunk (24 bit). This would allow us to have non-contiguous chunks. If we do this, we need a heuristic to figure out how many chunk indices to reserve. We could reserve enough indices to store the whole database plus some slack for fragmentation. If fragmentation exceeds our expectations because compaction hasn't run in a while, we could force one if we run out of indices. Something along those lines.