Add examples/ folder, move non-core bits there

[cgwalters]

The core of composefs today is the mount.composefs and mkcomposefs tools.

Let's move the other stuff into a separate subdirectory to make this clearer. These tools are not installed with make install even before this.

In the future I'd like to more strongly decouple these examples even more.

[giuseppe]

we will need composefs-from-json in c/storage: containers/storage#1646

I think we either need to install it, or move it into c/storage

[cgwalters]

we will need composefs-from-json in c/storage: containers/storage#1646

Ohhh. OK, so this then strongly relates to #125

Here's what I'd bikeshed: we actually move the core logic here into mkcomposefs, as something like mkcomposefs --from-stdin foo.cfs. Now, we could keep json as the input format...but ISTM we could do something even lower-dependency here, like a repeating format. Or...perhaps one of the variants of cpio format? There's some Go and Rust packages e.g. for reading/writing cpio already.

[cgwalters]

(Side note: wow I hadn't realized earlier that unlike tar, cpio does the sane thing and has the symlink target be in the same place as the regular file data; no hacky "GNU long links")

To elaborate slightly, proposed "cpio stdin protocol" would be the same format as the Linux kernel parses i.e. -H newc. The big exception here is that every regular file has zero data. Instead, the caller is responsible for computing the fsverity digest.

Except...oh, arggh...cpio doesn't support xattrs. (There is an attempt to add support to the Linux kernel).

Well...we could just reuse the proposed format anyways - in the end for this use case, we have the benefit of being able to just pick whatever we want. And I'd say for this the overhead of JSON, particularly parsing textual date formats is unwanted. So something like this "extended cpio" i.e. a binary format makes more sense to me.

And taking this out even farther - IMO we could consider dropping the C shared library for now, and have mkcomposefs internally when presented with an input directory, split into two processes communicating over a pipe - one which creates a serialized stream, and one which parses it to write to the backing store.

[alexlarsson]

In this particular case the json format is already prescribed by zstd:chunked, no? I mean, we can maybe also support a different. more efficient, format, but that will not help containers/storage.

[cgwalters]

In this particular case the json format is already prescribed by zstd:chunked, no?

I don't understand that at all. Asked in containers/storage#1646 (comment) too.

What metadata?

Oh...is this about integrating with an "on demand fetching" model using the fsverity digest as the key?

But that doesn't seem like a requirement for integration of c/storage and composefs. Also in fact ISTM it'd just be more efficient to use my proposed cpio variant instead of JSON there too.

But another way to say this is: I wouldn't tie on-demand fetching with c/storage support for composefs. We should be able to just optionally enable composefs as a backing store for existing standard OCI (e.g. tar+gzip, no zstd/estargz stuff) images - in the same way the ostree support works today.

[cgwalters]

This is obsoleted by #226 etc