Documentation for OCI Limitations

[dmlb2000]

I was trying to lookup some of the limitations of the OCI spec to understand how large an OCI container could get and I couldn't find anything.

Can we get a limitations document somewhere that discusses what the limits are on OCI?

Topics to consider:

• Links to Limitations of consumed specifications
  • Max JSON size?
  • Max layer size?
• How big can a OCI container be?
• How many layers can there be?
• How many files can be in an OCI container?

[tianon]

The spec (intentionally) doesn't explicitly define any hard limits on any of these things. 😅

Registry and client implementations might (reasonably!) do so however, and there was a productive discussion around this in opencontainers/distribution-spec#260, which led to the manifest size limit recommendation that was added in opencontainers/distribution-spec#293 (essentially, that clients should be able to reasonably expect that manifests less than 4MiB in size will be accepted and that registries should not typically set their maximum size lower than that).

[tianon]

(There are some historical functional/technical limits to some of these, like the limits on the raw number of layers an image could have in previous versions of runtimes like Docker which I believe have since either been worked around sufficiently to be removed entirely or raised sufficiently to not typically be a cause for concern in normal use, but nothing officially codified in the spec itself which is intended to be generic.)

[dmlb2000]

@tianon thanks for the links on the discussion that's vary helpful. Seems like that was focused more on the manifest size and what a registry would need to support. Is there any specifics in the OCI about the layer format? is that always TAR?

My perspective is that of research data and we can get pretty large with respect to what we want to hold. So, maybe think about the problem this way.

Can I create a 100Tb OCI container? This would contain mostly data, and on the order of 100k files. Let's not consider what a registry would need to manage a container that size, can I even create one that big?

[dmlb2000]

Another consideration are the tar formats, they have some limitations depending on the flavor of tar you use. Similar documents can be found for ZIP and ZIP64 if those are allowed...