Proposal for taking nydus store plugin as a non-core subproject

[imeoer]

Nydus image is a container accelerated image format provided by the Dragonfly image-service project, which offers the ability to pull image data on demand, without waiting for the entire image pull to complete and then start the container. It has been in production and shown vast improvements over the OCI v1 image format in terms of container launching speed, image space, and network bandwidth efficiency, as well as data integrity. Nydus image can be flexibly configured as a FUSE-based user-space filesystem or in-kernel FS with nydus daemon in user-space:

• Lightweight integration with VM-based containers runtime like KataContainers. KataContainers is supported Nydus as a native image acceleration solution.
• Nydus closely cooperates with Linux in-kernel disk filesystem containers' rootfs can directly be set up by EROFS over Fscache with lazy pulling capability. The corresponding changes has been in Linux kernel since v5.19.

Nydus has been running steadily for 3 years in Alibaba Cloud, Ant Group, and ByteDance, supporting large-scale container clusters and the creation of millions of containers per day, serving thousands of customers in various business fields.

Podman is a user-friendly container runtime, we'd like to make it support Nydus images as well, so we created the nydus store plugin project like stargz store plugin, the project implements an additional layer store plugin for CRI-O/Podman, so that nydus images can be mounted with lazy pulling. The plugin depends on additional layer store plugin feature, uses go fuse implementation to interact with podman in filesystem, and nydusd does the actual remote mount for nydus image.

Now we would like to ask the community's opinion if we can propose to make the nydus store plugin as a non-core subproject of the containers community, we believe that the donation to containers will attract more users and developers for nydus itself and bring much value to the containers community users.

Looking forward to any feedback, thanks!

[vrothberg]

Thanks for reaching out and for providing all the details, @imeoer!

@containers/podman-maintainers WDYT?

[giuseppe]

I am in favor of it as long as there is at least one maintainer for the project.

@imeoer who would be the maintainer?

If we agree to move forward, I can help to set up the repository here

[rhatdan]

I am fine with this, as long as it has maintainers. I would also like to see it packaged up into Fedora and made easy to be used by Podman.

[TomSweeneyRedHat]

SGTM, assuming a maintainer(s) is identified as @giuseppe noted and that appropriate tests and documentation are included. Thanks for bringing this forward @imeoer !

[imeoer]

@vrothberg @giuseppe @rhatdan @TomSweeneyRedHat

Many thanks to all, we currently have an initial version and will continue to refine and add tests and documentation. @fatelei and @imeoer will be the initial maintainers, the repo name may be crio-nydus-store-plugin? please let us know if there are any additional details and conduct of community to be aware of, thanks!

[baude]

hello @imeoer to be be more clear, when we discussed this as a team, we wanted it to actually work with podman. how close are you to that?

[imeoer]

@baude Thanks for the reply, the current implementation allows podman to basically run the nydus image, here is a demo and simple documentation, currently we need to continue to enhance the stability, functionality, and additional testing and documentation, will reach production availability in the future.

[baude]

ok, excellent; @giuseppe i can also help as well.

[imeoer]

@baude Thanks :) cc @giuseppe If have any details needed to be provided to create the repository, please help to tell, thanks.

[giuseppe]

should I create a new repository or do you prefer if I move an existing project?

If we go for a new repository, what should be its name? nydus-storage-plugin?

[imeoer]

should I create a new repository or do you prefer if I move an existing project?

If we go for a new repository, what should be its name? nydus-storage-plugin?

A new repository is good for me, and nydus-storage-plugin is ok, @fatelei and @imeoer will be the initial maintainers, thanks!

[giuseppe]

created https://github.com/containers/nydus-storage-plugin and added both of you as admin.

Welcome aboard!

[imeoer]

Thanks all! We are cleaning up the code and will push the initial version soon. cc @fatelei