[feature] [core-server-kit] Fully support the parallel, distributed filesystem Ceph as sys-cluster/ceph

[cuantar]

The Ceph filesystem now is mature and production-quality, capable of everything that GlusterFS and friends can do. We should support it, by adding the userspace tools as autogen at the latest version. Will need to verify that it's enabled in our kernel. Manual installation instructions are here:

https://docs.ceph.com/en/latest/install/index_manual/#install-manual

[geaaru]

This is something that I like too and probably it will be good to have a dedicated documentation page that helps users on setup a cluster.

[cuantar]

We might be able to harvest openrc init scripts from Alpine: https://pkgs.alpinelinux.org/package/edge/community/x86_64/ceph19-openrc

[cuantar]

Gentoo has it too: https://github.com/gentoo/gentoo/tree/master/sys-cluster/ceph

[cuantar]

There won't be a problem with it on Macaroni/MARK. Some of the tools won't work unless deployed in an environment with systemd, but the various daemons can be managed manually using openrc scripts/configs. It's still unclear to me whether tools like cephadm expect to link against systemd libraries, or if only certain functionality is broken at runtime. The latter case is true for Warewulf, that attempts to invoke systemd cli tools to restart services but doesn't link the libraries. Those calls fail, but the error message alerts me that I need to restart the services manually.