Can Flatcar Container Linux replace BurmillaOS in long term?

[olljanat]

There is only 9 votes in #186 when I'm writing this and lately in my daily work we have been evaluating Flatcar Container Linux which looks promising.

That why I would like to ask community to evaluate Flatcar and tell what what are cons compared to BurmillaOS?
Perhaps those can be solved by contributing that project?

Pros are quite clear:

• Supported by large companies
• Available in all public clouds
• Immutable file system
• Automatic updates

[benok]

Cons:

• Can't customize console

I don't use Kubernetes now; I use docker compose both in my home lab and (a little*) at work.
(*Internal server or public but internal use only server, just <20 users use)

I deploy server VMs both manually and semi-automatically using Terraform.
I'm using BurmillaOS VMs for both dev and prod servers.
(I have to use Windows laptops and I don't like local docker development. I prefer using Esxi or Proxmox.)

So I regularly use dev BurmillaOS VM's console when I set up a new service.

I tried CoreOS several years ago, but I didn't like its dumb console, so I chose RancherOS and the Debian console at that time.

Of course, I can set up a plain Debian VM for development,
but I prefer BurmillaOS's console/system upgradability and more consistent deployment with cloud-init.

I think BurmillaOS fits for home users and small business users like me.

But I also understand Flatcar's all pros.
Though I prefer BurmillaOS, I feel "using Debian for dev and Flatcar for prod" is also promising.

[olljanat]

On BurmillaOS console is just one of the containers.

You can get pretty much same user experience by spinning custom console container on top of Flatcar.

Basically you use need to have SSH server included to it and mount /var/run/docker.sock to it so docker commands works. You can see what is included to console from here

Home users, etc who don’t need newer kernel version and are happy with current features in BurmillaOS, most likely should be able to use v2.0.2 many years still. Eventually there most likely will be some breaking change in Docker which would require kernel update but my guess is that it will take at least 5 years because RedHat Enterprise Linux 8.10 is supported still quite long time and it is using 4.18.x kernels.

[PrplHaz4]

I tried flatcar right before I switched to BurmillaOS, and didn't use it long enough to identify any "real" issues. I decided against it mostly because of how RedHat it felt.

Not having the Debian console, and having to learn/use butane/ignition was more effort than I was willing to put in at the time.

[olljanat]

systemd definitely gives RedHat feelings for me too. I'm not big fan of systemd but most distros has switched to it so I guess that it is time to get familiar with anyway.

When it comes to Ignition, the logic of course is a bit different than in cloud-init but haven't see any showstoppers yet.

However, it is big issue in open source world that there is too many solutions to same problems which slowdown the progress.

Not having the Debian console

This looks to be common topic so perhaps I need to create burmilla/console repository which provides Debian/BurmillaOS console together with Ignition example for Flatcar.

[benok]

It may be a bit off the track but, I think we should consider using systemd enabled base image for the console.
https://github.com/trfore/docker-debian11-systemd

Several packages and commands don't work without systemd recently.
(Especially in Debian12)
Perhaps, we should have systemd for the console.

[olljanat]

It is related in way that because BurmillaOS doesn't use systemd it comes more and more hard to get those packages working.

What we use now is custom init + System Docker (customized version of Docker which stuck in very old version #28).
Rancher added this logic which mounts /sys/fs/cgroup/systemd even when BurmillaOS it selves does not need it but it does not necessarily be enough for applications which expect systemd to be used.

And if we would migrate to systemd and fix some other issues what there is then we would end up very near what Flatcar is.

[olljanat]

btw. Added prototype (without systemd for now) to burmilla/console. Needs a lot of work of course but idea looks to be working so for those who are interested, can try and open issues to that side.

[benok]

Rancher added this logic which mounts /sys/fs/cgroup/systemd even when BurmillaOS it selves does not need it but it does not necessarily be enough for applications which expect systemd to be used.

Thank you for pointing this out.
I've never tried systemd-enabled console on BurmillaOS yet, but I still think it's worth trying.
(It might not work correctly, but it can be better than now.)

I also want to try Flatcar and your custom console prototype.