Development update

[parke]

It has been about 9 months since I last published an update to Lxroot.

I still use Lxroot 24 hours a day, 7 days a week. I run Chromium, Steam, various AppImages, and various other programs inside of Lxroot.

Development has slowed (but not stopped!) because Lxroot does at least 80% of what I need.

In the last couple months, I have made (but not published) several improvements to Lxroot. The most notable improvements have to do with how overlays are specified. These improvements were inspired by my experiences using the vland helper script I wrote.

• Overlays have been reconceptualized.
• Overlays are now mostly implemented in Lxroot itself (i.e. not via numerous command-line arguments).
• The previous "full" and "partial" overlays have been removed.
• An overlay is now specified my creating an "overlay directory".
• The structure of the overlay directory specifies how Lxroot will create and configure (via bind mounts) the full mount namespace of the Lxroot environment.
• Counterintuitively, the base newroot directory is inside the overlay directory. (But it can be a symlink to a directory that is external to the overlay.)
• Consequently, complex mount namespaces can be specified and created without any command-line arguments and without any configuration files. (One could think of overlay directory itself as being the "configuration". But the overlay directory also contains many of the directories that will be bind-mounted together to create the new root of the environment.)
• Also, it is now possible to bind mount files (whereas previously only directories could be bind mounted).

As a result of the above improvements, I have stopped using and updating vland. (Although I may resume working on the non-overlay portions of vland at some unspecified point in the future.)

The above improvements have lowered the costs (in terms of human time, attention, and labor) of creating and maintaining more-complex mount namespaces. So I am now starting to perform tasks in Lxroot that I always want to, but that previously required too much labor to actually specify (and/or maintain) via the old command-line interface. Additionally, maintaining (i.e. making structural changes to) overlays that were specified on the command line always felt brittle to me.

I'm not sure when I will publish these improvements. Lxroot progresses faster when I don't publish changes, as it does take hours to prepare a set of changes for publication. And it probably will take several days to document the new "overlay directory" method of creating complex mount namespaces. Lxroot is a side project for me; Lxroot is a tool I use to pursue other, more important goals.

Most (but not all) of the improvements to Lxroot are only beneficial when working with overlays. If you are just working with a single chroot-directory, then overlay improvements obviously don't matter.

If anyone is interested in seeing or learning about my ongoing (and currently unpublished) improvements to Lxroot, please contact me.

Cheers!

[Nuc1eoN]

I'm not sure when I will publish these improvements. Lxroot progresses faster when I don't publish changes, as it does take hours to prepare a set of changes for publication. And it probably will take several days to document the new "overlay directory" method of creating complex mount namespaces. Lxroot is a side project for me; Lxroot is a tool I use to pursue other, more important goals.

Hi @parke, maybe you could continue your local development as usual and not push any updates to github. Then just cut a new release every interval of eg 6 months (or 12 months whatever fits best).

So every time you push an update to github you can properly update the documentation etc. But in between just develop locally as usual.

This way at least lxroot will always be properly documented and therefore usable for the average person. One release every year/half a year is not so scary to accomplish I think:) Probaly it's a better compromise than pushing undocumented "improvements"?

[parke]

Hi @parke, maybe you could continue your local development as usual and not push any updates to github. Then just cut a new release every interval of eg 6 months (or 12 months whatever fits best).

So every time you push an update to github you can properly update the documentation etc. But in between just develop locally as usual.

This way at least lxroot will always be properly documented and therefore usable for the average person. One release every year/half a year is not so scary to accomplish I think:) Probaly it's a better compromise than pushing undocumented "improvements"?

Thanks for the suggestion.

I believe the current documentation is mostly in sync with current published release, which was published in August 2022, two years ago.

It would help me if I understood why you are interested in seeing new versions of Lxroot, and/or documentation for those versions.

For example:

Are you already using Lxroot?

Are you considering attempting to use Lxroot for specific tasks?

Or are you just generally curious?

Or some other reason?

Cheers!

[Nuc1eoN]

I believe the current documentation is mostly in sync with current published release, which was published in August 2022, two years ago.

I was getting a different impression, since the latest readme states: "As of September 2023,...", therefore the current state seemed possibly fragmented to me.

Are you considering attempting to use Lxroot for specific tasks?

We are very soon releasing GoboLinux 017.1, following some minor releases 017.x.

For the next major GoboLinux 020, the plan is to implement a sandbox of a "GoboLinux base install". Against this sandbox new recipes would have to be tested (=build) in order to be accepted. The motivation behind this is, that often times recipes will succeed on one system, but fail on another due to divergent build environments. The goal is to make sure that a package would ALWAYS compile on a base non-updated GoboLinux environment (=the average user). See gobolinux/Freshen#5 (comment)

For that purpose we have multiple options: Docker (industry standard), Runner (GoboLinux own solution), Lxroot.

I would like to test each approach and see what works best. But the very earliest for this work to commence is December (we still have 017.x patch releases pending).

[parke]

Are you considering attempting to use Lxroot for specific tasks?

We are very soon releasing GoboLinux 017.1, following some minor releases 017.x.

For the next major GoboLinux 020, the plan is to implement a sandbox of a "GoboLinux base install". Against this sandbox new recipes would have to be tested (=build) in order to be accepted. The motivation behind this is, that often times recipes will succeed on one system, but fail on another due to divergent build environments. The goal is to make sure that a package would ALWAYS compile on a base non-updated GoboLinux environment (=the average user). See gobolinux/Freshen#5 (comment)

For that purpose we have multiple options: Docker (industry standard), Runner (GoboLinux own solution), Lxroot.

What features does Runner lack? Or what is inconvenient about using Runner? Was Runner designed for building packages, or for running them (or both)?

One interesting discovery I made (in 2022, I think) is that I can run fakeroot inside of Lxroot. This has allowed me to install Debian and Ubuntu packages inside of Lxroot. So if you want to avoid Docker, and focus on a solution (Lxroot, Runner, or something else) that runs as a single non-root user, I would definitely recommend exploring running fakeroot inside of either Lxroot or Runner. fakeroot has an undocumented environment variable (FAKEROOTDONTTRYCHOWN=1). By setting this, fakeroot can run inside of unprivileged namespaces. (If not set, fakeroot will call chown() and will be confused and exit due to the unexpected value of errno after the call to chown() fails.)

The last published version of Lxroot (version 0.22.1) included automatic setting of the FAKEROOTDONTTRYCHOWN environment variable.

I run apt-get inside of Lxroot as follows:

lxroot -nw /path/to/chroot -- fakeroot apt-get install package-name

-n gives me network access. -w gives me write access to everything inside /path/to/chroot. Inside of Lxroot, the user ID is non-root (i.e. not zero), but apt-get believes it is running as root due to fakeroot.

Another idea to look at is the use of overlay filesystems. Non-root users can create and mount overlay filesystems inside unprivileged namespaces. I believe I added overlay filesystem "support" to Lxroot, but I may have removed it later on because (1) I ended up not using it, and (2) I didn't like the user facing interface.

For Gobo, if you wanted to use an overlay filesystem, you could create the overlay using some other tool, and then Lxroot into the overlay. Lxroot would be completely unaware that it was running on an overlay filesystem, but Gobo would still get the benefits of using the overlay.

The new, unpublished features of Lxroot largely involve use cases where I create multiple, interconnected chroot-environments that share (and don't share) multiple directories in interesting and useful ways. All this was possible with version 22.1 of Lxroot, but it was conceptually burdensome for the end user to keep track of (and make changes to) the various interconnections.

If you provide me with a "GoboLinux base install" tarball (or a script that generates the base-install), I'm happy to see if I can build packages inside that base install via Lxroot.

My expectation is that there is a high probability that all packages will build inside of Lxroot. Perhaps there will be a few hurdles that will require some tweaking. But perhaps not. In almost every case I have encountered: when I think Lxroot is causing an incompatibility problem, it turns out that it is not actually Lxroot that is causing the problem.

I'm not aware of any improvements to Lxroot since version 22.1 that would provide meaningful benefit to your proposed use case.