Special paths defined as env variables in system files should be mapped as systemd specifiers

[rugk]

Example

E.g. say you have the following YAML:

version: "2.1"
services:
  grocy:
    image: lscr.io/linuxserver/grocy:latest
    container_name: grocy
    volumes:
      - $HOME/config:/config:Z
    ports:
      - 9283:80

This currently get's mapped as (Volume matters here):

[Container]
ContainerName=grocy
Image=lscr.io/linuxserver/grocy:latest
PublishPort=9283:80
Volume=$HOME/config:/config:Z

[Service]
Restart=always

[Install]
WantedBy=default.target

Problem

This does not work, as I dunno… some systemd thing…(?) I guess $HOME is not defined in that context.

in any case the error is something like if you want to start this Quadlet with podman:

(podman)[82122]: grocy.service: Invalid environment variable name evaluates to an empty string: HOME/config:/config:Z

Solution

It should replace some well-known env variables with their systemd equivalents.
HOME should be replaced with %h e.g.

Aka the result should be this:

Volume=%h/grocy-data:/config:Z

There are more such variables: https://www.freedesktop.org/software/systemd/man/latest/systemd.unit.html#Specifiers

[k9withabone]

Podlet does not currently perform interpolation of compose files like docker compose does. That should probably be on the README.

I just started working on a new Rust library to better implement the compose spec and replace the docker-compose-types library currently used. It will take a while though. The library will eventually support interpolation.

I feel like the proposed solution is out of scope for podlet. What should also be on the README, is that podlet is mostly meant to just get you started with quadlet/systemd. I thought this was kind of implied already but I'll try to make clearer.

[rugk]

I feel like the proposed solution is out of scope for podlet. What should also be on the README, is that podlet is mostly meant to just get you started with quadlet/systemd.

So the idea is to convert all docker-compose to quadlet files and then only modify these quadlet files?

I currently thought of it as an adapter for docker-compose, so I can start docker-compose files with podman (quadlets). In other words: I thought it is more a thing built into your "pipeline" that converts your configuration and starts it afterwards.

I just started working on a new Rust library to better implement the compose spec

That sounds awesome! I guess you're also aware of podman-compose, which I try to move away from by using this (because it did not work reliable in the past, but that's off-topic).
Anyway…

That should probably be on the README.

Yeah…

I feel like the proposed solution is out of scope for podlet.

Mind to explain, why, actually?
After all due to being (documented) part of systemd files, it is also part of Quadlet files. As such, it is the target format this project targets.
And it fixes a real-life deployment issue.

[k9withabone]

So the idea is to convert all docker-compose to quadlet files and then only modify these quadlet files?

Yup, that was what I was thinking when I added the compose feature and when I created podlet originally. It doesn't really make sense to maintain a compose file if your goal is to transition to using podman and quadlets. If you want to group containers together you can use pods. Podman v5.0.0 will add support for .pod quadlet files. For now, if you want a configuration file to define a pod, you have to use Kubernetes YAML, which can be generated using the podlet compose --pod option. Note that the current functionally will be renamed to --kube and --pod will generate a .pod file sometime after podman v5.0.0 is released.

I guess you're also aware of podman-compose

Yup. By the way, you can also use docker compose with podman. You just have to give it the podman Unix socket.

Mind to explain, why, actually?

Because it feels too much like trying to guess the user's intentions. Besides, you can use a pretty simple sed command for your example: sed 's/$HOME/%h/g'. You may also want to try using a named volume instead of a host directory so that you can just let podman worry about managing it.

[k9withabone]

Look, if you want a feature for using systemd specifiers I think you are going to have to be more specific because I'm not understanding what it is you want exactly.

• What should the option flag to activate the feature be named?
• Should it take any arguments?
• How does it change podlet's behavior?
• Which systemd specifiers should replace what values?
  • How to determine which values should be replaced?
• What kind of errors could happen?
  • How should they be handled or reported to the user?
• Any other details you feel is important for the feature.

I'd also be happy to accept a PR on this. Though how its implemented should be discussed first. It should also probably wait until after I get #52 done as implementing that will require changing the structs in the quadlet module. Those changes will also probably help with implementing whatever feature this turns into.

[k9withabone]

Also, just rereading your response now it sounds like you took my compose spec Rust library to mean that I'm making something like podman-compose. I'm not, the library is purely meant for verifying and deserializing compose files into Rust types. Maybe I shouldn't have said I'm implementing the compose spec? Though I'm not sure how else to describe it.

[rugk]

What should the option flag to activate the feature be named?

Dunno --use-systemd-specifiers maybe? Or just by default?

Should it take any arguments?

No, which ones should it?

How does it change podlet's behavior?

Replaces unique strings (maybe start with paths only; including variables), with the systemd specifiers defined here.

Which systemd specifiers should replace what values?

Yeah, maybe only paths.

How to determine which values should be replaced?

Implementation detail. 😜 No just kidding, maybe expand the paths indeed and then find and replace the paths with the specifiers?

What kind of errors could happen?

Hmm see none? Either a replacement string is found or not. I mean replacing is an optional thing, it's not required.

---

(getting off-topic…)

something like podman-compose

No, that's obviously clear. Though, of course you can use this project in a pipeline to convert your compose files. After all that's the aim…

It doesn't really make sense to maintain a compose file if your goal is to transition to using podman and quadlets.

Well… depends on your use cases. docker-compose is a well-known format and many tools support it, so it would allow you to move away in case podman changes something or your infrastructure (requirements) change etc. containers/podman#20218 is full of people migrating away after podman systemd generate is now marked as deprecated (this is how I came here), so there is a precedent and converting Quadlet files back to docker compose files seems as hard as this way. I just don't want to have to migrate another time, because some tooling breaks sth.
(That all said, i think Quadlets etc. are quite cool and happily migrate, just being cautious here to keep the podman-compose.yml files.)

Anyway, enough off-topic…

[k9withabone]

Should it take any arguments?

No, which ones should it?

I was thinking maybe a list of specifiers to use? With a default list if the option is given without a list?

How does it change podlet's behavior?

Replaces unique strings (maybe start with paths only; including variables),

I think this feature will have to wait until compose interpolation is in, and to require the --absolute-host-paths option from #52. That way podlet doesn't have to worry about replacing variables or relative paths. What do you mean by "unique strings"?

Which systemd specifiers should replace what values?

Yeah, maybe only paths.

So, from the list, "%C", "%d", "%E", "%h", "%L", "%S", "%t", "%T", and "%V"?

Many of those specify environment variables to use for user managers. What should happen if those variables aren't set or are empty? What does systemd do in that case? Should podlet fallback to the ones from the XDG Base Directory Specification? What if $HOME is not set?

Specifically for the temp directories, multiple environment variables and a set directory are listed. In what order should they take precedence?

What kind of errors could happen?

Hmm see none? Either a replacement string is found or not. I mean replacing is an optional thing, it's not required.

What if a directory that a specifier is supposed to replace cannot be determined? I think that should be a reported error as podlet won't do what the user expects otherwise.

Should this feature work on Linux only? What about other Unix-like OSs (e.g. macOS and FreeBSD)? What about Windows? Should it just error if not supported?

I just don't want to have to migrate another time, because some tooling breaks sth.

Fair enough, though I'm pretty sure quadlet is here to stay given that it originated out of a need from the automotive industry. Also, I think the "full of people" comment is exaggerating a bit given that there are only 5 (including you) non podman maintainers/contributors in that discussion and 2 of them commented only once.

I wish that docker wasn't as completely dominant as it is. Even the compose-spec, which is supposed to be vendor neutral, is very docker heavy.

I'm really not trying to be adversarial here. It's just that this feature is perhaps a bit more complicated than you thought. Also, I'm not a systemd expert. I've never tried to make a service that is portable across systems before.

[rugk]

What does systemd do in that case?

Probably the most relevant question as this should behave the same.

It's just a quick look, but I've found these relevants spots:

• definition: https://github.com/systemd/systemd/blob/de39202426e2d3d4d6fb509635d2462b5ded6e9c/src/core/unit-printf.c#L208-L276
• replacement: https://github.com/systemd/systemd/blob/de39202426e2d3d4d6fb509635d2462b5ded6e9c/src/libsystemd/sd-path/sd-path.c#L187-L391

Should podlet fallback to the ones from the XDG Base Directory Specification?

From what I read, yes it does, and this is the reason to use such variables in the first place.

What if $HOME is not set?

I am not so good in C, but apparently, it has some way more complex logic for this case than what I'd expect aka e.g. having hardcoded paths for root and nobody as users.¹

Specifically for the temp directories, multiple environment variables and a set directory are listed. In what order should they take precedence?

Well as I would read the doc always in reading order. The temp dir handling as I saw in source code is more complex though FYI.¹

What if a directory that a specifier is supposed to replace cannot be determined? I think that should be a reported error as podlet won't do what the user expects otherwise.

Yeah maybe. But given you still have a working (non-replaced/special) path, if it is not replaced, it is maybe no big deal.

What about Windows? Should it just error if not supported?

I have no idea. Only ever used that on Linux. As such, maybe just don't offer that feature there, or as you say, error.

---

You know, this idea was born of me using this tool, just FYI. So I thought this was a good idea. If not, yeah, maybe, I just thought it would be a cool convenience feature at the time when opening the issue.

Also maybe I should clarify I would be fine with replacing env variables (that have a defined mapping in systemd) for now, not all file paths or so. I guess (hope), this may ease implementation. I'll correct the title…

¹ Edit: As such, for the reverse mapping issue we have at hand here (from env vars to specifiers), this should not matter as far as I see.

[k9withabone]

Blocked on #63 and k9withabone/compose_spec_rs#3.