Running Mox on Kubernetes

[protocol6]

Hi,

First off, thanks for creating mox. :-)
It's is a really elegant single process solution for hosting your own mailserver; something which is getting increasingly complex.
You've clearly given a lot of thought to making things as easy as possible.

I'm in the process of setting up a mox server on kubernets, using the two latest docker images you offer on your registry.
Its almost working but I keep getting the following fatal startup error after mox.conf and domains.conf have been applied.

defaulted container "mox" out of: mox, prep-mox (init)
l=debug m="autotls setting allowed hostnames" pkg=mox hostnames=[mail.blahblah.org;webmail.blahblah.com] publicips=[127.0.0.1] or [0.0.0.0]
l=print m="starting as root, initializing network listeners" pkg=serve version=(devel)-go1.23.4 pid=1 moxconf=/mox/config/mox.conf domainsconf=/mox/config/domains.conf
C
l=print m="listening for smtp" pkg=smtpserver listener=public address=127.0.0.1:1587 protocol=submission
l=print m="listening for smtp" pkg=smtpserver listener=public address=127.0.0.1:1465 protocol=submissions
l=print m="listening for imap" pkg=imapserver listener=public addr=127.0.0.1:1993 protocol=imaps
l=print m="https listener" pkg=http name=public kinds=acme-tls-alpn-01 address=127.0.0.1:443
l=print m="http listener" pkg=http name=public kinds=metrics-http address=127.0.0.1:8010
l=print m="http listener" pkg=http name=public kinds="account-http at /,acme-http-01,admin-http at /admin/,webmail-http at /webmail/" address=127.0.0.1:8080
l=print m="starting as root, initializing network listeners" pkg=serve version=(devel)-go1.23.4 pid=1 moxconf=/mox/config/mox.conf domainsconf=/mox/config/domains.conf
l=print m="listening for smtp" pkg=smtpserver listener=public address=127.0.0.1:1025 protocol=smtp
l=fatal m="smtp: listen for smtp" err="listen tcp4 0.0.0.0:1025: bind: address already in use" pkg=smtpserver protocol=smtp listener=public
l=print m="stopping after child exit" pkg=mox exitcode=1

This occurs when User: root (or User: 0) in mox.conf and with MOX_SOCKETS="", MOX_DOCKER="yes" and MOXCONF="/mox/config/mox.conf. It also doesn't matter whether I use the default privileged ports or the custom non-privileged ports above, nor does it matter whether I use public ips 127.0.0.1 or 0.0.0.0

• I wonder if mox in the container is not able to drop privileges properly and thinks that its trying to bind the ports twice?
• Or is there a specific setting I should use for MOX_SOCKETS?
  • It would be very helpful to include a list allowed environment variables in the docs together with realistic sample values!

Thanks again for this great tool. Any help on this would be greatly appreciated. :-)

P.S.:
By the way, using User: mox (preferred) or User: 1000 always give me a different fatal error.

l=fatal m="mox must be started as root, and will drop privileges after binding required sockets (missing environment variable MOX_SOCKETS). If you updated from v0.0.1, update the mox.service file to start as root (privileges are dropped): ./mox config printservice >mox.service && sudo systemctl daemon-reload && sudo systemctl restart mox." pkg=mox

[mjl-]

Hi @protocol6, thanks for reporting. I think I understand what's going on. Mox initially starts as root, to bind to the ports (binding to ports <1024 is a privileged operaiton). Then it starts an unprivileged process to do most of the work (like serving connections on the ports it started listening on), with the uid of the the unprivileged user specificied in "User:" in mox.conf. But with "User: root" or "User: 0", the new process is started as root, and that second "unprivileged" process thinks it's the first kind of process again. We're seeing that in the logging, with the message "starting as root, initializing network listeners" happening twice.

Mox won't be able to start with the root/0 user as "User" in mox.conf. As far as I know, it's still not a good idea to run services like this (or any service) as root, even when in a container. You mentioned getting different errors with "User: mox" or "User: 1000". With "User: mox", I expect you'll get an error until you create the user in the container. But that's why specifying "User: " suggested. I wouldn't expect an error when you specify "User: 1000" (assuming you want to run mox with uid 1000). Which error are you getting? If it's about file permissions, the directories "config" and "data" should be owned by that uid. The first few lines of docker-compose.yml in the repo are trying to hint at that, https://github.com/mjl-/mox/blob/v0.0.13/docker-compose.yml#L1-L5. But perhaps that doesn't work, or it's something else. We should probably at least improve the documentation around this.

You should never specify MOX_SOCKETS yourself. It's used internally by mox to pass file descriptors from the initial privileged process to the second unprivileged process. The error is meant as a diagnostic. But I'm starting to think it's not helpful to mention it in the error message.

I think I should either at startup that "User: ..." isn't uid 0 (and give a clear error message instead of the current behaviour), and/or allow uid 0 and use the presence of MOX_SOCKETS (or some other flag) to check whether the process is supposed to be the first or second type of process. Do you have a reason to have "User: root" other than the "User: mox" and "User: 1000" resulting in errors?