Can't start endlessh service "Failed at step NAMESPACE", "status=1/FAILURE", "too many arguments"

[DaveTSG]

Hello.

I'm having a similar to issue 66 and 53, but I can't figure out a solution by myself.

Running Debian 12. Installed endlessh using the apt package manager, i.e. apt install endlessh
Config files and the endless program itself don't seem to be installed in the same directories as per the documentation (presumably because it was installed with the Debian package manager). Finding documentation for Debian specifically is ...challenging.

sudo endlessh -v -p 22
...works perfectly fine. I can attempt to log in from another machine and I can't get access, as expected.

I then tried to create the config file, log file, and error file in a folder in my user directory, i.e.:

mkdir /home/david/endlessh
touch /home/david/endlessh/config
touch /home/david/endlessh/endlessh.log
touch /home/david/endlessh/endlessh.err

I then copied the sample configuration file contents from your documentation into /home/david/endlessh/config, except I changed to port 22 and LogLevel to 1. Otherwise, the config file is identical.

systemctl status endlessh shows the .service file is located at /lib/systemd/system/endlessh.service.

sudo nano /lib/systemd/system/endlessh.service I then changed the ExecStart line to:
ExecStart=/usr/bin/endlessh -f /home/david/endlessh/config -v > /home/david/endlessh/endlessh.log 2> /home/david/endlessh/endlessh.err

The entire contents of /lib/systemd/system/endlessh.service is:

[Unit]
Description=Endlessh SSH Tarpit
Documentation=man:endlessh(1)
Requires=network-online.target

[Service]
Type=simple
Restart=always
RestartSec=30sec
ExecStart=/usr/bin/endlessh -f /home/david/endlessh/config -v > /home/david/endlessh/endlessh.log 2> /home/david/endlessh/endlessh.err
KillSignal=SIGTERM

# Stop trying to restart the service if it restarts too many times in a row
StartLimitInterval=5min
StartLimitBurst=4

StandardOutput=journal
StandardError=journal
StandardInput=null

PrivateTmp=true
PrivateDevices=true
ProtectSystem=full
ProtectHome=true
#InaccessiblePaths=/run /var

## If you want Endlessh to bind on ports < 1024
## 1) run: 
##     setcap 'cap_net_bind_service=+ep' /usr/local/bin/endlessh
## 2) uncomment following line
#AmbientCapabilities=CAP_NET_BIND_SERVICE
## 3) comment following line
PrivateUsers=true

NoNewPrivileges=true
ConfigurationDirectory=endlessh
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
MemoryDenyWriteExecute=true

[Install]
WantedBy=multi-user.target

(Note I have tried this both with and without the InaccessiblePaths=/run /var line being commented-out, but both times I had the same issue)

I then ran sudo systemctl --now enable endlessh. The stdout was:

Synchronizing state of endlessh.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable endlessh

I then ran systemctl status endlessh. The first time, the stdout contained "status=226/NAMESPACE". I went Googling and found the above-mentioned issues 53 and 66. They advised to comment-out the InaccessiblePaths line. Then I tried sudo systemctl --now enable endlessh again, with the following stdout:

● endlessh.service - Endlessh SSH Tarpit
     Loaded: loaded (/lib/systemd/system/endlessh.service; enabled; preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Fri 2024-07-12 11:53:40 AEST; 76ms ago
       Docs: man:endlessh(1)
    Process: 37873 ExecStart=/usr/bin/endlessh -f /home/david/endlessh/config -v > /home/david/endlessh/endlessh.log 2> /home/david/endlessh/endlessh.err (code=exited, status=1/FAILURE)
   Main PID: 37873

I also tried a system reboot at this point, just in case something wasn't working properly and needed to be restarted, but again, I ended up with the same error message as above.

If I try to run sudo endlessh -v > /home/david/endlessh/endlessh.log 2> /home/david/endlessh/endlessh.err manually, it works perfectly, and the file at /home/david/endlessh/endlessh.log is populated with log lines (but obviously this is not running as a service in the background, so isn't ideal).

Any idea what I'm doing wrong please?

Thanks.

P.S.: I'm a noob when it comes to services, so I'm assuming PEBKAC...

[DaveTSG]

I just thought to check journalctl | grep "endlessh", and saw endlessh: too many arguments in the log file.

I then shortened it to ExecStart=/usr/bin/endlessh just to see if I could get that to work, but nope. I got the same status=1/FAILURE error in the output of systemctl status endlessh.service.

[DaveTSG]

Hoping this'll help diagnose the issue...

sudo systemctl start endlessh
Job for endlessh.service failed because the control process exited with error code.
See "systemctl status endlessh.service" and "journalctl -xeu endlessh.service" for details.

su -
journalctl -xeu endlessh.service

The output from that includes the line: endlessh: fatal: Permission denied

I then thought maybe if I start the service as root...?

root@docker1:~# systemctl --now enable endlessh
Synchronizing state of endlessh.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable endlessh
root@docker1:~# systemctl status endlessh
● endlessh.service - Endlessh SSH Tarpit
     Loaded: loaded (/lib/systemd/system/endlessh.service; enabled; preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Fri 2024-07-12 13:30:30 AEST; 4s ago
       Docs: man:endlessh(1)
    Process: 9384 ExecStart=/usr/bin/endlessh (code=exited, status=1/FAILURE)
   Main PID: 9384 (code=exited, status=1/FAILURE)
        CPU: 35ms

...guess not.

[DaveTSG]

Found this, which suggested running journalctl -u endlessh.service -b (presumably as root):

Jul 12 13:36:33 docker1 endlessh[12222]: 2024-07-12T03:36:33.298Z Port 22
Jul 12 13:36:33 docker1 endlessh[12222]: 2024-07-12T03:36:33.298Z Delay 10000
Jul 12 13:36:33 docker1 endlessh[12222]: 2024-07-12T03:36:33.298Z MaxLineLength 32
Jul 12 13:36:33 docker1 endlessh[12222]: 2024-07-12T03:36:33.298Z MaxClients 4096
Jul 12 13:36:33 docker1 endlessh[12222]: 2024-07-12T03:36:33.298Z BindFamily IPv4 Mapped IPv6
Jul 12 13:36:33 docker1 endlessh[12222]: endlessh: fatal: Permission denied
Jul 12 13:36:33 docker1 systemd[1]: endlessh.service: Main process exited, code=exited, status=1/FAILURE
Jul 12 13:36:33 docker1 systemd[1]: endlessh.service: Failed with result 'exit-code'.

I then found issue 39, in which the OP (like me) didn't do the commenting as required for ports <1024. When I first read that, I thought that meant to have endlessh operate on ALL ports under 1024 (i.e. port 1, 2, 3, ... 1022, 1023). Of course, on a second-read, I realise it means "if I want endlessh to work on ANY port under 1024, I have to do it.

endlessh.service now contains:

[Unit]
Description=Endlessh SSH Tarpit
Documentation=man:endlessh(1)
Requires=network-online.target

[Service]
Type=simple
Restart=always
RestartSec=30sec
ExecStart=/usr/bin/endlessh -f /home/david/endlessh/config -v >/home/david/endlessh/endlessh.log 2>/home/david/endlessh/endlessh.err
KillSignal=SIGTERM

# Stop trying to restart the service if it restarts too many times in a row
StartLimitInterval=5min
StartLimitBurst=4

StandardOutput=journal
StandardError=journal
StandardInput=null

PrivateTmp=true
PrivateDevices=true
ProtectSystem=full
ProtectHome=true
#InaccessiblePaths=/run /var

## If you want Endlessh to bind on ports < 1024
## 1) run: 
##     setcap 'cap_net_bind_service=+ep' /usr/local/bin/endlessh
## 2) uncomment following line
AmbientCapabilities=CAP_NET_BIND_SERVICE
## 3) comment following line
#PrivateUsers=true

NoNewPrivileges=true
ConfigurationDirectory=endlessh
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
MemoryDenyWriteExecute=true

[Install]
WantedBy=multi-user.target

I then tried to run setcap 'cap_net_bind_service=+ep' /usr/local/bin/endlessh as instructed in the .service file, but got setcap: command not found. I eventually figured out setcap is installed in /sbin/setcap. And, of course, for Debian, endlessh is installed in /usr/bin/endlessh.

I then ran /sbin/setcap 'cap_net_bind_service=+ep' /usr/bin/endlessh
...there was nothing printed to stdout, so presumably this was ok?

...but nope, still getting that status=1/FAILURE error when I check the status of the endlessh service.

systemctl status endlessh.service | cat
● endlessh.service - Endlessh SSH Tarpit
     Loaded: loaded (/lib/systemd/system/endlessh.service; enabled; preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Fri 2024-07-12 13:59:35 AEST; 13s ago
       Docs: man:endlessh(1)
    Process: 4370 ExecStart=/usr/bin/endlessh -f /home/david/endlessh/config -v >/home/david/endlessh/endlessh.log 2>/home/david/endlessh/endlessh.err (code=exited, status=1/FAILURE)
   Main PID: 4370 (code=exited, status=1/FAILURE)
        CPU: 34ms

...that's about all I can think of to try. Hopefully someone smarter than I knows what I'm doing wrong...

[DaveTSG]

Ok, I might be getting closer...
/lib/systemd/system/endlessh.service now contains the following:

[Unit]
Description=Endlessh SSH Tarpit
Documentation=man:endlessh(1)
Requires=network-online.target

[Service]
Type=simple
Restart=always
RestartSec=30sec
ExecStart=/usr/bin/endlessh -f /home/david/endlessh/config
KillSignal=SIGTERM

# Stop trying to restart the service if it restarts too many times in a row
StartLimitInterval=5min
StartLimitBurst=4

StandardOutput=/home/david/endlessh/endlessh.log
StandardError=/home/david/endlessh/endlessh.err
StandardInput=null

PrivateTmp=true
PrivateDevices=true
ProtectSystem=full
ProtectHome=true
#InaccessiblePaths=/run /var

## If you want Endlessh to bind on ports < 1024
## 1) run: 
##     setcap 'cap_net_bind_service=+ep' /usr/local/bin/endlessh
## 2) uncomment following line
AmbientCapabilities=CAP_NET_BIND_SERVICE
## 3) comment following line
#PrivateUsers=true

NoNewPrivileges=true
ConfigurationDirectory=endlessh
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
MemoryDenyWriteExecute=true

[Install]
WantedBy=multi-user.target

I only just saw the StandardOutput and StandardError fields, so I populated them and removed them from the ExecStart line.

sudo systemctl daemon-reload && sudo systemctl restart endlessh.service && systemctl status endlessh.service results in...:

● endlessh.service - Endlessh SSH Tarpit
     Loaded: loaded (/lib/systemd/system/endlessh.service; enabled; preset: enabled)
     Active: active (running) since Fri 2024-07-12 14:05:50 AEST; 3s ago
       Docs: man:endlessh(1)
   Main PID: 6837 (endlessh)
      Tasks: 1 (limit: 18927)
     Memory: 320.0K
        CPU: 34ms
     CGroup: /system.slice/endlessh.service
             └─6837 /usr/bin/endlessh -f /home/david/endlessh/config

...and attempting to connect to port 22 from my other computer results in it getting stuck in the tarpit.

However, I note that the output is not going into /home/david/endlessh/endlessh.log; the only lines that are currently in that file are from the last time I tried to manually run sudo endlessh -v > /home/david/endlessh/endlessh.log 2> /home/david/endlessh/endlessh.err, despite new lines being output on my other computer (which is currently stuck in the tarpit).

Any ideas please on how to get the output into the correct files?

Thanks.

[DaveTSG]

Got it.

This explains...
StandardOutput=file:/var/log/endlessh.log

All I needed to do was add the file: at the start of the path to each file.
Working perfectly now. :)

[jibblet]

THANK YOU. This helped me tremendously.