Make ssh work on Windows

[mschubert]

If possible.

This could use the Linux subsystem in newer versions (Win>=10 IIRC)

Update: Powershell now includes SSH by default (or needs to be activated?) on Win10

https://github.com/PowerShell/openssh-portable

https://poweruser.blog/enabling-the-hidden-openssh-server-in-windows-10-fall-creators-update-1709-and-why-its-great-51c9d06db8df

[HenrikBengtsson]

An FYI: The built-in SSH client on Windows 10 v1803 (build 17134.523, 2018-07-10) does not support SSH reverse tunneling. Unfortunately, it looks like Windows 10 v1809 (build 17763.253, 2018-11-13) still doesn't support it properly - although there have been some improvements, socket connections fail over such tunnels. See my https://github.com/HenrikBengtsson/future/issues/244#issuecomment-437568082 for my brief notes on this.

UPDATE 2023-02-05: Reverse tunneling works with MS Windows' ssh too if one uses 127.0.0.1 instead of localhost, or if one uses ssh -4, cf. PowerShell/Win32-OpenSSH#1265 (comment). I've successfully made this the default in parallelly since April 2021.

[mschubert]

Looks like I could use the PuTTY command-line tool (at the cost of incurring an extra dependency).

I'll likely wait until at least some users request this, or the Windows-internal SSH gets fixed.

Thanks!

[mschubert]

This could maybe work with Ubuntu for Windows (H/T UCLA QCB)

[kkmann]

Hey, just ran into this issue as well. This would be super useful to have for beginners (dev on Windows RStudio, Q() on linux HPC).

Not sure whether I understand where the WSL comes into play here. Let me try to describe the issue in a bit more detail.

I am trying to follow the clustermq documentation for connecting to a (remote) slurm cluster for execution using ssh on a Win 10 system from an Rstudio session. The error message I get is

Connecting [xxx]@[yyy] via SSH ...
Error in initialize(...) : 
  Remote R process did not respond after 5000 seconds. Check your SSH server log.
In addition: Warning message:
In system(ssh_cmd, wait = TRUE, ignore.stdout = TRUE, ignore.stderr = TRUE) :
  'ssh' not found

This seems to indicate that clustermq is trying to use the system ssh command for communication. Ofc, ssh is not available by default on all Windows system / prompts. At least it is not on mine. Trying the set the path to the RStudio version (msys) does not work for me either because it then cannot create the .ssh configuration file in the right location (looking in '/home/[xxx]/.ssh' and it seems a bit hackish.

Setting a debug marker and stepping through the code, it seems that the issue is indeed caused by relying on the system ssh command, e.g., here:

clustermq/R/qsys_ssh.r

Line 26 in b5a7664

system(ssh_cmd, wait=TRUE, ignore.stdout=TRUE, ignore.stderr=TRUE)

Is there a specific reason not to go through the rOpenSci ssh package instead?

I tried to connect to the cluster from the debug location (see above) and I could send a 'hello world' to R fine. Wanted to check back with you before digging deeper into the code, maybe there is a good reason for not using the package. If I see it correctly, we need some port forwarding magic...

Otherwise, is there a way to (optionally) use the ssh version installed with RStudio?

[HenrikBengtsson]

Drive-by "FWIW" comment:

Ofc, ssh is not available by default on all Windows system / prompts. At least it is not on mine.

What MS Windows system are you on? AFAIU, ssh should be available on all MS Windows (>= 10) system, unless they have not been updated for a couple of years. MS Windows (<= 8.1) doesn't have ssh, but is End-of-Life as of 2023-01-10. I don't know about MS Windows servers, though.

[HenrikBengtsson]

I'll likely wait until at least some users request this, or the Windows-internal SSH gets fixed.

FYI, I'm not sure if you rely on reverse tunneling, but that works with MS Windows' ssh too if one uses 127.0.0.1 instead of localhost, or if one uses ssh -4, cf. PowerShell/Win32-OpenSSH#1265 (comment). I've successfully made this the default in parallelly since April 2021.

[kkmann]

What MS Windows system are you on?

Win 10 enterprise, org managed and as it seems with the optional openssh feature disabled x)

How do you feel about getting rid of the ssh system dependency and using the ssh package instead? Should work across all systems then, right?

[HenrikBengtsson]

Win 10 enterprise, org managed and as it seems with the optional openssh feature disabled x)

Thank you. I had no idea it was possible to disable it that way. Would you mind sharing exact details of how you navigate to that setting? That could be very useful for other Windows users experiencing similar problems, i.e. would be part of the first troubleshooting instructions to find out of SSH has been disabled.

How do you feel about getting rid of the ssh system dependency and using the ssh package instead? Should work across all systems then, right?

I'm not the maintainer of clustermq, but I'm the author of parallelly, which, like clustermq and base-R package parallel, rely on system calls to ssh, with fallbacks to the RStudio ssh and PuTTY's plink ssh. I asked about your system to learn more about users' compute environments and limitations.

Regarding using the ssh package instead: Personally, I would not switch out the existing solution has a long track record (here and in parallelly), because (i) you don't want to break something that works, and (ii) you never know what new problems are introduced. For example, on Linux, the ssh package, depends on the openssl package, and together they require system library libssl-dev and libssh-dev being available on the system. If not, they'll get installation errors (* see below) and if you're on a shared environment, the user doesn't have admin rights to install those. So, moving to use only the ssh package complicates life for Linux users.

Instead, I think using ssh as an alternative would be the best approach. FWIW, one thing I noticed when I looked at ssh was that I'm not sure if it currently supports reverse SSH tunneling. That's at least a quite essential feature of the parallelly package.

(*) I just tried on a fresh Ubuntu 22.04 setup and I got the below errors:

* installing *source* package ‘openssl’ ...
** package ‘openssl’ successfully unpacked and MD5 sums checked
** using staged installation
Using PKG_CFLAGS=
--------------------------- [ANTICONF] --------------------------------
Configuration failed because openssl was not found. Try installing:
 * deb: libssl-dev (Debian, Ubuntu, etc)
 * rpm: openssl-devel (Fedora, CentOS, RHEL)
 * csw: libssl_dev (Solaris)
 * brew: openssl (Mac OSX)
If openssl is already installed, check that 'pkg-config' is in your
PATH and PKG_CONFIG_PATH contains a openssl.pc file. If pkg-config
is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
-------------------------- [ERROR MESSAGE] ---------------------------
tools/version.c:1:10: fatal error: openssl/opensslv.h: No such file or directory
    1 | #include <openssl/opensslv.h>
      |          ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
--------------------------------------------------------------------
ERROR: configuration failed for package ‘openssl’
* removing ‘/home/henrik/R/x86_64-pc-linux-gnu-library/4.2/openssl’


 installing *source* package ‘ssh’ ...
** package ‘ssh’ successfully unpacked and MD5 sums checked
** using staged installation
Package libssh was not found in the pkg-config search path.
Perhaps you should add the directory containing `libssh.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libssh' found
Using PKG_CFLAGS=
Using PKG_LIBS=-lssh
---------------------------- [ANTICONF] --------------------------------
Configuration failed because libssh was not found. Try installing:
 * deb: libssh-dev (Debian, Ubuntu, etc)
 * rpm: libssh-devel (Fedora, EPEL)
 * brew: libssh (OSX)
If libssh is already installed, check that 'pkg-config' is in your
PATH and PKG_CONFIG_PATH contains a libssh.pc file. If pkg-config
is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
-------------------------- [ERROR MESSAGE] ------------------------------
<stdin>:1:10: fatal error: libssh/libssh.h: No such file or directory
compilation terminated.
-------------------------------------------------------------------------
ERROR: configuration failed for package ‘ssh’

[kkmann]

Great points, offering a fallback or documenting a reasonably stable workaround might be the better way to go.

The official docs on enabling openssh for Windows 10 are: https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse?tabs=gui

Looking at

clustermq/inst/SSH.tmpl

Lines 1 to 7 in 68c03c8

	ssh -o "ExitOnForwardFailure yes" -f \
	-R {{ ctl_port }}:localhost:{{ local_port }} \
	-R {{ job_port }}:localhost:{{ fwd_port }} \
	{{ ssh_host }} \
	"R --no-save --no-restore -e \
	'clustermq:::ssh_proxy(ctl={{ ctl_port }}, job={{ job_port }})' \
	> {{ ssh_log | /dev/null }} 2>&1"

it seems that reverse tunnelling might be required. I do not know whether the ssh package supports this, tagging @jeroen.

Would be great if we could use the ssh package as fallback on Windows.