Updates to the HPN-SSH pages.

hpnssh/HPNSSHInstallation.md

@@ -1,21 +1,16 @@
 ## HPN-SSH Installation
-
-<p>The process of installing HPN-SSH from source is a relatively painless process
+The process of installing HPN-SSH from source is a relatively painless process
 but does have some nuances. This document will go through the process step by 
 step to help you get the most from your installation. If you find any errors 
 please contact us at [email protected].
-</p>
 
 ### Step 1: Get the source code
-
 The official repository for HPN-SSH is found at 
 https://github.com/rapier1/hpn-ssh. Get a copy with
-<pre>
-git clone https://github.com/rapier1/hpn-ssh
-</pre>
+>`git clone https://github.com/rapier1/hpn-ssh`
 
-### Step 2: Install dependencies
 
+### Step 2: Install dependencies
 * What you need to install is dependent on your distribution but will include:
 * OpenSSL development package
 	* Debian: libssl-dev
@@ -29,9 +24,7 @@ git clone https://github.com/rapier1/hpn-ssh
 * Autoconf
 * Automake
 
-
 ### Step 3: Install optional dependencies
-
 This optional libraries will extend the functionality of HPN-SSH to allow the use of PAM 
 authentication, Kerberos, graphical password tools, etc. 
 
@@ -41,27 +34,18 @@ authentication, Kerberos, graphical password tools, etc.
 
 
 ### Step 4: Build the configure file
-
-> `generate ./configure with “autoreconf -f -i"`
-
-
+>  `autoreconf -f -i`
 
 ### Step 5: Configuration
-
 Configure the installation. You can get detailed information on how to do this by 
-issuing `./configure --help`. However, commonly you will want to change the default installation location of the binaries. This can be done with `--prefix=/[desired_path]`. For example, if you want the binaries installed into `/usr/bin` as opposed to the default of /usr/local/bin you’d use `./configure --prefix=/usr`. Other common options would be to 
-incorporate pam, kerberos, alternative SSL libraries, and so forth. However, for most users 
+issuing `./configure --help`. However, commonly you will want to change the default installation location of the binaries. This can be done with `--prefix=/[desired_path]`. For example, if you want the binaries installed into `/usr/bin` as opposed to the default of /usr/local/bin you’d use `./configure --prefix=/usr`. Other common options would be to incorporate pam, kerberos, alternative SSL libraries, and so forth. However, for most users 
 either no additional configuration options or modifying the prefix will suffice. 
 
-
 ### Step 6: Make
-
-Make the application with `make -j[<em>num cores</em>]`. So if you have an 8 core system 
+Make the application with `make -j[num cores]`. So if you have an 8 core system 
 you’d use `make -j8`
 
-
 ### Step 7: Installation
-
 After HPN-SSH successfully builds, install it with `sudo make install`. This will install the binaries, configuration files, and generate the unique host keys used. At this point you can make changes to the ssh client and server default configuration. These files are 
 found, generally, in /etc/hpnssh/ssh_config and sshd_config respectively. You may want to 
 change the default port from 2222 to some other value. You may also want to enable the 

hpnssh/homepage.md

@@ -3,53 +3,67 @@
 HPN-SSH is a series of modifications to OpenSSH, the predominant implementation of the ssh protocol. It was originally developed to address performance issues when using ssh on high speed long distance networks (also known as Long Fat Networks: LFNs). By taking advantage of automatically optimized receive buffers HPN-SSH could improve performance dramatically on these paths. Later advances include; disabling encryption after authentication to transport non-sensitive bulk data, modifying the AES-CTR cipher to use multiple CPU cores, more detailed connection logging, and peak throughput values in the scp progress bar. More information can be found on HPN-SSH page on the PSC website.
 
 ## Our Latest Release
-The latest release of HPN-SSH is 18.1.0 built against OpenSSH 9.4. This version includes a much improved ChaCha20-Poly1305 cipher, which is the default for both HPN-SSH and OpenSSH. The implemtation of the cipher is parallized to move the hard work of computing the keystream to independent threads. It also uses an optimized version of the Poly1305 methods found in OpenSSL. In comparison to OpenSSH 9.4 this version is 59% faster. 
+The latest release of HPN-SSH is 18.6.2 built against OpenSSH 9.9p2. This version includes a much improved ChaCha20-Poly1305 cipher, which is the default for both HPN-SSH and OpenSSH. The implemtation of the cipher is parallized to move the hard work of computing the keystream to independent threads. It also uses an optimized version of the Poly1305 methods found in OpenSSL. In comparison to OpenSSH 9.4 this version is 59% faster. 
 
-Get the latest source code release from [https://github.com/rapier1/hpn-ssh ](). Packages for Ubuntu and Fedora will be availble from the PPAs listed below. 
+Get the latest source code release from [https://github.com/rapier1/hpn-ssh ](). 
+
+Packages for Ubuntu, Debian, and Fedora are available!  
+
+Ubuntu:
+
+>`sudo add-apt-repository ppa:rapier1/hpnssh`
+
+Fedora RPMs can be added with:
+>`sudo dnf copr enable rapier1/hpnssh`
+
+Debian packages:
+
+[https://download.opensuse.org/repositories/home:/rapier1/]()
+
+## Do you use HPN-SSH?
+Let us know by sending a quick note to [[email protected]](mailto:[email protected]). Knowing that you are using hpn-ssh and how you are using it lets us figure out where future development should go. 
 
 ## Developing the next version of HPN-SSH
 
-The developers of HPN-SSH at the Pittsburgh Supercomputing Center (PSC) have recently received a grant from the National Science Foundation to develop and incorporate new features and optimizations. This grant will provide direct support to developers at PSC for two years. The goal of this grant (NSF Award#: 2004012) is to provide HPN-SSH with the level of performance required in modern high performance computing.
+The developers of HPN-SSH at the Pittsburgh Supercomputing Center (PSC) were awarded a grant in 2020 from the National Science Foundation to develop and incorporate new features and optimizations. This grant provided direct support to developers at PSC for four years. The goal of this grant (NSF Award#: 2004012) was to provide HPN-SSH with the level of performance required in modern high performance computing. We are currently competing for a new grant and we'll let you know if and when we get it. We have some exciting ideas in store.
 
-## What are you working on?
+## What did you work on?
 
-We’ve identified six different areas where we would like to focus our efforts. This is not meant to be an exhaustive list but is more of a starting point for our deliverables. Depending on community input this list may change to develop advances of highest interest. The six initial proposed areas of work are:
+We identified six different areas where we focused our efforts. This wass not meant to be an exhaustive list but is more of a starting point for our deliverables. The six initial proposed areas of work were:
 
 **Automatically resume failed transfers:** There is nothing quite as frustrating as having scp or sftp fail in the middle of a large transfer. Currently ssh does not have a mechanism to allow for failed transfers to restart from the point of failure. HPN-SSH is proposing to develop a mechanism to reliably resume failed transfers. We expect to do this by computing a hash of the partial file and compare it to a corresponding byte range of the original file. If these match then HPN-SSH will append the missing information to the partial file. If they do not match then the entire file will be transferred.
 
-This is available as of HPN-SSH 15v4 for OpenSSH 8.7.
+*This is available as of HPN-SSH 15v4 for OpenSSH 8.7.*
 
 **Incorporate AES-NI into the AES-CTR cipher:** The AES-NI instruction set is a set of on die instructions that use hardware acceleration to increase the performance of common AES functions. The result is that on CPUs that support AES-NI the default AES-CTR cipher is faster than HPN-SSH’s multithreaded cipher. We will work on incorporating AES-NI into the multithreaded cipher. We expect that this will allow for faster transfers when ssh is CPU bound.
 
-This is available as of HPN-SSH 15v4 for OpenSSH 8.7.
+*This is available as of HPN-SSH 15v4 for OpenSSH 8.7.*
 
 **Parallelization of CHACHA20 cipher:** CHACHA20 is a fast secure cipher that is the current default for OpenSSH. Initial investigation indicates that CHACHA20 can be transformed into a multithreaded cipher. This will allow the workload to be distributed across more CPU cores and should allow for faster transfers. We believe this will be important in situations where multiple users are simultaneously transferring files to the same host.
 
-This is available as of HPN-SSH 18.1.0 for OpenSSH 9.4. 
+*This is available as of HPN-SSH 18.1.0 for OpenSSH 9.4. *
 
 **Inline Network Telemetry:** Sometimes figuring out why a ssh connection is underperforming is a difficult task. To help with diagnostics HPN-SSH will deploy network telemetry. In this diagnostic mode both the client and server will periodically query network statistics (such as retransmits, out of order packets, time spent buffer limited, and so forth) and store this data for analysis. This data may also be periodically displayed to the user. Initially this will be limited to Unix variants that have the TCP_INFO struct (Linux and *BSD).
 
-This is available as of HPN-SSH 15v5 for OpenSSH 8.8.
+*This is available as of HPN-SSH 15v5 for OpenSSH 8.8.*
 
 **Pipelining HMAC generation:** The Hash-based Message Authentication Code (HMAC) is a one way cryptographic hash used by ssh to ensure that a datagram has not been modified en-route between the hosts. This ensures that the data has not been subjected to a man in the middle attack. In OpenSSH this is a step in a very linear process. No other work can be conducted (such as encrypting other data) while the HMAC is being computed. In many cases this can act a bottleneck on throughput. HPN-SSH is proposing to pipeline this process in order to mitigate this bottleneck as much as possible.
 
 Update: The primary method of computing the MAC now happens after encryption which prevents pipelining for working effectively. However, we've been able to significantly improve the performance of the UMAC used for the AES counter cipher. In our tests this resulted in a 20% performance improvement. However, this requires HPN-SSH being bult against OpenSSL 3.0. We've also improved the performance of the Poly1305 cipher used by the default cipher of Chacha20 by using assembly, as oppessed to C, to generate the MAC. While the assembler isn't supported on every platform it will fall back to using the C code if it isn't. 
 
 **Packaging and Distribution:** HPN-SSH was, for a very long time, only available as a series of patches. Later it became a GitHub repo. This turned out to be a non-optimal method of distributing HPN-SSH to the public. With this in mind we will be working to provide precompiled packages for a variety of operating systems and Linux distributions and the creation of canonical package repositories (such as PPAs). We will also be reaching out to distribution maintainers to make HPN-SSH an option for all of their users.
 
-Our first online repo is now available for Ubuntu. Add it with:
-
->`sudo add-apt-repository ppa:rapier1/hpnssh`
-
-Fedora RPMs can be added with:
->`sudo dnf copr enable rapier1/hpnssh`
+*This is an ongoing process.*
 
 ## What can I do to help? 
 
 ### Join our HPN-SSH community mailing list 
 
 Stay up-to-date on progress and improvements to HPN-SSH by [joining this list, intended for both developers and users](https://lists.psc.edu/mailman/listinfo/hpnssh-community).
 
+### Submit big reports, code improvements, ideas, and problems
+You can email use directly at [[email protected]](mailto: [email protected]), join our mailing list, or reach out to us through our [GitHub repository](https://github.com/rapier1/hpn-ssh). GitHub is a great place to let us know about bugs, issues, or just to start a discussion. 
+
 ### Make a donation
 
 If you care about HPN-SSH there is no better way to show your support than making a donation to the Pittsburgh Supercomputing Center. I do not personally receive any money from these donations but your support ends up supporting our work. Any amount is worth while - even a dollar will show PSC and CMU your support for our work. Seriously, show your support in order to both keep HPN-SSH current and fund new improvements.

hpnssh/papers-presentations.md

@@ -1,9 +1,11 @@
 
 
-- **"Application Layer Network Window Management in the SSH Protocol"** - This was the first write up about our work. It is important to note that this should be seen as an introduction to what we were working on at the time.
-- **SuperComputing 2004 Poster** - This is the full PDF of the poster we presented at the SuperComputing 2004 conference. This provides more detail, some graphs illustrating the problem, and some of our results.
-   - A text version of the above poster. No layout so it's easier to read.
-- **A presentation on HPN-SSH given at the Internet2 Spring 2005 Members Meeting.** A useful synopsis of what we've done and where we plan on going.
-- **"High Speed Bulk Data Transfer on the Grid Using the SSH Protocol"** - Submitted to the Gridnets 2007 conference.
-- **"High Speed Bulk Data Transfer Using the SSH Protocol"** - This is an updated and revised version of the above Gridnets paper that was accepted for publication in the proceedings of the Mardi Gras Conference 2008 sponsored by LSU and ACM SIGAPP
-- **"Enabling High Performance Bulk Data Transfers with SSH"** - the presentation of the above paper.
+- [**"Application Layer Network Window Management in the SSH Protocol"**](https://figshare.com/articles/journal_contribution/Application_Layer_Network_Window_Management_in_the_SSH_Protocol/4974977?file=8369528) - This was the first write up about our work. It is important to note that this should be seen as an introduction to what we were working on at the time.
+
+- [**"High Speed Bulk Data Transfer Using the SSH Protocol"**](https://dl.acm.org/doi/10.1145/1341811.1341824) - This is an updated and revised version of aprior paper submitted to the 2007 Gridnets conferemce. This version was part of the proceedings of the Mardi Gras Conference 2008 sponsored by LSU and ACM SIGAPP
+
+- [**2023 NSF PI Meeting Poster**](https://zenodo.org/records/6840900) - A poster on current progress from our 2020 NSF development grant. This was presented at a meeting of National Science Foundation Principal Investigators (PI).
+
+- [**HPN-SSH: A High Performance SSH for HPC**](https://figshare.com/articles/presentation/HPN-SSH_A_High_Performance_SSH_for_HPC/24530371?file=43088701) - a brief presentation on HPN-SSH delivered at the 2022 Internet 2 Technology Exchange in Denver Colorado. 
+
+There are quite a few more presentations and posters that have been written over the years but these are available online. As we find more legacy material and generate new data we'll update this page. 

hpnssh/readme.md

@@ -5,7 +5,7 @@ These options can be used on the command line in the typical '-o Option ' forma
 ## LibreSSL Support
 Changes in LibreSSL version 3.5 and 3.6 prevent the use of the threaded AES CTR cipher. 
 In those cases HPNSSH will fallback to the serial versionof the AES CTR cipher. A warning
-is printed to stderr. This shoudl be resolved in LibreSSL 3.7.
+is printed to stderr. This was resolved in LibreSSL 3.7.
 
 ## Automatic Port Fallback
 Starting with version 17v3 the hpnssh client now uses TCP port 2222 to connect automatically as this is the default hpnsshd port. However, we understand that many users will be end up connecting standard SSH servers on port 22. To make the easier for users the client will fall back to port 22 in the event that there is no hpnssh server running on port 2222. The behaviour can be modifed as follows:
@@ -93,7 +93,6 @@ will, after authentication disable both encryption *and* the MAC while transferr
 
 
 ## HPN specific options
-
 <dl>
 <dt>TcpRcvBuf=[int]KB client </dt> 
 <dd>Set the TCP socket receive buffer to <em>int</em> Kilobytes. It can be set up to the maximum socket size allowed by the system. This is useful in situations where the TCP receive window is set low but the maximum buffer size is set higher (as is typical). This works on a per TCP connection basis. You can also use this to artificially limit the transfer rate of the connection. In these cases the throughput will be no more than n/RTT. The minimum buffer size is 1KB . Default is the current system wide TCP receive buffer size.
@@ -132,6 +131,6 @@ the multithreaded MT-AES-CTR cipher. Set to no by default.
 This patch was conceived, designed, and led by Chris Rapier ([email protected]) The majority of the actual coding for versions up to HPN12v1 was performed by Michael Stevens ([email protected]). The MT-AES-CTR cipher was implemented by Ben Bennet ([email protected]) and improved by Mike Tasota ([email protected]) an NSF REU grant recipient for 2013. Mitchell Dorrell ([email protected]) has provided invaluable assistance on improving the performance of ChaCha20 and Poly1305 along with other critical work. Allan Jude provided the code for the NoneMac and buffer normalization. This work was financed, in part, by Cisco System, Inc., the National Library of Medicine, and the National Science Foundation.
 
 ### Sponsors: 
-Thanks to Niklas Hambuchen for being the first sponsor of HPN-SSH via github's sponsor program! Join our sponsors at <a href="https://github.com/sponsors/rapier1">our github</a>, view our <a href="https://www.psc.edu/hpn-ssh-home/support/">funding page</a>, or contact at hpnssh@psc.edu. 
+Thanks to Niklas Hambuchen for being the first sponsor of HPN-SSH via github's sponsor program! Join our sponsors at <a href="https://github.com/sponsors/rapier1">our github</a>, view our <a href="https://www.psc.edu/hpn-ssh-home/support/">funding page</a>, or contact at [hpn-ssh@psc.edu](mailto: [email protected]).