fork and maintain node-http-proxy / http-proxy

[williamstein]

This is somewhat similar in spirit to to #8272

We use the node-http-proxy library for proxying, and have many years. Unfortunately, it's totally unmaintained now , despite having almost 15 million downloads a week. There's 106 open PR's and 509 open issues, and there was exactly one patch release in the last 10 years, over 4 years ago. From the issue tracker, it looks like there are no other similar libraries that handle websockets. Most interestingly, this PR fixes "a major memory leak" in websocket proxying. The leak wasn't a problem before node 15.6, but a change in nodejs around then broke things for all node versions thereafter. This one little bug known (with a fix) since 2021 has been significantly negatively impacting CoCalc since whenever we upgraded to Node 15.6.

JupyterHub also had the exact same problem: jupyterhub/configurable-http-proxy#434 They fixed it by using this fork:

https://github.com/Jimbly/http-proxy-node16

I just skimmed the source code of hub-proxy, and it's relatively small (e.g., ~500 SLOC total). Not surprisingly, there are many high security vulnerabilities in dependencies though. Since nobody else will properly maintain this package, when we get time we should fork it and maintain a package with up to date dependencies, and also cherry pick some of the fixes from the PR's. When we do, we should advertise our fork here:

http-party/node-http-proxy#1687

If we make a fork, it would also be good to welcome the JupyterHub devs to use it (and/or contribute).

Any suggestions for what to call such a fork?

• http-proxy-2 might be available: https://www.npmjs.com/package/http-proxy-2

[haraldschilly]

There are also tickets about other types of leaks, e.g. http-party/node-http-proxy#1586

[haraldschilly]

Coming from this memory leak test example → I saw it uses a fork. So, maybe http-proxy-node16 is a good starting point for websocket related improvements. (npmjs)

[williamstein]

Started a rewrite here based on http-proxy-node16

https://github.com/sagemathinc/http-proxy-3

and

https://www.npmjs.com/package/http-proxy-3

This is certainly not ready for production or anything like that yet, because we need to rewrite the examples and tests and ensure they all work. But it does work for proxying jupyterlab and nats already in cocalc, which is a good sign.

[williamstein]

#8329