Best Practice for HTTP Proxy Support?

[allquixotic]

Over in jsonrpsee which depends on hyper, the jsonrpsee maintainers observed that hyper-proxy is lacking in continuing updates/maintainership.

This is probably of most concern related to keeping the dependencies up-to-date within the Cargo.toml of hyper-proxy, assuming the crate is feature-complete and doesn't have any known bugs that are the responsibility of hyper-proxy to fix. But still, the jsonrpsee maintainers would prefer not to depend on a crate that is going stale, and doubly so when it pulls in security-related crates like rustls.

My question is, do the maintainers of hyper (this crate) consider the hyper-proxy crate to be the best crate for adding proxy middleware to a hyper HTTP client? Or are you planning on subsuming this functionality into the hyper crate itself? Or is there another crate that you would recommend that has a better maintainership record than hyper-proxy?

The jsonrpsee maintainers alluded to possibly writing this functionality themselves within the jsonrpsee crate if there's no better way, but that seems like a layering violation (a library that implements a proto on top of HTTP(S) as well as other transports shouldn't have to reinvent how to work with HTTP proxies).

My preference would be for the hyper maintainers to identify a supported way to add an HTTP proxy to a client. Then I can revise my pull request to jsonrpsee to use that recommended approach.

[seanmonstar]

My question is, do the maintainers of hyper (this crate) consider the hyper-proxy crate to be the best crate for adding proxy middleware to a hyper HTTP client? Or is there another crate that you would recommend that has a better maintainership record than hyper-proxy?

I can't comment to the quality of various crates that depend on hyper. There's too many for us to monitor. And often they have subtle differences that meet different needs of the creator. Thousands (millions?) of proxies exists, all different, because of individual needs.

Or are you planning on subsuming this functionality into the hyper crate itself?

No, there are no plans to put that directly in hyper. As mentioned, there's too many subtle differences. hyper's goal is to provide a safe and fast HTTP implementation, and let others build what they need on top.

[allquixotic]

Thanks for your response!

Thousands (millions?) of proxies exists, all different, because of individual needs.

Oh, sorry, I wasn't asking for hyper to implement the functionality of an HTTP proxy. I was just asking if hyper planned to support sending traffic through a proxy as part of an HTTP request. :)

Examples of other HTTP implementations that support the client specifying they'd like the traffic sent through a CONNECT proxy:

• Java SE, builtin
• C#, .NET Core builtin
• C/GLib via libsoup
• Qt/C++
• C/libcurl
• Python via requests
• Ruby, standard library
• Nim, std library
• Golang, std library
• Swift, Foundation API

Of all the 10 most popular languages, the only one in which I couldn't find easy HTTP proxy support in was JavaScript. And there are definitely packages in npm that support this; I just couldn't find it in 5 seconds of googling, and it's not in the core API.

I realize there are different types of proxies, like SOCKS vs HTTP; some implementations support TLS to the proxy server as well as connecting to https destinations; and some require authentication. There are also many, many different ways for operating systems to specify "global" proxy settings that are desired to be applied to all programs.

However, there's no need to implement every conceivable proxy protocol and configuration. In my experience, proxy server configuration is converging on a very small subset of protocols, with plain HTTP being by far the most popular, and SOCKS5 as less common but still in use. Everything else is so uncommon that it's probably not worth talking about.

Some of the "big hitters" that are used extremely often in rock-solid stable, production APIs from the above include the Java, C#, and libcurl APIs, which all support at least HTTP proxies, and some also support SOCKS.

I'm just an application developer. I wish I didn't have to choose between the safety and performance of Rust, and a basic feature like HTTP proxy support in Rust's most popular HTTP library. The Rust ecosystem is really excellent, and has nearly everything I need out of the box. Most crates (including hyper itself) come with batteries included, i.e. all the features you need are already there, to minimize the code an app developer has to write.

This is just an edge case where an extremely common feature seems to have been left out. The demand was high enough that someone built a crate with this functionality tacked onto hyper, consisting of 871 SLOC of Rust (including empty lines). But I think the feature would have a better chance of long-term maintainership under the hyperium/hyper umbrella rather than where it sits now.

Thanks for reading and considering :)

[seanmonstar]

FWIW, the reqwest crate does provide a way to configure various rules and kinds of proxies. That crate is considered a more-opinionated batteries-included HTTP client.

hyper chooses not to include too many batteries because people have different opinions on how best to implement them, with various pros and cons. You could think of hyper as Node's http module: it's meant to be a stable core, to build on top of.

[allquixotic]

Nice. I see that reqwest does depend on hyper as well. Mainly I'm married to hyper because jsonrpsee depends directly on hyper (instead of reqwest, which might have been a better choice for them, perhaps).

I guess it's between me and the jsonrpsee maintainers to work out, but it looks like my app is going to move forward using a fork of jsonrpsee that patches in HTTP proxy support using the hyper-proxy crate for now. I'd rather directly use upstream libraries, but I like the API binding semantics of jsonrpsee too much to stop using that crate, and I really need the http proxy support at least for development (it may not be necessary in production; it's not yet clear).

[niklasad1]

Fair enough, I think we could migrate to reqwest instead but would be cool to extract the reqwest proxy stuff to a maintained repo :)