Any plans to implement it in Rust?

[Numenorean]

I see you also developing in rust, so any plans to reimplement same functionality(or like a lite version) in rust(probably using boringssl or openssl)?

[gaukas]

Actually I liked and highly support this idea.

If it is challenging to build a Rust library from rustls that supports ALL features in uTLS, at least it would be valuable to have a library to allow customization to a certain extent, or being able to (almost) perfectly mimic one, or a few, popular TLS implementations/browsers.

[gaukas]

Any early proposal or project draft is welcome and wanted!

[Gowee]

I patched rustls for that purpose for noisy-shuttle. To keep to the changes minimum, I add a new rustls::ClientConnection::new_with that takes a custom client random, a custom session id and a function as an additional param which takes a rustls::Message and modifies it per fingerprints. The implementation isn't porting features from utls while solely exists as a simplified alternative for my needs. And now it is a little outdated.

Porting utls to Rust would be a great idea. I am willing to contribute if anyone is gonna start such. Or I could start a project myself, but I may not be able to maintain it in a timely fashion on my own.

[gaukas]

Thanks for sharing @Gowee! Your fork looks great and indeed contains many features that a rutls library could benefit from.

I am willing to contribute if anyone is gonna start such.

Sure, I don't mind kick this off by forking a popular and mostly-complete TLS library in Rust. Is Rustls a good starting point or what are the other choices? I'm not super familiar with the TLS ecosystem in Rust.

I may not be able to maintain it in a timely fashion on my own.

That is totally fine. Either way I don't believe on the refraction side we have enough capacity for rutls, so it will be more likely a community driven project. We could start it here under refraction's github org for now and decide its fate later.

---

Or utlsr, open for ideas on naming!

[RPRX]

对 Rust 来说，或许直接使用 Chromium 的代码？XTLS/Xray-core#2257 (comment)

---

For Rust, maybe just use the Chromium code? XTLS/Xray-core#2257 (comment)

[raintean]

对 Rust 来说，或许直接使用 Chromium 的代码？XTLS/Xray-core#2257 (comment)

其实尝试做一个cronet的rust binding就OK了. 这个对于rust来说不难.
但是我在做的过程中突然想到一个问题, 我们很难在同一个程序里面提供多个版本的cronet.
换句话说, 这个库强对应一个chrome的版本. 这个就不是很科学了.

---

For Rust, maybe just use Chromium's code? XTLS/Xray-core#2257 (comment)

Actually try to make a rust binding for cronet and you'll be fine. It's not hard for rust.
But one thing that occurred to me while I was doing it is that it is very difficult to provide multiple versions of cronet in the same program.
In other words, the library has to be compatible with one version of Chrome. That's not very scientific.

[fedosgad]

AFAIK, @mhils (author of mitmproxy) has looked for Rust analog of utls around time of mitmproxy v8 release. Maybe he knows something useful about such libraries?

[gaukas]

Thanks for tagging/linking @fedosgad. Out of curiosity, (how) did mitmproxy solve the TLS ClientHello fingerprinting issue in the end?

[mhils]

We don't have ClientHello mimicry in mitmproxy yet. Right now we're using OpenSSL via pyca/cryptography, which makes this pretty hard to implement. I looked into switching to rustls a while ago, but that attempt was abandoned due to compatibility issues (unrelated to mimicry) back then.

[gaukas]

we're using OpenSSL via pyca/cryptography

Does it indicate that mitmproxy produces a relatively unique/unpopular ClientHello fingerprint or not? (Better if would like to provide it in tlsfingerprint.io's format, you may pull from https://client.tlsfingerprint.io/, so we could examine how relatively popular it is)

compatibility issues

I would entertain your insights on this if you would love to elaborate. What kind of compatibility issue and if there are any other good paths Rust (and potentially other programming language) programmer should take to counter the ClientHello fingerprinting?

i.e., if not then perhaps we should try to start building a rustls-based rutls, or alternatively, as @RPRX and @raintean mentioned here, a chromium/cronet Rust binding.

(Disclaimer: refraction is not against such project, but unlikely to spare any capacity on leading/moderating the project. I would suggest starting a new organization and adding interested developers in, unless there is already something we could contribute to.)

[mhils]

Does it indicate that mitmproxy produces a relatively unique/unpopular ClientHello fingerprint or not? (Better if would like to provide it in tlsfingerprint.io's format, you may pull from https://client.tlsfingerprint.io/, so we could examine how relatively popular it is)

$ mitmdump --mode reverse:https://client.tlsfingerprint.io/ &
$ curl -k http://localhost:8080
{"tls_record_version":769,"tls_handshake_version":771,"cipher_suites":[4866,4867,4865,49195,49199,49196,49200,52393,52392,158,159,52394,49187,49191,49161,49171,49188,49192,49162,49172,103,107,156,157,60,61,47,53,255],"compression_methods":[0],"extensions":[0,11,10,35,22,23,13,43,45,51],"extensions_normalized":[0,10,11,13,22,23,35,43,45,51],"server_name":"client.tlsfingerprint.io","supported_groups":[29,23,30,25,24,256,257,258,259,260],"ec_point_formats":[0,1,2],"signature_algorithms":[1027,1283,1539,2055,2056,2057,2058,2059,2052,2053,2054,1025,1281,1537,771,769,770,1026,1282,1538],"alpn":null,"compress_certificate":null,"record_size_limit":[],"supported_versions":[772,771],"psk_key_exchange_modes":[1],"key_share":[29],"application_settings":null,"user_agent":"curl/7.81.0","nid":5075520528980946569,"norm_nid":-7307055319671900602,"id":"466fdf07a41cae89","norm_id":"9a981fe6b4d3f646"}

I would entertain your insights on this if you would love to elaborate. What kind of compatibility issue and if there are any other good paths Rust (and potentially other programming language) programmer should take to counter the ClientHello fingerprinting?

One major blocker for us back then was the lack of support for IPs as dnsAltNames. Rustls supports that nowadays though. :)
Another issue for us was the lack of support for older TLS versions. We have users who put mitmproxy in front of outdated IoT devices to upgrade them to a recent TLS version. Typically not a concern otherwise.

[gaukas]

$ mitmdump --mode reverse:https://client.tlsfingerprint.io/ &
$ curl -k http://localhost:8080
{"tls_record_version":769,"tls_handshake_version":771,"cipher_suites":[4866,4867,4865,49195,49199,49196,49200,52393,52392,158,159,52394,49187,49191,49161,49171,49188,49192,49162,49172,103,107,156,157,60,61,47,53,255],"compression_methods":[0],"extensions":[0,11,10,35,22,23,13,43,45,51],"extensions_normalized":[0,10,11,13,22,23,35,43,45,51],"server_name":"client.tlsfingerprint.io","supported_groups":[29,23,30,25,24,256,257,258,259,260],"ec_point_formats":[0,1,2],"signature_algorithms":[1027,1283,1539,2055,2056,2057,2058,2059,2052,2053,2054,1025,1281,1537,771,769,770,1026,1282,1538],"alpn":null,"compress_certificate":null,"record_size_limit":[],"supported_versions":[772,771],"psk_key_exchange_modes":[1],"key_share":[29],"application_settings":null,"user_agent":"curl/7.81.0","nid":5075520528980946569,"norm_nid":-7307055319671900602,"id":"466fdf07a41cae89","norm_id":"9a981fe6b4d3f646"}

https://tlsfingerprint.io/id/N/9a981fe6b4d3f646

{"tls_record_version":769,"tls_handshake_version":771,"cipher_suites":[4866,4867,4865,49195,49199,49196,49200,52393,52392,158,159,52394,49187,49191,49161,49171,49188,49192,49162,49172,103,107,156,157,60,61,47,53,255],"compression_methods":[0],"extensions":[0,11,10,35,16,22,23,13,43,45,51],"extensions_normalized":[0,10,11,13,16,22,23,35,43,45,51],"server_name":"client.tlsfingerprint.io","supported_groups":[29,23,30,25,24,256,257,258,259,260],"ec_point_formats":[0,1,2],"signature_algorithms":[1027,1283,1539,2055,2056,2057,2058,2059,2052,2053,2054,1025,1281,1537,771,769,770,1026,1282,1538],"alpn":["h2","http/1.1"],"compress_certificate":null,"record_size_limit":[],"supported_versions":[772,771],"psk_key_exchange_modes":[1],"key_share":[29],"application_settings":null,"user_agent":"curl/7.81.0","nid":-8632760564826273281,"norm_nid":1256227648061341672,"id":"883246012e8789ff","norm_id":"116f0463dfb87fe8"}

https://tlsfingerprint.io/id/N/116f0463dfb87fe8

Unfortunately it does look like relatively unique and niche, with no known User Agents to us at our database... Our observation could be biased since it is from a university.

[mhils]

Thanks. I can also offer

{"tls_record_version":769,"tls_handshake_version":771,"cipher_suites":[4866,4867,4865,49195,49199,49196,49200,52393,52392,158,159,52394,49187,49191,49161,49171,49188,49192,49162,49172,103,107,156,157,60,61,47,53,255],"compression_methods":[0],"extensions":[0,11,10,35,16,22,23,13,43,45,51],"extensions_normalized":[0,10,11,13,16,22,23,35,43,45,51],"server_name":"client.tlsfingerprint.io","supported_groups":[29,23,30,25,24,256,257,258,259,260],"ec_point_formats":[0,1,2],"signature_algorithms":[1027,1283,1539,2055,2056,2057,2058,2059,2052,2053,2054,1025,1281,1537,771,769,770,1026,1282,1538],"alpn":["h2","http/1.1"],"compress_certificate":null,"record_size_limit":[],"supported_versions":[772,771],"psk_key_exchange_modes":[1],"key_share":[29],"application_settings":null,"user_agent":"curl/7.81.0","nid":-8632760564826273281,"norm_nid":1256227648061341672,"id":"883246012e8789ff","norm_id":"116f0463dfb87fe8"}

which you get if mitmproxy sends an ALPN because the client sent one. 😅

But yeah - I expect mitmproxy to be easily fingerprintable right now. I'd be happy to fix that eventually, but I'm prioritizing paid work at the moment. :)

[RPRX]

从工作量的角度来说，直接绑定 Chromium 的网络栈前期可能麻烦些，但后期的维护成本比较低，且可以获得比较好的效果。

修改 rustls，前期比较简单，但后期的维护成本比较高，会遇到一些 utls 遇到过的问题：

• 若 rustls 不支持 BoringSSL 的某些新功能，我们得自己实现
• 一些行为也要参考 Chrome 去进行修改
• 即使这样，还是会有很多细节无法做到一致

所以我个人倾向于直接绑定 Chromium 的网络栈，且由于它的维护成本低，放在该 org 下应该也不会有太大的负担。

---

From a workload perspective, directly binding to Chromium's web stack can be a bit of a pain up front, but it's cheaper to maintain later on and you can get better results.

Modifying rustls is easier up front, but more expensive to maintain later, and you'll run into some of the problems that utls has encountered:

• If rustls doesn't support some of BoringSSL's new features, we'll have to implement them ourselves.
• Some behaviors have to be modified with reference to Chrome.
• Even then, there are still a lot of details that are not consistent.

So my personal preference is to bind directly to Chromium's web stack, and since it's low-maintenance, it shouldn't be too much of a burden to put under that org.

[fedosgad]

But yeah - I expect mitmproxy to be easily fingerprintable right now. I'd be happy to fix that eventually, but I'm prioritizing paid work at the moment. :)

I can confirm that mitmproxy is easily fingerprintable - there are DDoS protection services (Cloudflare, for example) that use ClientHello fingerprints to do their job. I have encountered websites that do not allow user to access them when using mitmproxy but work fine without it. That was the primary reason why I wrote mirror_proxy a while ago.

[RPRX]

Actually try to make a rust binding for cronet and you'll be fine. It's not hard for rust. But one thing that occurred to me while I was doing it is that it is very difficult to provide multiple versions of cronet in the same program. In other words, the library has to be compatible with one version of Chrome. That's not very scientific.

这应该不成问题，通常我们只用最新的 Chrome 指纹，你的库有计划公开吗，大概什么时候可以发布？@raintean

---

That shouldn't be a problem, usually we only use the latest Chrome fingerprints, do you have any plans to make your library public and roughly when will it be released?

[raintean]

这应该不成问题，通常我们只用最新的 Chrome 指纹，你的库有计划公开吗，大概什么时候可以发布？@raintean

@RPRX 目前只是个人实验项目, 暂时没有公开的计划.

---

This shouldn't be a problem, usually we only use the latest Chrome fingerprints, do you have any plans to make your library public and roughly when will it be released?

It's just a personal experiment at the moment, no plans to make it public.

[Gowee]

Some relevant discussions are here: rustls/rustls#1125, rustls/rustls#1421.

[noorjs]

Building a wrapper for cronet only makes sense for the short term since it will be easier to maintain, there is already some open sourced code to help get started. However, building an independent TLS lib, i.e. forking boringSSL, rustls, etc makes the most sense for the long term. Most browsers are using some variation of openSSL, so maintaining a fork of openSSL (or boringSSL) with features found in uTLS seems to be the end game.

[gaukas]

A possibly off-topic update to this issue: at Refraction Networking we have recently created water (it is under me only because it is yet to be cleaned up and formally adopted by Refraction Networking) and it has a twin brother in Rust water-rs.

If everything goes well, we will have the ability to build uTLS into a WATER-compatible, pluggable WebAssembly Transport Module (WATM) and use it in Rust with water-rs.

For now, we are blocking on TinyGo and Go.

• TinyGo: lacks ability to correctly compile crypto/tls: https://tinygo.org/docs/reference/lang-support/stdlib/#cryptotls, which has only less dependencies than uTLS itself.
• Go: lacks ability to export WebAssembly functions (pending proposals available on //go:wasmexport directive)

If either one of these two could be unblocked at some point, we can advance our progress on building uTLS.wasm as an alternative solution to uTLS in Rust.

---

Update: We found a workaround and are able to build uTLS to a WebAssembly Transport Module which can be used by water. (12MB)

[gaukas]

Update: We found a workaround and are able to build uTLS to a WebAssembly Transport Module which can be used by water. (12MB)

Another update: We were able to further reduce the size down to ~3MB. See watm's release.

[kyochikuto]

Good work on watm, but aren't projects such as restls and noisy-shuttle already utilizing JA3 for this purpose in Rust?

[gaukas]

as restls and noisy-shuttle

I'm not familiar with noisy-shuttle, but I'm very sure restls implemented the client in Go to use uTLS's TLS ClientHello parrots.

If you could point me to the TLS client implementation of noisy-shuttle, I would be very interested in taking a look.

[kyochikuto]

as restls and noisy-shuttle

I'm not familiar with noisy-shuttle, but I'm very sure restls implemented the client in Go to use uTLS's TLS ClientHello parrots.

If you could point me to the TLS client implementation of noisy-shuttle, I would be very interested in taking a look.

You're right restls does not depend on JA3 my mistake! But for noisy-shuttle, here's the fingerprint builder in opts.rs and the implementation seems to be in the fp.rs.

Project's README example indicates a raw format supply for fingerprints paramaters:

specifying a TLS fingerprint (chrome):

./noisy-shuttle client 127.0.0.1:1080 server.addr.example:443 www.example.com Teap0taa --tls-ja3 769,2570-4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,2570-0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-2570-21,2570-29-23-24,0 --tls-alpn h2,http/1.1 --tls-sigalgos 1027,2052,1025,1283,2053,1281,2054,1537 --tls-versions 2570,772,771 --tls-keyshare 2570

[gaukas]

Interesting! Thanks for the info, I will take a look into this, especially how they handle new/unimplemented extensions.

But also it seems that noisy-shuttle itself is not doing a "real" TLS handshake, since some non-deprecated fields in the client hello were overwritten for other purposes. Anyways I believe it should be at least partially meaningful towards a utls-rust.