Releases: wi1dcard/fingerproxy
v1.1.0
This release contains one new feature and one bugfix.
Fixed number of cipher suites and extensions in JA4
TLS certificates hot-reloading
Fingerproxy now watches TLS certificate files. Certificate will be hot-reloaded when the files change. This is useful on production environments where certificates might get renewed while fingerproxy runs for months.
Change log
48f9e55 (HEAD, tag: v1.1.0, origin/master) Watch TLS cert file changes to update certs when needed. (#20)
cb0ae8e Fix JA4 number of cipher suites and extensions (#19)
Contributors
Assets 16
- 17.2 MB
2024-08-23T12:42:20Z - 17.2 MB
2024-08-23T12:42:20Z - 17.2 MB
2024-08-23T12:42:20Z - 17.2 MB
2024-08-23T12:42:21Z - 17.2 MB
2024-08-23T12:42:21Z - 17.2 MB
2024-08-23T12:42:22Z - 17.2 MB
2024-08-23T12:42:22Z - 18.4 MB
2024-08-23T12:42:19Z - 18.4 MB
2024-08-23T12:42:19Z - 18.4 MB
2024-08-23T12:42:19Z -
2024-08-23T12:41:05Z -
2024-08-23T12:41:05Z - Loading
v1.0.1
v1.0.0
v0.6.1
v0.6.0
This release mainly improved the example echo-server. Thanks to @hellodword's idea and help.
Refactored Echo Server
The echo-server has been totally refactored in this release, added a new API /json/detail.
Just like wwhtrbbtt/TrackMe, /json/detail prints all raw info captured by fingerproxy, as well as the human-readable, parsed TLS cipher suites, extensions, etc. This helps debug and investigate.
Here is an example usage:
# download and start echo server
./echo-server_linux_amd64 -verbose
# in another terminal
curl https://localhost:8443/json/detail --insecure --http2 | jqExample output (click to open)
{
"detail": {
"metadata": {
"ClientHelloRecord": "FgMBAgABAAH8AwNlhx0inELmjHnGk5KYmt+kigFGk2b2xNjYo7AyU6dTlyBgdna7azCBa6kK22fUTW3DodYTAz4QHJwueJDe+1GCKAA+EwITAxMBwCzAMACfzKnMqMyqwCvALwCewCTAKABrwCPAJwBnwArAFAA5wAnAEwAzAJ0AnAA9ADwANQAvAP8BAAF1AAAADgAMAAAJbG9jYWxob3N0AAsABAMAAQIACgAWABQAHQAXAB4AGQAYAQABAQECAQMBBAAQAA4ADAJoMghodHRwLzEuMQAWAAAAFwAAADEAAAANADAALgQDBQMGAwgHCAgIGggbCBwICQgKCAsIBAgFCAYEAQUBBgEDAwMBAwIEAgUCBgIAKwAFBAMEAwMALQACAQEAMwAmACQAHQAgT3KA46yLJAIAA2p/Tr1yBq4o+qs5Zc07oIPCwyLJcA0AFQCyAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==",
"ConnectionState": {
"Version": 772,
"HandshakeComplete": true,
"DidResume": false,
"CipherSuite": 4865,
"NegotiatedProtocol": "h2",
"NegotiatedProtocolIsMutual": true,
"ServerName": "localhost",
"PeerCertificates": null,
"VerifiedChains": null,
"SignedCertificateTimestamps": null,
"OCSPResponse": null,
"TLSUnique": null
},
"HTTP2Frames": {
"Settings": [
{
"Id": 3,
"Val": 100
},
{
"Id": 4,
"Val": 10485760
},
{
"Id": 2,
"Val": 0
}
],
"WindowUpdateIncrement": 1048510465,
"Priorities": null,
"Headers": [
{
"Name": ":method",
"Value": "GET",
"Sensitive": false
},
{
"Name": ":scheme",
"Value": "https",
"Sensitive": false
},
{
"Name": ":authority",
"Value": "localhost:8443",
"Sensitive": false
},
{
"Name": ":path",
"Value": "/json/detail",
"Sensitive": false
},
{
"Name": "user-agent",
"Value": "curl/8.6.0",
"Sensitive": false
},
{
"Name": "accept",
"Value": "*/*",
"Sensitive": false
}
]
}
},
"user_agent": "curl/8.6.0",
"ja3": {
"Type": 22,
"Version": 769,
"MessageLen": 0,
"HandshakeType": 1,
"HandshakeLen": 0,
"HandshakeVersion": 771,
"SessionIDLen": 32,
"CipherSuiteLen": 62,
"CipherSuites": [
4866,
4867,
4865,
49196,
49200,
159,
52393,
52392,
52394,
49195,
49199,
158,
49188,
49192,
107,
49187,
49191,
103,
49162,
49172,
57,
49161,
49171,
51,
157,
156,
61,
60,
53,
47,
255
],
"ExtensionLen": 373,
"SNI": "localhost",
"SupportedGroups": [
29,
23,
30,
25,
24,
256,
257,
258,
259,
260
],
"SupportedPoints": "AAEC",
"AllExtensions": [
0,
11,
10,
16,
22,
23,
49,
13,
43,
45,
51,
21
],
"ReadableCipherSuites": [
"TLS_AES_256_GCM_SHA384 (0x1302)",
"TLS_CHACHA20_POLY1305_SHA256 (0x1303)",
"TLS_AES_128_GCM_SHA256 (0x1301)",
"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)",
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)",
"TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9f)",
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)",
"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)",
"TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xccaa)",
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)",
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)",
"TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9e)",
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)",
"TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x6b)",
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)",
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)",
"TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (0x67)",
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)",
"TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x39)",
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)",
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)",
"TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x33)",
"TLS_RSA_WITH_AES_256_GCM_SHA384 (0x9d)",
"TLS_RSA_WITH_AES_128_GCM_SHA256 (0x9c)",
"TLS_RSA_WITH_AES_256_CBC_SHA256 (0x3d)",
"TLS_RSA_WITH_AES_128_CBC_SHA256 (0x3c)",
"TLS_RSA_WITH_AES_256_CBC_SHA (0x35)",
"TLS_RSA_WITH_AES_128_CBC_SHA (0x2f)",
"TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0xff)"
],
"ReadableAllExtensions": [
"server_name (0x0)",
"ec_point_formats (0xb)",
"supported_groups (0xa)",
"application_layer_protocol_negotiation (0x10)",
"encrypt_then_mac (0x16)",
"extended_master_secret (0x17)",
"post_handshake_auth (0x31)",
"signature_algorithms (0xd)",
"supported_versions (0x2b)",
"psk_key_exchange_modes (0x2d)",
"key_share (0x33)",
"padding (0x15)"
],
"ReadableSupportedGroups": [
"x25519 (0x1d)",
"secp256r1 (0x17)",
"x448 (0x1e)",
"secp521r1 (0x19)",
"secp384r1 (0x18)",
"ffdhe2048 (0x100)",
"ffdhe3072 (0x101)",
"ffdhe4096 (0x102)",
"ffdhe6144 (0x103)",
"ffdhe8192 (0x104)"
]
},
"ja3_raw": "771,4866-4867-4865-49196-49200-159-52393-52392-52394-49195-49199-158-49188-49192-107-49187-49191-103-49162-49172-57-49161-49171-51-157-156-61-60-53-47-255,0-11-10-16-22-23-49-13-43-45-51-21,29-23-30-25-24-256-257-258-259-260,0-1-2",
"ja4": {
"Protocol": 116,
"TLSVersion": 772,
"SNI": 100,
"NumberOfCipherSuites": 31,
"NumberOfExtensions": 12,
"FirstALPN": "h2",
"CipherSuites": [
47,
51,
53,
57,
60,
61,
103,
107,
156,
157,
158,
159,
255,
4865,
4866,
4867,
49161,
49162,
49171,
49172,
49187,
49188,
49191,
49192,
49195,
49196,
49199,
49200,
52392,
52393,
52394
],
"Extensions": [
10,
11,
13,
21,
22,
23,
43,
45,
49,
51
],
"SignatureAlgorithms": [
1027,
1283,
1539,
2055,
2056,
2074,
2075,
2076,
2057,
2058,
2059,
2052,
2053,
2054,
1025,
1281,
1537,
771,
769,
770,
1026,
1282,
1538
],
"ReadableCipherSuites": [
"TLS_RSA_WITH_AES_128_CBC_SHA (0x2f)",
"TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x33)",
"TLS_RSA_WITH_AES_256_CBC_SHA (0x35)",
"TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x39)",
"TLS_RSA_WITH_AES_128_CBC_SHA256 (0x3c)",
"TLS_RSA_WITH_AES_256_CBC_SHA256 (0x3d)",
"TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (0x67)",
"TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x6b)",
"TLS_RSA_WITH_AES_128_GCM_SHA256 (0x9c)",
"TLS_RSA_WITH_AES_256_GCM_SHA384 (0x9d)",
"TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9e)",
"TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9f)",
"TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0xff)",
"TLS_AES_128_GCM_SHA256 (0x1301)",
"TLS_AES_256_GCM_SHA384 (0x1302)",
"TLS_CHACHA20_POLY1305_SHA256 (0x1303)",
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)",
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)",
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)",
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)",
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)",
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)",
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)",
"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)",
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)",
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)",
...Contributors
Assets 16
v0.5.0
This release improved docs and fixed missing priority field in HTTP2 fingerprint under certain scenario. Thanks to @hellodword.
Some timeouts have also been added to the CLI options, allowing users to modify as needed:
-reverse-proxy-flush-interval string
See https://pkg.go.dev/net/http/httputil#ReverseProxy.FlushInterval, equivalent to $REVERSE_PROXY_FLUSH_INTERVAL (default "100ms")
-timeout-http-idle string
See https://pkg.go.dev/net/http#Server.IdleTimeout, equivalent to $TIMEOUT_HTTP_IDLE (default "180s")
-timeout-http-read string
See https://pkg.go.dev/net/http#Server.ReadTimeout, equivalent to $TIMEOUT_HTTP_READ (default "60s")
-timeout-http-write string
See https://pkg.go.dev/net/http#Server.WriteTimeout, equivalent to $TIMEOUT_HTTP_WRITE (default "60s")
-timeout-tls-handshake string
Timeout for TLS handshakes, equivalent to $TIMEOUT_TLS_HANDSHAKE (default "10s")
f0b7407 (HEAD, tag: v0.5.0, origin/master) Add missing priority in the HEADER frame (#8)
b9869b3 Add help text for metrics.
21a2725 Implement HTTP and TLS timeouts in CLI options.
0bb5584 Improve godoc.
Contributors
Assets 9
v0.4.1
v0.4.0
This release introduced a few new CLI flags, error log improvements, and two breaking changes.
Notable changes listed below.
Breaking Change
In order to keep the code simple and "one and only one solution for one problem", in commit c2473c8, public variable GetReverseProxyHandler and methods DefaultTLSConfig(), StartPrometheusClient(), DefaultReverseProxyHTTPHandler(), DefaultProxyServer() have now been removed.
Now with fingerproxy CLI, you can only customize fingerprinting algorithm via GetHeaderInjectors. To customize reverse proxy or TLS server parameters, see Use as a Library in README.md.
Breaking Change of reverseproxy Package
HTTPHandler.ReverseProxy and HTTPHandler.SetReverseProxyRewriteFunc has been removed in commit b6a145d.
In fact, HTTPHandler.ReverseProxy is immutable. While calling NewHTTPHandler, a rewrite function in ReverseProxy is set to support header injection. Changing internal ReverseProxy disables rewrite, which can cause header injectin malfunction.
Clearer Error Log
Some TLS server error logs such as EOF, connection reset by peer, and TLS handshake timed-out, now have been suspended in commit ed4366f.
These errors are usually caused by clients or unstable network. To see these logs, specify --verbose to enable verbose logs.
More Configurable Options in CLI
New flag --duration-metric-buckets, --preserve-host are available. Run fingerproxy --help to get more information.
Fix fingerprint_duration Metric Prefix
The default metrics prefix fingerproxy_ was missed in fingerprint_duration metric. Now it has been added. Fingerproxy metrics now look like:
fingerproxy_fingerprint_duration_seconds_bucket
fingerproxy_fingerprint_duration_seconds_count
fingerproxy_fingerprint_duration_seconds_sum
fingerproxy_requests_total
...
Production-Ready (almost 😂)
More tests have been added, such as JA4 tests from the JA4 official repo.
Fingerproxy has been deployed on some staging servers of subscan.io for a few weeks. We currently have 12 Fingerproxy instances, serving ~ 4 millions of requests per day.
In the next few releases, Fingerproxy may become production-ready state. Let's see! 😉
Full Changelog
9d53f51 (HEAD, tag: v0.4.0, origin/master) Fixup e2e test.
c5d72db Add tests for -preserve-host.
ed4366f Suspend some TLS handshake client error log, moved to verbose log.
c2473c8 Breaking change: remove GetReverseProxyHandler, DefaultTLSConfig(), StartPrometheusClient(), DefaultReverseProxyHTTPHandler(), DefaultProxyServer().
b6a145d Breaking change: remove HTTPHandler.ReverseProxy and SetReverseProxyRewriteFunc().
e283cb7 Improve README.
052d130 Implement official JA4 tests. (#6)
v0.3.1
v0.3.0
This release introduced a few new features, some bug fixes, and one breaking change. Performance, tests, and error handling has been signaficantly improved.
Notable changes listed below.
Breaking Change
In commit 78b2ab0, variable fingerproxy.DefaultReverseProxyHTTPHandler has been renamed to fingerproxy.GetReverseProxyHTTPHandler. Now fingerproxy.DefaultReverseProxyHTTPHandler is a function instead of a variable. The default value of GetReverseProxyHTTPHandler is DefaultReverseProxyHTTPHandler.
Kubernetes Liveness Probe
Commit 480707b introduced support of Kubernetes Readiness and Liveness probes. Example:
apiVersion: v1
kind: Pod
metadata:
name: fingerproxy
spec:
containers:
- name: fingerproxy
image: fingerproxy
livenessProbe:
httpGet:
path: /
port: 443
scheme: httpsIt is possible to use your prober as well. Check out HTTPHandler.IsProbeRequest.
Debug Server (pprof)
Build with tag debug to enable the debug server.
Handle Plain HTTP Requests
Fingerproxy now returns a HTTP 400 Bad Request if clients send plain HTTP requests before closing the connection.
TLS Handshake Timeout
TLS handshakes now has a default timeout of 10 seconds.
JA4 Fixes
89a31de Fixed non-ascii ALPN.
f095b8d Fixed "extension data should not be empty" while parsing pre_shard_key extension.
CI tests
Many tests have been added in CI to improve observability and prevent possible memory leak. Check out GitHub Actions.
Full Change Log
390ff19 (HEAD, tag: v0.3.0, origin/master) Add tests for reverseproxy.
480707b Add support for Kubernetes liveness/readiness probes.
4c307fa Improve e2e memtest.
a047cc8 Fix CI build-test-deps.
7a5a318 Add /fingerproxy to .gitignore.
1ff645f Add e2e memtest.
05f964d Improve CI tests.
25160f9 Add todo customize transport.
78b2ab0 Breaking change: rename DefaultReverseProxyHTTPHandler to GetReverseProxyHTTPHandler.
2809a6c Move verbose print client hello data to proxyserver.
a3cd1f2 Run e2e test with fingerproxy, not echo server.
3504cc0 Add JA4 benchmark.
2d8c796 Add CI load test. (#4)
7dca465 Fix echo server didn't return after error.
b243ddb Improve hijack error handling.
633573e Do not return incomplete ClientHello.
d061743 Refactor capture ClientHello to reduce memcopy and support TLSHandshakeTimeout.
3e45a0b Update vlogf.
2c33d47 Cleanup TODO.
38a0a0b Verbose log client hello in header injector.
bcf6363 Print raw ClientHello record in echo server.
5e21a19 Improve HTTP/1.1 server error handling.
df09abd Separate CI "release" workflow.
a1831f9 Add e2e test. (#3)
08b4ce3 Improve channel listener.
a1266ef Improve JA4 tests.
89a31de Fix non-ascii ALPN.
b05eecb Fix CI test. (#2)
a22cdda CI test.
f095b8d Fix "extension data should not be empty" while parsing pre_shard_key extension.