-
Notifications
You must be signed in to change notification settings - Fork 80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failure when retrieving reports from the hosted NSnitch deployment #15
Comments
Additionally, the randomizer endpoint isn't returning CORS headers, resulting in the response being blocked by the browser. |
Hi, @elyscape, thank you very much for pointing this out. We are in the process of eliminating that CORS issue. (there is an error-handling branch that does not provide any headers with it's response) Regarding the 404, can i ask you for some particularities? Lastly, is the 404ing on the report page, a spot-, or commonly occurring issue for you? Thank you! |
Trying it again now, I get Amsterdam as the node city.
At present, it happens for all report requests. |
Thank you, I'll investigate. In the meantime, before issuing the report requests, did you regenerate the URLs, or did you try with the old ones? (they become stale after 10 minutes, or so) |
I tried using the old URLs, I tried generating new URLs, and I tried manually inventing URLs of my own. None worked. If report URLs are intended to work only after having being returned by the randomizer, the problem is likely the fact that the hostnames being returned have capital letters; DNS canonically requires ASCII characters in labels to be downcased, and public recursive resolvers likely perform this transformation before recursing. |
Hi @elyscape , sorry for the radio silence. So i can see 2 issues here, merged into this scenario: Now, I can assure you, if the request reaches our servers, the DNS lookups have been executed successfully, and since you received the NSnitch API error message, it definitely reaches our servers. Can you try running this command? (it is essentially a composite one-liner of the two commands you already ran) For example, i get the following reply: As for the CORS error, I'm currently investigating the source of it. |
I still get this: $ curl -s `curl -s https://nstoro.com/api/v1/randomizer?api_response=true | jq '.["data"]["url"]' | awk -F\" '{ print $2 }'`
{"status":"ERROR","type":"TENTA_NSNITCH","data":null,"message":"Not Found","code":404} Throwing in a command to print the returned URL to stderr reveals that the hostname still has capital letters: $ curl -s `curl -s https://nstoro.com/api/v1/randomizer?api_response=true | jq '.["data"]["url"]' | awk -F\" '{ print $2 }' | tee /dev/stderr`
https://Discerning-Fan-SorestPhantasm.nstoro.com/api/v1/report
{"status":"ERROR","type":"TENTA_NSNITCH","data":null,"message":"Not Found","code":404} As an aside, instead of passing the output of $ curl -s https://nstoro.com/api/v1/randomizer?api_response=true | jq '.["data"]["url"]' -r
https://Lagniappes-Cycling4revet66Syncing.nstoro.com/api/v1/report |
Can you try supplying a custom dns option to both curl commands with |
I had to compile curl with support for the $ /usr/local/opt/curl/bin/curl -s --dns-servers 99.192.182.100 -s "$(/usr/local/opt/curl/bin/curl -s --dns-servers 99.192.182.100 https://nstoro.com/api/v1/randomizer?api_response=true | jq '.["data"]["url"]' -r)"
{"status":"ERROR","type":"TENTA_NSNITCH","data":null,"message":"Not Found","code":404} |
Well, that is something. Now we know that it's not about the capital words, because we don't modify the char case |
Indeed. That being said, other recursive DNS resolvers might, so it's probably a good idea to make the word list be entirely lower-case. |
You're probably right, there might be some that do, i haven't yet saw any, but yeah. |
$ curl -i -H 'Origin: https://tenta.com' https://nstoro.com/api/v1/randomizer?api_response=true
HTTP/2 200
access-control-allow-origin: https://tenta.com
cache-control: no-cache, no-store, must-revalidate
content-type: text/plain; charset=utf-8
content-length: 152
date: Fri, 02 Mar 2018 21:31:53 GMT
{"status":"OK","type":"TENTA_NSNITCH","data":{"url":"https://multiplied82RubiconnedBrighten4Smallage.nstoro.com/api/v1/report"},"message":"","code":200}
$ curl -i -H 'Origin: https://tenta.com' https://multiplied82RubiconnedBrighten4Smallage.nstoro.com/api/v1/report
HTTP/2 404
content-type: text/plain; charset=utf-8
content-length: 86
date: Fri, 02 Mar 2018 21:32:30 GMT
{"status":"ERROR","type":"TENTA_NSNITCH","data":null,"message":"Not Found","code":404} So yes on the randomizer but not on the subdomains |
$ curl -i -H 'Origin: https://tenta.com' https://nstoro.com/api/v1/invalid
HTTP/2 404
content-type: text/plain; charset=utf-8
content-length: 86
date: Fri, 02 Mar 2018 21:34:05 GMT
{"status":"ERROR","type":"TENTA_NSNITCH","data":null,"message":"Not Found","code":404} Also not on other error responses. |
Found the reason for that. From the documentation:
Unfortunately, in basically every place that error codes are used, w.WriteHeader(djo.Code) |
Good catch, @elyscape ! Thanks! |
No dice: $ curl -i --http1.1 -H 'Origin: https://tenta.com' https://nstoro.com/api/v1/randomizer?api_response=true ; echo
HTTP/1.1 200 OK
Access-Control-Allow-Origin: https://tenta.com
Cache-Control: no-cache, no-store, must-revalidate
Date: Fri, 02 Mar 2018 22:57:25 GMT
Content-Length: 148
Content-Type: text/plain; charset=utf-8
{"status":"OK","type":"TENTA_NSNITCH","data":{"url":"https://Escot91Planchets42loadstarInnervate.nstoro.com/api/v1/report"},"message":"","code":200}
$ curl -i --http1.1 -H 'Origin: https://tenta.com' https://Escot91Planchets42loadstarInnervate.nstoro.com/api/v1/report ; echo
HTTP/1.1 404 Not Found
Date: Fri, 02 Mar 2018 22:57:39 GMT
Content-Length: 86
Content-Type: text/plain; charset=utf-8
{"status":"ERROR","type":"TENTA_NSNITCH","data":null,"message":"Not Found","code":404} Same deal with $ /usr/local/opt/curl/bin/curl --dns-servers 99.192.182.100 -i --http1.1 -H 'Origin: https://tenta.com' https://nstoro.com/api/v1/randomizer?api_response=true ; echo
HTTP/1.1 200 OK
Access-Control-Allow-Origin: https://tenta.com
Cache-Control: no-cache, no-store, must-revalidate
Date: Fri, 02 Mar 2018 23:00:27 GMT
Content-Length: 150
Content-Type: text/plain; charset=utf-8
{"status":"OK","type":"TENTA_NSNITCH","data":{"url":"https://ranker4matterless79Diacodium-Jetliner.nstoro.com/api/v1/report"},"message":"","code":200}
$ /usr/local/opt/curl/bin/curl --dns-servers 99.192.182.100 -i --http1.1 -H 'Origin: https://tenta.com' https://ranker4matterless79Diacodium-Jetliner.nstoro.com/api/v1/report ; echo
HTTP/1.1 404 Not Found
Date: Fri, 02 Mar 2018 23:00:39 GMT
Content-Length: 86
Content-Type: text/plain; charset=utf-8
{"status":"ERROR","type":"TENTA_NSNITCH","data":null,"message":"Not Found","code":404} |
Hmm... This is surreal. |
Nope. This is consistent across multiple different machines on different networks with different stub resolvers. |
Hi @elyscape! Please let me know of the outcome, |
Using Tenta DNS as the resolver: ded7ed8aeb0a:/ root# cat /etc/resolv.conf
nameserver 99.192.182.100
ded7ed8aeb0a:/ root# curl -s 'https://nstoro.com/api/v1/randomizer?api_response=true' | python -m json.tool
{
"code": 200,
"data": {
"url": "https://minibus-staleness-lipectomy68gratinated.nstoro.com/api/v1/report"
},
"message": "",
"status": "OK",
"type": "TENTA_NSNITCH"
}
ded7ed8aeb0a:/ root# curl -s 'https://minibus-staleness-lipectomy68gratinated.nstoro.com/api/v1/report' | python -m json.tool
{
"code": 200,
"data": {
"additional_edns0_records": 0,
"checking_disabled": false,
"client_subnet": null,
"dns_cookie": false,
"dnssec_enabled": true,
"ip": "147.75.100.185",
"ip_family": "v4",
"net_type": "tcp",
"node_location": {
"city": "Amsterdam",
"country": "Netherlands",
"iso_country": "NL",
"localized_location": {
"en": "Amsterdam, Netherlands"
},
"location": "Amsterdam, Netherlands",
"network": {
"as_number": 396093,
"as_owner": "Tenta",
"isp": "Tenta",
"organization": "PacketNet BV"
},
"position": {
"latitude": 52.370216,
"longitude": 4.895168,
"time_zone": "Europe/Amsterdam",
"uncertainty_km": 10
},
"tor_node": null
},
"nsid_requested": false,
"recursion_desired": false,
"request_location": {
"city": "New York",
"country": "United States",
"iso_country": "US",
"localized_location": {
"de": "New York City, New York, USA",
"en": "New York, New York, United States",
"es": "Nueva York, Nueva York, Estados Unidos",
"fr": "New York, New York, \u00c9tats-Unis",
"ja": "\u30cb\u30e5\u30fc\u30e8\u30fc\u30af, \u30cb\u30e5\u30fc\u30e8\u30fc\u30af\u5dde, \u30a2\u30e1\u30ea\u30ab\u5408\u8846\u56fd",
"pt-BR": "Nova Iorque, Nova Iorque, Estados Unidos",
"ru": "\u041d\u044c\u044e-\u0419\u043e\u0440\u043a, \u041d\u044c\u044e-\u0419\u043e\u0440\u043a, \u0421\u0428\u0410",
"zh-CN": "\u7ebd\u7ea6, \u7ebd\u7ea6\u5dde, \u7f8e\u56fd"
},
"location": "New York, New York, United States",
"network": {
"as_number": 54825,
"as_owner": "Packet Host, Inc.",
"isp": "Costra S.A.",
"organization": "Packet Host"
},
"position": {
"latitude": 40.7139,
"longitude": -74.0079,
"time_zone": "America/New_York",
"uncertainty_km": 1
},
"tor_node": null
},
"request_time": "1520966780",
"tcp_keep_alive_enabled": false,
"tcp_keep_alive_ms": 0,
"tls_enabled": true
},
"message": "",
"status": "OK",
"type": "TENTA_NSNITCH"
} Using Google DNS as the resolver: 83aeb9d126db:/ root# cat /etc/resolv.conf
nameserver 8.8.8.8
83aeb9d126db:/ root# curl -s 'https://nstoro.com/api/v1/randomizer?api_response=true' | python -m json.tool
{
"code": 200,
"data": {
"url": "https://base-lodgepoles4iniquitous17polonism.nstoro.com/api/v1/report"
},
"message": "",
"status": "OK",
"type": "TENTA_NSNITCH"
}
83aeb9d126db:/ root# curl -s 'https://base-lodgepoles4iniquitous17polonism.nstoro.com/api/v1/report' | python -m json.tool
{
"code": 404,
"data": null,
"message": "Not Found",
"status": "ERROR",
"type": "TENTA_NSNITCH"
} |
Using Quad9 as the resolver also fails: 0af051fdf399:/ root# cat /etc/resolv.conf
nameserver 9.9.9.9
0af051fdf399:/ root# curl -s 'https://nstoro.com/api/v1/randomizer?api_response=true' | python -m json.tool
{
"code": 200,
"data": {
"url": "https://peewees-clasp-trochleas49troika.nstoro.com/api/v1/report"
},
"message": "",
"status": "OK",
"type": "TENTA_NSNITCH"
}
0af051fdf399:/ root# curl -s 'https://peewees-clasp-trochleas49troika.nstoro.com/api/v1/report' | python -m json.tool
{
"code": 404,
"data": null,
"message": "Not Found",
"status": "ERROR",
"type": "TENTA_NSNITCH"
} |
Interestingly, sometimes requesting the report multiple times succeeds: 34700a4469e4:/ root# cat /etc/resolv.conf
nameserver 8.8.8.8
34700a4469e4:/ root# curl -s 'https://nstoro.com/api/v1/randomizer?api_response=true' | python -m json.tool
{
"code": 200,
"data": {
"url": "https://sizarshipsdialogists-typo92ouakari.nstoro.com/api/v1/report"
},
"message": "",
"status": "OK",
"type": "TENTA_NSNITCH"
}
34700a4469e4:/ root# curl -s 'https://sizarshipsdialogists-typo92ouakari.nstoro.com/api/v1/report' | python -m json.tool
{
"code": 404,
"data": null,
"message": "Not Found",
"status": "ERROR",
"type": "TENTA_NSNITCH"
}
34700a4469e4:/ root# curl -s 'https://sizarshipsdialogists-typo92ouakari.nstoro.com/api/v1/report' | python -m json.tool
{
"code": 404,
"data": null,
"message": "Not Found",
"status": "ERROR",
"type": "TENTA_NSNITCH"
}
34700a4469e4:/ root# curl -s 'https://sizarshipsdialogists-typo92ouakari.nstoro.com/api/v1/report' | python -m json.tool
{
"code": 404,
"data": null,
"message": "Not Found",
"status": "ERROR",
"type": "TENTA_NSNITCH"
}
34700a4469e4:/ root# curl -s 'https://sizarshipsdialogists-typo92ouakari.nstoro.com/api/v1/report' | python -m json.tool
{
"code": 200,
"data": {
"additional_edns0_records": 0,
"checking_disabled": true,
"client_subnet": {
"draft_mode": false,
"ip": "70.42.243.0",
"ip_family": "v4",
"netmask": 24,
"subnet_location": {
"city": "Redwood City",
"country": "United States",
"iso_country": "US",
"localized_location": {
"de": "Redwood City, Kalifornien, USA",
"en": "Redwood City, California, United States",
"es": "Redwood City, California, Estados Unidos",
"fr": "Redwood City, Californie, \u00c9tats-Unis",
"ja": "\u30ec\u30c3\u30c9\u30a6\u30c3\u30c9\u30b7\u30c6\u30a3, \u30ab\u30ea\u30d5\u30a9\u30eb\u30cb\u30a2\u5dde, \u30a2\u30e1\u30ea\u30ab\u5408\u8846\u56fd",
"pt-BR": "Redwood City, Calif\u00f3rnia, Estados Unidos",
"ru": "\u0420\u0435\u0434\u0432\u0443\u0434 \u0421\u0438\u0442\u0438, \u041a\u0430\u043b\u0438\u0444\u043e\u0440\u043d\u0438\u044f, \u0421\u0428\u0410",
"zh-CN": "Redwood City, \u52a0\u5229\u798f\u5c3c\u4e9a\u5dde, \u7f8e\u56fd"
},
"location": "Redwood City, California, United States",
"network": {
"as_number": 12182,
"as_owner": "Internap Network Services Corporation",
"isp": "Internap Network Services Corporation",
"organization": "Internap Network Services Corporation"
},
"position": {
"latitude": 37.4096,
"longitude": -122.2991,
"time_zone": "America/Los_Angeles",
"uncertainty_km": 50
},
"tor_node": null
}
},
"dns_cookie": false,
"dnssec_enabled": true,
"ip": "74.125.186.2",
"ip_family": "v4",
"net_type": "udp",
"node_location": {
"city": "Seattle",
"country": "United States",
"iso_country": "US",
"localized_location": {
"en": "Seattle, Washington, United States"
},
"location": "Seattle, Washington, United States",
"network": {
"as_number": 396093,
"as_owner": "Tenta",
"isp": "Tenta",
"organization": "WolfeNet"
},
"position": {
"latitude": 47.608013,
"longitude": -122.33517,
"time_zone": "America/Log_Angeles",
"uncertainty_km": 10
},
"tor_node": null
},
"nsid_requested": false,
"recursion_desired": false,
"request_location": {
"city": "The Dalles",
"country": "United States",
"iso_country": "US",
"localized_location": {
"de": "The Dalles, Oregon, USA",
"en": "The Dalles, Oregon, United States",
"es": "The Dalles, Oreg\u00f3n, Estados Unidos",
"fr": "The Dalles, Oregon, \u00c9tats-Unis",
"ja": "The Dalles, \u30aa\u30ec\u30b4\u30f3\u5dde, \u30a2\u30e1\u30ea\u30ab\u5408\u8846\u56fd",
"pt-BR": "City of The Dalles, Oreg\u00e3o, Estados Unidos",
"ru": "The Dalles, \u041e\u0440\u0435\u0433\u043e\u043d, \u0421\u0428\u0410",
"zh-CN": "The Dalles, \u4fc4\u52d2\u5188\u5dde, \u7f8e\u56fd"
},
"location": "The Dalles, Oregon, United States",
"network": {
"as_number": 15169,
"as_owner": "Google LLC",
"isp": "Google",
"organization": "Google Translate"
},
"position": {
"latitude": 45.5447,
"longitude": -121.1543,
"time_zone": "America/Los_Angeles",
"uncertainty_km": 1000
},
"tor_node": null
},
"request_time": "1520967545",
"tcp_keep_alive_enabled": false,
"tcp_keep_alive_ms": 0,
"tls_enabled": false
},
"message": "",
"status": "OK",
"type": "TENTA_NSNITCH"
} |
Great! We managed to reproduce the scenario, after some trial and error. The other issues, CORS and domain-name capitalization are taken care of. We will merge them in shortly. |
Weird. I wonder if this might be a result of the NS record having a TTL of 0. My logic is that some recursors might query for the NS record, get a TTL of 0 in the response, immediately expire the record, and then decide “well now I’ve got nothing” and return NXDOMAIN. |
We found the issue. Thanks |
Everything seems to work over here! That being said, the Browser Privacy Test page is now having issues due to the fact that it sends 4 requests to nstoro.com and 4 requests to nameinspector.org, as nameinspector.org:
|
Those issues have now been resolved, but I've noticed that I'm seeing the original issue 100% of the time when trying to use Cloudflare DNS. |
Are there still issues relating to this outstanding? It seems to work for me today with 1.1.1.1 as the DNS server. |
I'm still seeing it with Cloudflare DNS. |
Attempting to retrieve a report from the public NSwitch endpoints results in a 404 failure, breaking the Browser Privacy Test page. Consider:
Noticing that the returned hostname had capital letters, which are normalized away by DNS, I tried manually specifying it to
dig
and as aHost
header withcurl
. No dice. I also tried doing it in lowercase and with a lowercase UUID that I had generated. They all returned 404s.The text was updated successfully, but these errors were encountered: