checkpoint

censoredplanet · Sep 22, 2023 · 3ff545d · 3ff545d
1 parent 2ecab41
commit 3ff545d
Showing 1 changed file with 47 additions and 39 deletions.
diff --git a/docs/base_tables.md b/docs/base_tables.md
@@ -208,42 +208,50 @@ The DNS (Satellite) data included the following alternative set of columns. (Man
 |                           |
 | measurement_id            | STRING       | A uuid which is the same for observations which are part of the same measurement. </br> If there are 5 retries of a scan they will all have the same id. </br> eg. `a08df2fe70d54092916b8df87e330f47` |
 | source                    | STRING       | The name of the .tar.gz scan file this row came from. </br> eg. `CP_Satellite-2020-08-20-05-58-35` </br> Used internally and for debugging |
-|                           |
-| **Answers**               | REPEATED STRUCT | Contains the following fields. Each represents an IP address answer received from the resolver, and subsequent metadata for that IP. |
-|                           |
-
-|  |  |  |
-
-answers	RECORD	REPEATED		
-
-	RECORD	REPEATED				
-ip	STRING	NULLABLE				
-asn	INTEGER	NULLABLE				
-as_name	STRING	NULLABLE				
-ip_organization	STRING	NULLABLE				
-censys_http_body_hash	STRING	NULLABLE				
-censys_ip_cert	STRING	NULLABLE				
-matches_control	RECORD	NULLABLE				
-match_confidence	FLOAT	NULLABLE				
-http_error	STRING	NULLABLE				
-http_response_status	STRING	NULLABLE				
-http_response_headers	STRING	REPEATED				
-http_response_body	STRING	NULLABLE				
-http_analysis_is_known_blockpage	BOOLEAN	NULLABLE				
-http_analysis_page_signature	STRING	NULLABLE				
-https_error	STRING	NULLABLE				
-https_tls_version	INTEGER	NULLABLE				
-https_tls_cipher_suite	INTEGER	NULLABLE				
-https_tls_cert	BYTES	NULLABLE				
-https_tls_cert_common_name	STRING	NULLABLE				
-https_tls_cert_issuer	STRING	NULLABLE				
-https_tls_cert_start_date	TIMESTAMP	NULLABLE				
-https_tls_cert_end_date	TIMESTAMP	NULLABLE				
-https_tls_cert_alternative_names	STRING	REPEATED				
-https_tls_cert_has_trusted_ca	BOOLEAN	NULLABLE				
-https_tls_cert_matches_domain	BOOLEAN	NULLABLE				
-https_response_status	STRING	NULLABLE				
-https_response_headers	STRING	REPEATED				
-https_response_body	STRING	NULLABLE				
-https_analysis_is_known_blockpage	BOOLEAN	NULLABLE				
-https_analysis_page_signature	STRING	NULLABLE	
+|                               |
+| **Answers**                   | REPEATED STRUCT | Contains the following fields. Each represents an IP address answer received from the resolver, and subsequent metadata for that IP. |
+|                               |
+| answers.ip                    | STRING  |  |
+| answers.asn                   | INTEGER |  |
+| answers.as_name               | STRING  |  |
+| answers.ip_organization       | STRING  |  |
+| answers.censys_http_body_hash | STRING  |  |
+| answers.censys_ip_cert        | STRING  |  |
+|                                               |
+| **Matches Control**                           | Whether the metadata of the returned IP matches the expected metadata of a control measurement | 
+|                                               |
+| answers.matches_control                       | REPEATED RECORD |  |
+| answers.matches_control.ip                    | BOOLEAN         |  |
+| answers.matches_control.censys_http_body_hash | BOOLEAN         |  |
+| answers.matches_control.censys_ip_cert        | BOOLEAN         |  |
+| answers.matches_control.asn                   | BOOLEAN         |  |
+| answers.matches_control.as_name               | BOOLEAN         |  |
+| answers.match_confidence                      | FLOAT           | Value from 0-1. Confidince that this IP response matches a control measurement |
+|                                           |
+| **HTTP Request**                          | Metadata from the HTTP request made to the returned IP | 
+|                                           |
+| answers.http_error                        | STRING          |  |
+| answers.http_response_status              | STRING          |  |
+| answers.http_response_headers             | REPEATED STRING |  |
+| answers.http_response_body                | STRING          |  |
+| answers.http_analysis_is_known_blockpage  | BOOLEAN         |  |
+| answers.http_analysis_page_signature      | STRING          |  |
+|                                           |
+| **HTTPS Request**                         | Metadata from the HTTPS request made to the returned IP | 
+|                                           |
+| answers.https_error                       | STRING          |  |
+| answers.https_tls_version                 | INTEGER         |  |
+| answers.https_tls_cipher_suite            | STRING          |  |
+| answers.https_tls_cert                    | BYTES           |  |
+| answers.https_tls_cert_common_name        | STRING          |  |
+| answers.https_tls_cert_issuer             | STRING          |  |
+| answers.https_tls_cert_start_date         | TIMESTAMP       |  |
+| answers.https_tls_cert_end_date           | TIMESTAMP       |  |
+| answers.https_tls_cert_alternative_names  | REPEATED STRING |  |
+| answers.https_tls_cert_has_trusted_ca     | BOOLEAN         |  |
+| answers.https_tls_cert_matches_domain     | BOOLEAN         |  |
+| answers.https_response_status             | STRING          |  |
+| answers.https_response_headers            | REPEATED STRING |  |
+| answers.https_response_body               | STRING          |  |
+| answers.https_analysis_is_known_blockpage | BOOLEAN         |  |
+| answers.https_analysis_page_signature     | STRING          |  |