Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datadome 2.23.0 (#6) #715

Merged
merged 5 commits into from
Nov 20, 2024
Merged

Datadome 2.23.0 (#6) #715

merged 5 commits into from
Nov 20, 2024

Conversation

MickaelDatadome
Copy link
Contributor

@MickaelDatadome MickaelDatadome commented Oct 24, 2024

  • updating to DataDome Fastly Module 2.23.0

MickaelDatadome and others added 5 commits October 24, 2024 15:19
* Update datacenters.json

* Added basename sanitize to original snippet name, preventing path traversal

* Bump to 1.2.208

* Update datacenters.json

* wp

* Update datacenters.json

* Bump to 1.2.209

* netacea integration v5.7.0

* Update datacenters.json

* Bump to 1.2.210

* Release notes update for 1.2.210

* Update datacenters.json

* php 8.3

* added opensearch

* fix for fastly#682

* fix for fastly#682

* Bump to 1.2.211

* fix fastly#682

* updating to DataDome Fastly Module 2.19.4

Signed-off-by: Mickaël Guichard <[email protected]>

* Bump to 1.2.212

* ci: Use GITHUB_OUTPUT envvar instead of set-output command

`save-state` and `set-output` commands used in GitHub Actions are deprecated and [GitHub recommends using environment files](https://github.blog/changelog/2023-07-24-github-actions-update-on-save-state-and-set-output-commands/).

This PR updates the usage of `::set-output` to `"$GITHUB_OUTPUT"`

Instructions for envvar usage from GitHub docs:

https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-output-parameter

* Module and documentation updates

* Documenation wording change

* Spelling fix

* Add support for Brotli static compression

* Add testing for 2.4.6 and PHP 8.3

* Bump to 1.2.213

* Rate limiting improvement

* fix code quality

* Bump to 1.2.214

* Netacea Magento module updated with additional logging

* Fix for checking if current IP is in maintenance IP list

* Bump to 1.2.215

* fix GEOIP redirection causes 404 fastly#693

* Bump to 1.2.216

* Bump to 1.2.217

* Netacea Magento module updated with PURGE requests handling

* Fixing deprecated usage - adding cast to string and tags size check

* Bump to 1.2.218

* Removing trailing comma in WAF constructor to be compatible with PHP 7.2

* Bump to 1.2.219

* Update datacenters.json

* Don't display API keys and tokens on endpoint update - they shouldn't be visible after configuration

* In 2.4 path is /pub/errors

* Bump to 1.2.220

* updating to DataDome Fastly Module 2.22.0

---------

Signed-off-by: Mickaël Guichard <[email protected]>
Co-authored-by: github-actions <vvuksan [email protected]>
Co-authored-by: ivanviduka <[email protected]>
Co-authored-by: Vladimir Vuksan <[email protected]>
Co-authored-by: Vladimir Vuksan <[email protected]>
Co-authored-by: Domagoj Potkoc <[email protected]>
Co-authored-by: Richard Walkden <[email protected]>
Co-authored-by: Arun Sathiya <[email protected]>
Co-authored-by: mizdebski-netacea <[email protected]>
Co-authored-by: Vladimir Vuksan <[email protected]>
Co-authored-by: Marek Izdebski <[email protected]>
@MickaelDatadome MickaelDatadome changed the title Datadome 2.22.0 (#6) Datadome 2.23.0 (#6) Nov 8, 2024
@harmony7
Copy link
Member

This PR introduces optional features to restore referrer headers and enhance GraphQL request detection, updates VCL configurations for improved DataDome integration, and supports stricter TLS settings and additional headers.

This PR adds the following properties:

restore_referrer (boolean) Enable referrer restoration (optional)
default: false
True to restore original referrer when a challenge is passed.
graphql_support (boolean) Enable GraphQL support for POST requests (optional)
default: false
True to retrieve GraphQL information and improve detection.

This PR makes changes to the VCL that enable the above new properties.

Click for details on VCL changes

In init, it makes the following changes to VCL:

 sub set_origin_header {
   if (req.backend.is_origin) {
     if (req.backend == datadome) {
         # Remove all unexpected headers
         header.filter_except(bereq, "x-datadome-params", "accept-charset", "accept-language", "x-requested-with", "x-fl-productid", "x-flapi-session-id", "fastly-orig-accept-encoding", "cache-control", "client-id", "connection", "pragma", "accept", "headers-list", "host", "origin", "server-hostname", "server-name", "x-forwarded-for", "user-agent", "referer", "request", "content-type", "from", "true-client-ip", "via", "x-real-ip", "sec-ch-device-memory", "sec-ch-ua", "sec-ch-ua-arch", "sec-ch-ua-full-version-list", "sec-ch-ua-mobile", "sec-ch-ua-model", "sec-ch-ua-platform", "sec-fetch-dest", "sec-fetch-mode", "sec-fetch-site", "sec-fetch-user");
         set bereq.http.x-datadome-params:key = "{{datadome_api_key}}";
         set bereq.http.x-datadome-params:requestmodulename = "FastlyMagento";
-        set bereq.http.x-datadome-params:moduleversion = "2.19.4";
+        set bereq.http.x-datadome-params:moduleversion = "2.23.0";
         set bereq.http.x-datadome-params:timerequest = time.start.usec;
         set bereq.http.x-datadome-params:servername = server.identity;
         set bereq.http.x-datadome-params:serverregion = server.region;
         set bereq.http.x-datadome-params:ip = urlencode(client.ip);
         set bereq.http.x-forwarded-proto = urlencode(req.protocol);
         set bereq.http.x-datadome-params:authorizationlen = std.strlen(req.http.authorization);
         # Truncating Headers - Start
         set bereq.http.accept-charset = substr(req.http.accept-charset, 0, 128);
         set bereq.http.accept-language = substr(req.http.accept-language, 0, 256);
         set bereq.http.x-requested-with = substr(req.http.x-requested-with, 0, 128);
         set bereq.http.x-fl-productid = substr(req.http.x-fl-productid, 0, 64);
         set bereq.http.x-flapi-session-id = substr(req.http.x-flapi-session-id, 0, 64);
         set bereq.http.fastly-orig-accept-encoding = substr(req.http.fastly-orig-accept-encoding, 0, 128);
         set bereq.http.cache-control = substr(req.http.cache-control, 0, 128);
         set bereq.http.client-id = substr(req.http.client-id, 0, 128);
         set bereq.http.connection = substr(req.http.connection, 0, 128);
         set bereq.http.pragma = substr(req.http.pragma, 0, 128);
         set bereq.http.accept = substr(req.http.accept, 0, 512);
         set bereq.http.headers-list = substr(req.http.headers-list, 0, 512);
         set bereq.http.host = substr(req.http.host, 0, 512);
         set bereq.http.origin = substr(req.http.origin, 0, 512);
         set bereq.http.server-hostname = substr(req.http.server-hostname, 0, 512);
         set bereq.http.server-name = substr(req.http.server-name, 0, 512);
         if( std.strlen(req.http.x-forwarded-for) \u003e 512 ) {
             # Truncate from the end
             set bereq.http.x-forwarded-for = substr(req.http.x-forwarded-for, -512);
         } else {
             set bereq.http.x-forwarded-for = req.http.x-forwarded-for;
         }
         set bereq.http.user-agent = substr(req.http.user-agent, 0, 768);
         set bereq.http.referer = substr(req.http.referer, 0, 1024);
         set bereq.http.request = substr(req.http.request, 0, 2048);
         set bereq.http.content-type = substr(req.http.content-type, 0, 64);
         set bereq.http.from = substr(req.http.from, 0, 128);
         set bereq.http.true-client-ip = substr(req.http.true-client-ip, 0, 128);
         set bereq.http.via = substr(req.http.via, 0, 256);
         set bereq.http.x-real-ip = substr(req.http.x-real-ip, 0, 128);
         set bereq.http.sec-ch-device-memory = substr(req.http.sec-ch-device-memory, 0, 8);
         set bereq.http.sec-ch-ua = substr(req.http.sec-ch-ua, 0, 128);
         set bereq.http.sec-ch-ua-arch = substr(req.http.sec-ch-ua-arch, 0, 16);
         set bereq.http.sec-ch-ua-full-version-list = substr(req.http.sec-ch-ua-full-version-list, 0, 256);
         set bereq.http.sec-ch-ua-mobile = substr(req.http.sec-ch-ua-mobile, 0, 8);
         set bereq.http.sec-ch-ua-model = substr(req.http.sec-ch-ua-model, 0, 128);
         set bereq.http.sec-ch-ua-platform = substr(req.http.sec-ch-ua-platform, 0, 32);
         set bereq.http.sec-fetch-dest = substr(req.http.sec-fetch-dest, 0, 32);
         set bereq.http.sec-fetch-mode = substr(req.http.sec-fetch-mode, 0, 32);
         set bereq.http.sec-fetch-site = substr(req.http.sec-fetch-site, 0, 64);
         set bereq.http.sec-fetch-user = substr(req.http.sec-fetch-user, 0, 8);
         # Truncating Headers - End
         if (req.http.x-datadome-clientid) {
             set bereq.http.x-datadome-params:clientid = urlencode(substr(req.http.x-datadome-clientid, 0, 128));
             set bereq.http.x-datadome-x-set-cookie = "true";
         } else {
             set bereq.http.x-datadome-params:clientid = urlencode(substr(req.http.cookie:datadome, 0, 128));
         }
         set bereq.http.x-datadome-params:cookieslen = std.strlen(req.http.cookie);
         # enforce gzip encoding between Fastly and DataDome
         set bereq.http.accept-encoding = "gzip";
+        # disable ng-waf inspection on DataDome requests
+        set bereq.http.x-sigsci-no-inspection = "true";
     } else {
         # prevent leak of the key
         unset bereq.http.x-datadome-params;
     }
   }
 }
 
 backend datadome {
     .host = "api-fastly.datadome.co";
     .port = "8443";
+    .max_tls_version = "1.3";
+    .min_tls_version = "1.2";
     .connect_timeout = {{datadome_connect_timeout}}ms;
     .first_byte_timeout = {{datadome_between_bytes_timeout}}ms;
     .between_bytes_timeout = {{datadome_between_bytes_timeout}}ms;
     .max_connections = 200;
     .ssl = true;
     .dynamic = true;
     .probe = {
         .request = "HEAD /.well-known/healthcheck-datadome HTTP/1.1" "Host: api-fastly.datadome.co" "Connection: close" "User-Agent: Varnish/fastly (healthcheck)";
         .expected_response = 200;
         .initial = 5;
         .interval = 2s;
         .threshold = 1;
         .timeout = 2s;
         .window = 5;
       }
 }

In fetch, it makes the following changes:

 if (req.backend == datadome) {
   declare local var.status STRING;
   set var.status = beresp.status;
   # check that it is real ApiServer response
   if (var.status != beresp.http.x-datadomeresponse) {
     restart;
   }
   unset beresp.http.x-datadomeresponse;
   # copy datadome headers
   set req.http.x-datadome-headers-pairs:x-datadome-headers = urlencode(beresp.http.x-datadome-headers);
 
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+x-set-cookie( |$)+") {
     set req.http.x-datadome-headers-pairs:x-set-cookie = urlencode(beresp.http.x-set-cookie);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+x-datadome-server( |$)+") {
     set req.http.x-datadome-headers-pairs:x-datadome-server = urlencode(beresp.http.x-datadome-server);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+x-datadome( |$)+") {
     set req.http.x-datadome-headers-pairs:x-datadome = urlencode(beresp.http.x-datadome);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+content-type( |$)+") {
     set req.http.x-datadome-headers-pairs:content-type = urlencode(beresp.http.content-type);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+charset( |$)+") {
     set req.http.x-datadome-headers-pairs:charset = urlencode(beresp.http.charset);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+cache-control( |$)+") {
     set req.http.x-datadome-headers-pairs:cache-control = urlencode(beresp.http.cache-control);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+pragma( |$)+") {
     set req.http.x-datadome-headers-pairs:pragma = urlencode(beresp.http.pragma);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+access-control-allow-credentials( |$)+") {
     set req.http.x-datadome-headers-pairs:access-control-allow-credentials = urlencode(beresp.http.access-control-allow-credentials);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+access-control-expose-headers( |$)+") {
     set req.http.x-datadome-headers-pairs:access-control-expose-headers = urlencode(beresp.http.access-control-expose-headers);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+access-control-allow-origin( |$)+") {
     set req.http.x-datadome-headers-pairs:access-control-allow-origin = urlencode(beresp.http.access-control-allow-origin);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+x-datadome-cid( |$)+") {
     set req.http.x-datadome-headers-pairs:x-datadome-cid = urlencode(beresp.http.x-datadome-cid);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+x-dd-b( |$)+") {
     set req.http.x-datadome-headers-pairs:x-dd-b = urlencode(beresp.http.x-dd-b);
   }
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+x-dd-type( |$)+") {
     set req.http.x-datadome-headers-pairs:x-dd-type = urlencode(beresp.http.x-dd-type);
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-dd-type( |$)+") {
     set req.http.x-dd-type = beresp.http.x-dd-type;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-botname( |$)+") {
     set req.http.x-datadome-botname = beresp.http.x-datadome-botname;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-botfamily( |$)+") {
     set req.http.x-datadome-botfamily = beresp.http.x-datadome-botfamily;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-isbot( |$)+") {
     set req.http.x-datadome-isbot = beresp.http.x-datadome-isbot;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-captchapassed( |$)+") {
     set req.http.x-datadome-captchapassed = beresp.http.x-datadome-captchapassed;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-devicecheckpassed( |$)+") {
     set req.http.x-datadome-devicecheckpassed = beresp.http.x-datadome-devicecheckpassed;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-traffic-rule-response( |$)+") {
     set req.http.x-datadome-traffic-rule-response = beresp.http.x-datadome-traffic-rule-response;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-captchaurl( |$)+") {
     set req.http.x-datadome-captchaurl = beresp.http.x-datadome-captchaurl;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-requestid( |$)+") {
     set req.http.x-datadome-requestid = beresp.http.x-datadome-requestid;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-score( |$)+") {
     set req.http.x-datadome-score = beresp.http.x-datadome-score;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-ruletype( |$)+") {
     set req.http.x-datadome-ruletype = beresp.http.x-datadome-ruletype;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-matchedmodels( |$)+") {
     set req.http.x-datadome-matchedmodels = beresp.http.x-datadome-matchedmodels;
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-sessionid( |$)+") {
     set req.http.x-datadome-sessionid = beresp.http.x-datadome-sessionid;
   }
   # don\u0027t forget about ApiServer\u0027s cookies
   if (beresp.http.x-datadome-headers ~ "(?i)(^| )+set-cookie( |$)+") {
     set req.http.x-datadome-headers-pairs:set-cookie = urlencode(beresp.http.set-cookie);
   }
 
   # Continue only if ApiServer returns expected blocked status
   if (beresp.status != 403 \u0026\u0026 beresp.status != 401 \u0026\u0026 beresp.status != 301 \u0026\u0026 beresp.status != 302) {
     unset beresp.http.x-datadome-headers;
     unset beresp.http.x-datadome-request-headers;
     set req.http.x-datadome-cookie = beresp.http.x-datadome-cookie; # Allow Session Feature
     restart;
   }
 
   # ok, it is banned request, cleanup it a bit
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-dd-type( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-dd-type( |$)+") {
       unset beresp.http.x-dd-type;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-botname( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-botname( |$)+") {
       unset beresp.http.x-datadome-botname;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-botfamily( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-botfamily( |$)+") {
       unset beresp.http.x-datadome-botfamily;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-isbot( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-isbot( |$)+") {
       unset beresp.http.x-datadome-isbot;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-captchapassed( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-captchapassed( |$)+") {
       unset beresp.http.x-datadome-captchapassed;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-devicecheckpassed( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-devicecheckpassed( |$)+") {
       unset beresp.http.x-datadome-devicecheckpassed;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-traffic-rule-response( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-traffic-rule-response( |$)+") {
       unset beresp.http.x-datadome-traffic-rule-response;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-captchaurl( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-captchaurl( |$)+") {
       unset beresp.http.x-datadome-captchaurl;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-requestid( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-requestid( |$)+") {
       unset beresp.http.x-datadome-requestid;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-score( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-score( |$)+") {
       unset beresp.http.x-datadome-score;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-ruletype( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-ruletype( |$)+") {
       unset beresp.http.x-datadome-ruletype;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-matchedmodels( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-matchedmodels( |$)+") {
       unset beresp.http.x-datadome-matchedmodels;
     }
   }
   if (beresp.http.x-datadome-request-headers ~ "(?i)(^| )+x-datadome-sessionid( |$)+") {
     if (beresp.http.x-datadome-headers !~ "(?i)(^| )+x-datadome-sessionid( |$)+") {
       unset beresp.http.x-datadome-sessionid;
     }
   }
   unset beresp.http.x-datadome-headers;
   unset beresp.http.x-datadome-request-headers;
+  # restore method
+  if (req.http.x-datadome-params:method) {
+    set req.method = urldecode(req.http.x-datadome-params:method);
+  }
 }

In error it makes the following changes:

-if (req.backend == datadome) { 
+if (req.backend == datadome \u0026\u0026 req.http.x-datadome-params) { 
+  # the backend is set to \u0027DataDome\u0027 and 
+  # the presence of a header confirms it\u0027s an error during DataDome process 
   restart; 
 }

It makes no changes to deliver.

In recv it makes the following changes:

   {{#if logging_endpoint}}
   ## Debug DataDome
   log {"syslog "} req.service_id {" {{logging_endpoint}} :: "}
       " timestamp=%22" now
       "%22 client_ip=" req.http.Fastly-Client-IP
       " request=" req.method
       " url=%22" cstr_escape(req.url.path)
       "%22 restarts=" req.restarts
       " DataDomeDebug=" "Before_DataDome"
       " fastlyFF=" fastly.ff.visits_this_service;
   ##
   {{/if}}# Configure the regular expression below to match URLs that
 # should be checked by DataDome
 if (fastly.ff.visits_this_service == 0 \u0026\u0026 req.restarts == 0 \u0026\u0026 req.method != "FASTLYPURGE" \u0026\u0026 !(req.url.path ~ "{{datadome_exclusion_ext}}" \u0026\u0026 (req.method == "GET" || req.method == "HEAD"))) {
 
   set req.backend = datadome;
   unset req.http.x-datadome-params;
   set req.http.x-datadome-params:method = urlencode(req.method);
   set req.http.x-datadome-params:postparamlen = urlencode(req.http.content-length);
   set req.method = "GET";
   set req.http.x-datadome-params:tlsprotocol = urlencode(tls.client.protocol);
   set req.http.x-datadome-params:tlscipherslist = urlencode(tls.client.ciphers_list);
   set req.http.x-datadome-params:tlsextensionslist = urlencode(tls.client.tlsexts_list);
   set req.http.x-datadome-params:ja3 = urlencode(tls.client.ja3_md5);
+  set req.http.x-datadome-params:ja4 = urlencode(tls.client.ja4);
+
+  if (var.datadome_enable_graphql_support == true \u0026\u0026 req.http.x-datadome-params:method == "POST" \u0026\u0026 req.http.content-type ~ "(?i)application\\/json" \u0026\u0026 req.url.path ~ "(?i)graphql") {
+    declare local var.graphQLQuery STRING;
+    set var.graphQLQuery = std.strstr(req.body, "%22query%22");
+    if (var.graphQLQuery ~ "(?m)(query|mutation|subscription)?\\s*([A-Za-z_][A-Za-z0-9_]*)?\\s*[\\({@]") {
+      set req.http.x-datadome-params:graphqlOperationType = if(re.group.1, re.group.1, "query");
+      if (re.group.2) {
+        set req.http.x-datadome-params:graphqlOperationName = substr(re.group.2, 0, 128);
+      }
+    }
+  }
+
+  if (var.datadome_restore_referrer == true \u0026\u0026 req.url ~ "dd_referrer=") {
+    
+    declare local var.decoded_url STRING;
+    declare local var.complete_request_url STRING;
+    set var.decoded_url = urldecode(req.url);
+    set var.complete_request_url = req.protocol "://" req.http.host querystring.filter(var.decoded_url, "dd_referrer");
+  
+    if (urldecode(querystring.clean(req.http.referer)) == querystring.clean(var.complete_request_url)) {
+        # Referer header is the same as the current URL
+
+        if (std.strlen(querystring.get(req.url, "dd_referrer")) \u003e 0) {
+            # Not empty `dd_referrer`, restore header `referer` with param value
+            set req.http.referer = urldecode(querystring.get(req.url, "dd_referrer"));
+        } else if (querystring.get(req.url, "dd_referrer") == "") {
+            # Empty `dd_referrer` param, remove header `referer`
+            unset req.http.referer;
+        } # Not set `dd_referrer`, do nothing
+        
+        # Remove `dd_referrer` query param from URL
+        set req.url = querystring.filter(req.url, "dd_referrer");
+    }
+  }
   {{#if logging_endpoint}}
   ## Debug DataDome
   log {"syslog "} req.service_id {" {{logging_endpoint}} :: "}
       " timestamp=%22" now
       "%22 client_ip=" req.http.Fastly-Client-IP
       " request=" req.method
       " host=" req.http.host
       " url=%22" cstr_escape(req.url)
       "%22 request_referer=%22" cstr_escape(req.http.Referer)
       "%22 request_user_agent=%22" cstr_escape(req.http.User-Agent)
       "%22 request_accept_language=%22" cstr_escape(req.http.Accept-Language)
       "%22 request_accept_charset=%22" cstr_escape(req.http.Accept-Charset)
       "%22 contentLength=" req.http.Content-Length
       " restarts=" req.restarts
       " DataDomeDebug=" "To_DataDome"
       " fastlyFF=" fastly.ff.visits_this_service;
   ##
   {{/if}}
   return (pass);
 } else {
   if (req.http.x-datadome-params:method) {
     set req.method = urldecode(req.http.x-datadome-params:method);
     # After a restart, clustering is disabled. This re-enables it.
     set req.http.fastly-force-shield = "1";
   }
   unset req.http.x-datadome-params;
   {{#if logging_endpoint}}
   ## Debug DataDome
   log {"syslog "} req.service_id {" {{logging_endpoint}} :: "}
       " timestamp=%22" now
       "%22 client_ip=" req.http.Fastly-Client-IP
       " request=" req.method
       " host=" req.http.host
       " url=%22" cstr_escape(req.url)
       "%22 request_referer=%22" cstr_escape(req.http.Referer)
       "%22 request_user_agent=%22" cstr_escape(req.http.User-Agent)
       "%22 request_accept_language=%22" cstr_escape(req.http.Accept-Language)
       "%22 request_accept_charset=%22" cstr_escape(req.http.Accept-Charset)
       "%22 contentLength=" req.http.Content-Length
       " restarts=" req.restarts
       " DataDomeDebug=" "Bypass_DataDome"
       " fastlyFF=" fastly.ff.visits_this_service;
   ##
   {{/if}}
 }
 
 # we\u0027re using the first restart for datadome, update a part of fastly code
 # we can\u0027t replace whole macros because we haven\u0027t got any idea about backends
 if (req.restarts == 1) {
   if (!req.http.x-timer) {
     set req.http.x-timer = "S" time.start.sec "." time.start.usec_frac;
   }
   set req.http.x-timer = req.http.x-timer ",VS0";
 }
 
 set var.fastly_req_do_shield = (req.restarts \u003c= 1);

It makes no changes to miss.

It makes no changes to pass.

Copy link
Member

@harmony7 harmony7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me.

@harmony7 harmony7 merged commit a1e73a8 into fastly:master Nov 20, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants