[DENG-9376] Publish DOH adoption and usage rates #8238

scholtzan · 2025-10-09T19:44:46Z

Description

https://bugzilla.mozilla.org/show_bug.cgi?id=1982897

This query aggregates and publishes DOH adoption and usage rates.
@edugfilho This is similar to #7367, so you're probably the best person to review. I'll file a separate PR to backfill this once the tables have been deployed.

Related Tickets & Documents

https://mozilla-hub.atlassian.net/browse/DENG-9376

Reviewer, please follow this checklist

sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/schema.yaml

BenWu · 2025-10-16T19:27:19Z

sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/query.sql

+  SELECT
+    DATE(submission_timestamp) AS submission_date,
+    'doh.evaluate_v2_heuristics' AS metric,
+    "value" AS key,
+    COALESCE(extra.value, '') AS value,
+    CASE
+      WHEN COUNT(DISTINCT client_info.client_id) >= 5000
+        THEN normalized_country_code
+      ELSE 'OTHER'
+    END AS country_code,
+    COUNT(DISTINCT client_info.client_id) AS client_count
+  FROM
+    `moz-fx-data-shared-prod.firefox_desktop.events` AS e,
+    UNNEST(e.events) AS event,
+    UNNEST(event.extra) AS extra
+  WHERE
+    DATE(submission_timestamp) = @submission_date
+    AND client_info.client_id IS NOT NULL
+    AND client_info.app_channel = 'release'
+    AND event.category = 'doh'
+    AND event.name = 'evaluate_v2_heuristics'
+    AND extra.key = 'value'
+  GROUP BY
+    submission_date,
+    key,
+    value,
+    normalized_country_code


These queries should be a lot more efficient with events stream. I think this is equivalent:

SELECT DATE(submission_timestamp) AS submission_date, 'doh.evaluate_v2_heuristics' AS metric, "value" AS key, COALESCE(STRING(event_extra.value), '') AS value, CASE WHEN COUNT(DISTINCT client_id) >= 5000 THEN normalized_country_code ELSE 'OTHER' END AS country_code, COUNT(DISTINCT client_id) AS client_count FROM `moz-fx-data-shared-prod.firefox_desktop.events_stream` AS e WHERE DATE(submission_timestamp) = @submission_date AND client_id IS NOT NULL AND client_info.app_channel = 'release' AND event_category = 'doh' AND event_name = 'evaluate_v2_heuristics' GROUP BY submission_date, key, value, normalized_country_code

sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/query.sql

BenWu

Missing comma in the query but otherwise LGTM

…e_v1/query.sql Co-authored-by: Ben Wu <[email protected]>

dataops-ci-bot · 2025-10-16T21:24:24Z

Integration report for "Update sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/query.sql"

`sql.diff`

Click to expand!

diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_ech_adoption_rate.py /tmp/workspace/generated-sql/dags/bqetl_ech_adoption_rate.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_ech_adoption_rate.py	2025-10-16 21:20:17.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_ech_adoption_rate.py	2025-10-16 21:23:06.000000000 +0000
@@ -68,6 +68,34 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
+    wait_for_firefox_desktop_derived__events_stream__v1 = ExternalTaskSensor(
+        task_id="wait_for_firefox_desktop_derived__events_stream__v1",
+        external_dag_id="bqetl_glean_usage",
+        external_task_id="firefox_desktop.firefox_desktop_derived__events_stream__v1",
+        execution_delta=datetime.timedelta(days=-1, seconds=79200),
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    telemetry_derived__doh_adoption_rate__v1 = bigquery_etl_query(
+        task_id="telemetry_derived__doh_adoption_rate__v1",
+        destination_table="doh_adoption_rate_v1",
+        dataset_id="telemetry_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="[email protected]",
+        email=[
+            "[email protected]",
+            "[email protected]",
+            "[email protected]",
+        ],
+        date_partition_parameter="submission_date",
+        depends_on_past=False,
+    )
+
     telemetry_derived__ech_adoption_rate__v1 = bigquery_etl_query(
         task_id="telemetry_derived__ech_adoption_rate__v1",
         destination_table="ech_adoption_rate_v1",
@@ -79,4 +107,10 @@
         depends_on_past=False,
     )
 
+    telemetry_derived__doh_adoption_rate__v1.set_upstream(wait_for_copy_deduplicate_all)
+
+    telemetry_derived__doh_adoption_rate__v1.set_upstream(
+        wait_for_firefox_desktop_derived__events_stream__v1
+    )
+
     telemetry_derived__ech_adoption_rate__v1.set_upstream(wait_for_copy_deduplicate_all)
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_glean_usage.py /tmp/workspace/generated-sql/dags/bqetl_glean_usage.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_glean_usage.py	2025-10-16 21:20:22.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_glean_usage.py	2025-10-16 21:23:10.000000000 +0000
@@ -3899,12 +3899,26 @@
         parent_group=task_group_firefox_desktop,
     ) as firefox_desktop_derived__events_stream__v1_external:
         ExternalTaskMarker(
+            task_id="bqetl_public_data_json__wait_for_firefox_desktop_derived__events_stream__v1",
+            external_dag_id="bqetl_public_data_json",
+            external_task_id="wait_for_firefox_desktop_derived__events_stream__v1",
+            execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=75600)).isoformat() }}",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_experiments_daily__wait_for_firefox_desktop_derived__events_stream__v1",
             external_dag_id="bqetl_experiments_daily",
             external_task_id="wait_for_firefox_desktop_derived__events_stream__v1",
             execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=82800)).isoformat() }}",
         )
 
+        ExternalTaskMarker(
+            task_id="bqetl_ech_adoption_rate__wait_for_firefox_desktop_derived__events_stream__v1",
+            external_dag_id="bqetl_ech_adoption_rate",
+            external_task_id="wait_for_firefox_desktop_derived__events_stream__v1",
+            execution_date="{{ (execution_date - macros.timedelta(seconds=7200)).isoformat() }}",
+        )
+
         firefox_desktop_derived__events_stream__v1_external.set_upstream(
             firefox_desktop_derived__events_stream__v1
         )
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_public_data_json.py /tmp/workspace/generated-sql/dags/bqetl_public_data_json.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_public_data_json.py	2025-10-16 21:20:12.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_public_data_json.py	2025-10-16 21:22:57.000000000 +0000
@@ -151,6 +151,20 @@
         image=docker_image,
     )
 
+    export_public_data_json_telemetry_derived__doh_adoption_rate__v1 = GKEPodOperator(
+        task_id="export_public_data_json_telemetry_derived__doh_adoption_rate__v1",
+        name="export_public_data_json_telemetry_derived__doh_adoption_rate__v1",
+        arguments=["script/publish_public_data_json"]
+        + [
+            "--query_file=sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/query.sql"
+        ]
+        + ["--destination_table=doh_adoption_rate${{ds_nodash}}"]
+        + ["--dataset_id=telemetry_derived"]
+        + ["--project_id=moz-fx-data-shared-prod"]
+        + ["--parameter=submission_date:DATE:{{ds}}"],
+        image=docker_image,
+    )
+
     export_public_data_json_telemetry_derived__ech_adoption_rate__v1 = GKEPodOperator(
         task_id="export_public_data_json_telemetry_derived__ech_adoption_rate__v1",
         name="export_public_data_json_telemetry_derived__ech_adoption_rate__v1",
@@ -221,6 +235,26 @@
         wait_for_copy_deduplicate_all
     )
 
+    export_public_data_json_telemetry_derived__doh_adoption_rate__v1.set_upstream(
+        wait_for_copy_deduplicate_all
+    )
+    wait_for_firefox_desktop_derived__events_stream__v1 = ExternalTaskSensor(
+        task_id="wait_for_firefox_desktop_derived__events_stream__v1",
+        external_dag_id="bqetl_glean_usage",
+        external_task_id="firefox_desktop.firefox_desktop_derived__events_stream__v1",
+        execution_delta=datetime.timedelta(seconds=10800),
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    export_public_data_json_telemetry_derived__doh_adoption_rate__v1.set_upstream(
+        wait_for_firefox_desktop_derived__events_stream__v1
+    )
+
     export_public_data_json_telemetry_derived__ech_adoption_rate__v1.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -257,6 +291,7 @@
             export_public_data_json_glam_derived__client_probe_counts_firefox_desktop_nightly__v1,
             export_public_data_json_glean_auto_events_derived__apps_auto_events_metadata__v1,
             export_public_data_json_mozregression_aggregates__v1,
+            export_public_data_json_telemetry_derived__doh_adoption_rate__v1,
             export_public_data_json_telemetry_derived__ech_adoption_rate__v1,
             export_public_data_json_telemetry_derived__ssl_ratios__v1,
         ]
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry: doh_adoption_rate
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived: doh_adoption_rate_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/doh_adoption_rate/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/doh_adoption_rate/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/doh_adoption_rate/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/doh_adoption_rate/metadata.yaml	2025-10-16 21:16:42.000000000 +0000
@@ -0,0 +1,15 @@
+friendly_name: Doh Adoption Rate
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:dataops-managed/external-fides
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.telemetry_derived.doh_adoption_rate_v1
+require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/doh_adoption_rate/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/doh_adoption_rate/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/doh_adoption_rate/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/doh_adoption_rate/view.sql	2025-10-16 21:13:48.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.telemetry.doh_adoption_rate`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.telemetry_derived.doh_adoption_rate_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/metadata.yaml	2025-10-16 21:16:43.000000000 +0000
@@ -0,0 +1,33 @@
+friendly_name: DOH Adoption Rate
+description: |-
+  A distinct client count of a combination of metrics to measure
+  the adoption of DOH (DNS over HTTPS) in Firefox.
+owners:
+- [email protected]
+labels:
+  incremental: true
+  owner1: ascholtz
+  public_bigquery: true
+  public_json: true
+  review_bugs:
+  - '1982897'
+  dag: bqetl_ech_adoption_rate
+scheduling:
+  dag_name: bqetl_ech_adoption_rate
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: true
+    expiration_days: null
+  range_partitioning: null
+  clustering: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  query.sql:
+  - moz-fx-data-shared-prod.firefox_desktop.events_stream
+  - moz-fx-data-shared-prod.firefox_desktop.metrics
+require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/query.sql	2025-10-16 21:13:48.000000000 +0000
@@ -0,0 +1,253 @@
+WITH unioned AS (
+  /* ---------- doh.evaluate_v2_heuristics ---------- */
+  SELECT
+    DATE(submission_timestamp) AS submission_date,
+    'doh.evaluate_v2_heuristics' AS metric,
+    "value" AS key,
+    COALESCE(STRING(event_extra.value), '') AS value,
+    CASE
+      WHEN COUNT(DISTINCT client_id) >= 5000
+        THEN normalized_country_code
+      ELSE 'OTHER'
+    END AS country_code,
+    COUNT(DISTINCT client_id) AS client_count
+  FROM
+    `moz-fx-data-shared-prod.firefox_desktop.events_stream`
+  WHERE
+    DATE(submission_timestamp) = @submission_date
+    AND client_id IS NOT NULL
+    AND client_info.app_channel = 'release'
+    AND event_category = 'doh'
+    AND event_name = 'evaluate_v2_heuristics'
+  GROUP BY
+    submission_date,
+    key,
+    value,
+    normalized_country_code
+  UNION ALL
+  /* ---------- doh.state_disabled ---------- */
+  SELECT
+    DATE(submission_timestamp) AS submission_date,
+    'doh.state_disabled' AS metric,
+    '' AS key,
+    '' AS value,
+    CASE
+      WHEN COUNT(DISTINCT client_id) >= 5000
+        THEN normalized_country_code
+      ELSE 'OTHER'
+    END AS country_code,
+    COUNT(DISTINCT client_id) AS client_count
+  FROM
+    `moz-fx-data-shared-prod.firefox_desktop.events_stream`
+  WHERE
+    DATE(submission_timestamp) = @submission_date
+    AND client_id IS NOT NULL
+    AND client_info.app_channel = 'release'
+    AND event_category = 'doh'
+    AND event_name = 'state_disabled'
+  GROUP BY
+    submission_date,
+    key,
+    value,
+    normalized_country_code
+  UNION ALL
+  /* ---------- doh.state_enabled ---------- */
+  SELECT
+    DATE(submission_timestamp) AS submission_date,
+    'doh.state_enabled' AS metric,
+    '' AS key,
+    '' AS value,
+    CASE
+      WHEN COUNT(DISTINCT client_id) >= 5000
+        THEN normalized_country_code
+      ELSE 'OTHER'
+    END AS country_code,
+    COUNT(DISTINCT client_id) AS client_count
+  FROM
+    `moz-fx-data-shared-prod.firefox_desktop.events_stream`
+  WHERE
+    DATE(submission_timestamp) = @submission_date
+    AND client_id IS NOT NULL
+    AND client_info.app_channel = 'release'
+    AND event_category = 'doh'
+    AND event_name = 'state_enabled'
+  GROUP BY
+    submission_date,
+    key,
+    value,
+    normalized_country_code
+  UNION ALL
+  /* ---------- doh.state_manually_disabled ---------- */
+  SELECT
+    DATE(submission_timestamp) AS submission_date,
+    'doh.state_manually_disabled' AS metric,
+    '' AS key,
+    '' AS value,
+    CASE
+      WHEN COUNT(DISTINCT client_id) >= 5000
+        THEN normalized_country_code
+      ELSE 'OTHER'
+    END AS country_code,
+    COUNT(DISTINCT client_id) AS client_count
+  FROM
+    `moz-fx-data-shared-prod.firefox_desktop.events_stream`
+  WHERE
+    DATE(submission_timestamp) = @submission_date
+    AND client_id IS NOT NULL
+    AND client_info.app_channel = 'release'
+    AND event_category = 'doh'
+    AND event_name = 'state_manually_disabled'
+  GROUP BY
+    submission_date,
+    key,
+    value,
+    normalized_country_code
+  UNION ALL
+  /* ---------- doh.state_policy_disabled ---------- */
+  SELECT
+    DATE(submission_timestamp) AS submission_date,
+    'doh.state_policy_disabled' AS metric,
+    '' AS key,
+    '' AS value,
+    CASE
+      WHEN COUNT(DISTINCT client_id) >= 5000
+        THEN normalized_country_code
+      ELSE 'OTHER'
+    END AS country_code,
+    COUNT(DISTINCT client_id) AS client_count
+  FROM
+    `moz-fx-data-shared-prod.firefox_desktop.events_stream`
+  WHERE
+    DATE(submission_timestamp) = @submission_date
+    AND client_id IS NOT NULL
+    AND client_info.app_channel = 'release'
+    AND event_category = 'doh'
+    AND event_name = 'state_policy_disabled'
+  GROUP BY
+    submission_date,
+    key,
+    value,
+    normalized_country_code
+  UNION ALL
+  /* ---------- networking.doh_heuristics_attempts ---------- */
+  SELECT
+    DATE(submission_timestamp) AS submission_date,
+    'networking.doh_heuristics_attempts' AS metric,
+    'networking.doh_heuristics_attempts' AS key,
+    CAST(metrics.counter.networking_doh_heuristics_attempts AS STRING) AS value,
+    CASE
+      WHEN COUNT(DISTINCT client_info.client_id) >= 5000
+        THEN metadata.geo.country
+      ELSE 'OTHER'
+    END AS country_code,
+    COUNT(DISTINCT client_info.client_id) AS client_count
+  FROM
+    `moz-fx-data-shared-prod.firefox_desktop.metrics`
+  WHERE
+    DATE(submission_timestamp) = @submission_date
+    AND client_info.client_id IS NOT NULL
+    AND client_info.app_channel = 'release'
+    AND metrics.counter.networking_doh_heuristics_attempts IS NOT NULL
+  GROUP BY
+    submission_date,
+    key,
+    value,
+    metadata.geo.country
+  UNION ALL
+  /* ---------- networking.doh_heuristics_pass_count ---------- */
+  SELECT
+    DATE(submission_timestamp) AS submission_date,
+    'networking.doh_heuristics_pass_count' AS metric,
+    'networking.doh_heuristics_attempts' AS key,
+    CAST(metrics.counter.networking_doh_heuristics_pass_count AS STRING) AS value,
+    CASE
+      WHEN COUNT(DISTINCT client_info.client_id) >= 5000
+        THEN metadata.geo.country
+      ELSE 'OTHER'
+    END AS country_code,
+    COUNT(DISTINCT client_info.client_id) AS client_count
+  FROM
+    `moz-fx-data-shared-prod.firefox_desktop.metrics`
+  WHERE
+    DATE(submission_timestamp) = @submission_date
+    AND client_info.client_id IS NOT NULL
+    AND client_info.app_channel = 'release'
+    AND metrics.counter.networking_doh_heuristics_pass_count IS NOT NULL
+  GROUP BY
+    submission_date,
+    key,
+    value,
+    metadata.geo.country
+  UNION ALL
+  /* ---------- networking.doh_heuristics_result ---------- */
+  SELECT
+    DATE(submission_timestamp) AS submission_date,
+    'networking.doh_heuristics_result' AS metric,
+    'networking.doh_heuristics_result' AS key,
+    CAST(metrics.quantity.networking_doh_heuristics_result AS STRING) AS value,
+    CASE
+      WHEN COUNT(DISTINCT client_info.client_id) >= 5000
+        THEN metadata.geo.country
+      ELSE 'OTHER'
+    END AS country_code,
+    COUNT(DISTINCT client_info.client_id) AS client_count
+  FROM
+    `moz-fx-data-shared-prod.firefox_desktop.metrics`
+  WHERE
+    DATE(submission_timestamp) = @submission_date
+    AND client_info.client_id IS NOT NULL
+    AND client_info.app_channel = 'release'
+    AND metrics.quantity.networking_doh_heuristics_result IS NOT NULL
+  GROUP BY
+    submission_date,
+    key,
+    value,
+    metadata.geo.country
+  UNION ALL
+  /* ---------- security.doh.settings.provider_choice_value ---------- */
+  SELECT
+    DATE(submission_timestamp) AS submission_date,
+    'security.doh.settings.provider_choice_value' AS metric,
+    "value" AS key,
+    COALESCE(STRING(event_extra.value), '') AS value,
+    CASE
+      WHEN COUNT(DISTINCT client_id) >= 5000
+        THEN normalized_country_code
+      ELSE 'OTHER'
+    END AS country_code,
+    COUNT(DISTINCT client_id) AS client_count
+  FROM
+    `moz-fx-data-shared-prod.firefox_desktop.events_stream`
+  WHERE
+    DATE(submission_timestamp) = @submission_date
+    AND client_id IS NOT NULL
+    AND client_info.app_channel = 'release'
+    AND event_category = 'security.doh.settings'
+    AND event_name = 'provider_choice_value'
+  GROUP BY
+    submission_date,
+    key,
+    value,
+    normalized_country_code
+)
+SELECT
+  submission_date,
+  metric,
+  key,
+  value,
+  country_code,
+  SUM(client_count) AS total_client_count
+FROM
+  unioned
+GROUP BY
+  submission_date,
+  metric,
+  key,
+  value,
+  country_code
+ORDER BY
+  submission_date DESC,
+  metric,
+  country_code,
+  key,
+  value
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/schema.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/schema.yaml	2025-10-16 21:13:48.000000000 +0000
@@ -0,0 +1,39 @@
+fields:
+- mode: NULLABLE
+  name: submission_date
+  type: DATE
+  description: The date when the telemetry data was submitted to the server.
+- mode: NULLABLE
+  name: metric
+  type: STRING
+  description: The specific DOH (DNS-over-HTTPS) metric being measured. Can be one
+    of doh.evaluate_v2_heuristics (DOH heuristics evaluation), doh.state_disabled
+    (DOH disabled state), doh.state_enabled (DOH enabled state), doh.state_manually_disabled
+    (DOH manually disabled by user), doh.state_policy_disabled (DOH disabled by policy),
+    networking.doh_heuristics_attempts (count of heuristics attempts), networking.doh_heuristics_pass_count
+    (count of heuristics that passed), networking.doh_heuristics_result (result code
+    of heuristics evaluation), or security.doh.settings.provider_choice_value (selected
+    DOH provider).
+- mode: NULLABLE
+  name: key
+  type: STRING
+  description: The key or dimension associated with the metric value. For event-based
+    metrics with extra fields, this represents the field name (e.g., 'value'). For
+    counter/quantity metrics, this is typically the metric name itself.
+- mode: NULLABLE
+  name: value
+  type: STRING
+  description: The value associated with the metric. For event extra fields, this
+    contains the extra field value. For counters and quantities, this contains the
+    stringified numeric value. May be empty for events without extra fields.
+- mode: NULLABLE
+  name: country_code
+  type: STRING
+  description: The ISO country code where the client is located, normalized by geolocation.
+    Countries with fewer than 5000 distinct clients are aggregated into 'OTHER' for
+    privacy reasons.
+- mode: NULLABLE
+  name: total_client_count
+  type: INTEGER
+  description: The total count of distinct clients (by client_id) that reported this
+    metric, key, value, and country combination on the submission date.

Link to full diff

[DENG-9376] Publish DOH adoption and usage rates

4164f19

scholtzan requested a review from edugfilho October 9, 2025 19:44

This comment has been minimized.

Sign in to view

BenWu reviewed Oct 16, 2025

View reviewed changes

kwindau approved these changes Oct 16, 2025

View reviewed changes

Add schema and use events_stream table

12406eb

scholtzan force-pushed the publish-doh-aggregates branch from edbd732 to 12406eb Compare October 16, 2025 20:40

Merge branch 'main' into publish-doh-aggregates

ed3f64e

This comment has been minimized.

Sign in to view

BenWu reviewed Oct 16, 2025

View reviewed changes

sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/query.sql Outdated Show resolved Hide resolved

BenWu reviewed Oct 16, 2025

View reviewed changes

Update sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rat…

9861060

…e_v1/query.sql Co-authored-by: Ben Wu <[email protected]>

scholtzan added this pull request to the merge queue Oct 20, 2025

Any commits made after this event will not be merged.

github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Oct 20, 2025

scholtzan added this pull request to the merge queue Oct 20, 2025

Any commits made after this event will not be merged.

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 20, 2025

scholtzan added this pull request to the merge queue Oct 21, 2025

Any commits made after this event will not be merged.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[DENG-9376] Publish DOH adoption and usage rates #8238

[DENG-9376] Publish DOH adoption and usage rates #8238

scholtzan commented Oct 9, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

Uh oh!

BenWu Oct 16, 2025

Uh oh!

This comment has been minimized.

Uh oh!

BenWu left a comment

Uh oh!

dataops-ci-bot commented Oct 16, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[DENG-9376] Publish DOH adoption and usage rates #8238

[DENG-9376] Publish DOH adoption and usage rates #8238

Conversation

scholtzan commented Oct 9, 2025

Description

Related Tickets & Documents

Uh oh!

This comment has been minimized.

This comment has been minimized.

Uh oh!

BenWu Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

Uh oh!

BenWu left a comment

Choose a reason for hiding this comment

Uh oh!

dataops-ci-bot commented Oct 16, 2025

Integration report for "Update sql/moz-fx-data-shared-prod/telemetry_derived/doh_adoption_rate_v1/query.sql"

sql.diff

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

`sql.diff`