-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* blink features report * tested * no reference * formatted date * Update README.md Co-authored-by: Barry Pollard <[email protected]> --------- Co-authored-by: Barry Pollard <[email protected]>
- Loading branch information
1 parent
4f09997
commit 65f098c
Showing
10 changed files
with
224 additions
and
47 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
|
|
||
| publish("features", { | ||
| schema: "blink_features", | ||
| type: "incremental", | ||
| protected: true, | ||
| bigquery: { | ||
| partitionBy: "yyyymmdd", | ||
| clusterBy: ["client", "rank"] | ||
| }, | ||
| tags: ["blink_features_report"] | ||
| }).preOps(ctx => ` | ||
| DELETE FROM ${ctx.self()} | ||
| WHERE yyyymmdd = DATE '${constants.current_month}'; | ||
| CREATE TEMP FUNCTION features(payload STRING) | ||
| RETURNS ARRAY<STRUCT<id STRING, name STRING, type STRING>> LANGUAGE js AS | ||
| """ | ||
| function getFeatureNames(featureMap, featureType) { | ||
| try { | ||
| return Object.entries(featureMap).map(([key, value]) => { | ||
| // After Feb 2020 keys are feature IDs. | ||
| if (value.name) { | ||
| return {'name': value.name, 'type': featureType, 'id': key}; | ||
| } | ||
| // Prior to Feb 2020 keys fell back to IDs if the name was unknown. | ||
| if (idPattern.test(key)) { | ||
| return {'name': '', 'type': featureType, 'id': key.match(idPattern)[1]}; | ||
| } | ||
| // Prior to Feb 2020 keys were names by default. | ||
| return {'name': key, 'type': featureType, 'id': ''}; | ||
| }); | ||
| } catch (e) { | ||
| return []; | ||
| } | ||
| } | ||
| var $ = JSON.parse(payload); | ||
| if (!$._blinkFeatureFirstUsed) return []; | ||
| var idPattern = new RegExp('^Feature_(\d+)$'); | ||
| return getFeatureNames($._blinkFeatureFirstUsed.Features, 'default') | ||
| .concat(getFeatureNames($._blinkFeatureFirstUsed.CSSFeatures, 'css')) | ||
| .concat(getFeatureNames($._blinkFeatureFirstUsed.AnimatedCSSFeatures, 'animated-css')); | ||
| """; | ||
| `).query(ctx => ` | ||
| SELECT | ||
| date AS yyyymmdd, | ||
| client, | ||
| url, | ||
| feature.feature AS feature, | ||
| feature.type, | ||
| feature.id, | ||
| rank | ||
| FROM ( | ||
| SELECT | ||
| date, | ||
| client, | ||
| page AS url, | ||
| payload, | ||
| rank, | ||
| feature | ||
| FROM ${ctx.ref("all", "pages")}, | ||
| UNNEST(features) AS feature | ||
| WHERE | ||
| date = '${constants.current_month}' AND | ||
| is_root_page = TRUE | ||
| ${constants.dev_rank_filter} | ||
| ) | ||
| `) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| publish("usage", { | ||
| schema: "blink_features", | ||
| type: "incremental", | ||
| protected: true, | ||
| tags: ["blink_features_report"] | ||
| }).preOps(ctx => ` | ||
| DELETE FROM ${ctx.self()} | ||
| WHERE yyyymmdd = REPLACE('${constants.current_month}', '-', ''); | ||
| `).query(ctx => ` | ||
| SELECT | ||
| REPLACE(CAST(date AS STRING), '-', '') AS yyyymmdd, | ||
| client, | ||
| id, | ||
| feature, | ||
| type, | ||
| num_urls, | ||
| total_urls, | ||
| num_urls / total_urls AS pct_urls, | ||
| sample_urls | ||
| FROM ( | ||
| SELECT | ||
| yyyymmdd AS date, | ||
| client, | ||
| id, | ||
| feature, | ||
| type, | ||
| COUNT(DISTINCT url) AS num_urls, | ||
| ARRAY_AGG(url ORDER BY rank, url LIMIT 100) AS sample_urls | ||
| FROM ${ctx.ref("blink_features", "features")} | ||
| WHERE | ||
| yyyymmdd = '${constants.current_month}' | ||
| ${constants.dev_rank_filter} | ||
| GROUP BY | ||
| yyyymmdd, | ||
| client, | ||
| id, | ||
| feature, | ||
| type | ||
| ) | ||
| JOIN ( | ||
| SELECT | ||
| date, | ||
| client, | ||
| COUNT(DISTINCT page) AS total_urls | ||
| FROM ${ctx.ref("all", "pages")} | ||
| WHERE | ||
| date = '${constants.current_month}' AND | ||
| is_root_page = TRUE | ||
| ${constants.dev_rank_filter} | ||
| GROUP BY | ||
| date, | ||
| client | ||
| ) | ||
| USING (date, client) | ||
| `) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters