Skip to content

Conversation

@scholtzan
Copy link
Collaborator

Description

Dry running CREATE VIEW statements is potentially faster as the dry run doesn’t need to access any partition metadata. This PR is re-writing queries as CREATE VIEW statements

Related Tickets & Documents

https://mozilla-hub.atlassian.net/browse/DENG-9705

Reviewer, please follow this checklist

@scholtzan scholtzan force-pushed the dryrun-view-dml branch 3 times, most recently from 037e8c6 to 9f0881f Compare October 27, 2025 22:16
@dataops-ci-bot

This comment has been minimized.

@dataops-ci-bot

This comment has been minimized.

@scholtzan scholtzan force-pushed the dryrun-view-dml branch 2 times, most recently from 8c899d6 to d643564 Compare October 28, 2025 18:01
@dataops-ci-bot

This comment has been minimized.

@dataops-ci-bot
Copy link

Integration report for "[run-tests] Substitute UDFs"

sql.diff

Click to expand!
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:49:24.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:52:42.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-27"
+  DATE(submission_timestamp) >= "2025-10-28"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:52:42.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-27"
+  DATE(submission_timestamp) >= "2025-10-28"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:52:42.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-27"
+  DATE(submission_timestamp) >= "2025-10-28"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/analysis/bqetl_default_task_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/analysis/bqetl_default_task_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/analysis/bqetl_default_task_v1/query.sql	2025-10-28 18:49:24.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/analysis/bqetl_default_task_v1/query.sql	2025-10-28 18:48:16.000000000 +0000
@@ -2,6 +2,6 @@
 SELECT
   *
 FROM
-  (SELECT 1)
+  (SELECT 1 AS field)
 WHERE
   FALSE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:52:42.000000000 +0000
@@ -119,7 +119,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-27"
+  DATE(submission_timestamp) >= "2025-10-28"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-28 19:02:08.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-28 19:02:08.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-28 19:02:08.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-28 19:02:08.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-10-28 19:02:08.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-10-28 18:54:07.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-10-28 18:54:09.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:52:43.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-27"
+  DATE(submission_timestamp) >= "2025-10-28"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -51,7 +51,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:52:43.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-27"
+  DATE(submission_timestamp) >= "2025-10-28"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-28 19:02:09.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -51,7 +51,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-28 18:52:43.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-27"
+  DATE(submission_timestamp) >= "2025-10-28"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-28 19:02:10.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml	2025-10-28 19:02:09.000000000 +0000
@@ -1,6 +1,10 @@
-friendly_name: Broken Site Report
+friendly_name: App-specific view for Glean ping "broken-site-report"
 description: |-
-  Please provide a description for the query
+  This a view that UNIONs the stable ping tables
+  across all channels of the Glean application "Firefox for Android"
+  (org_mozilla_firefox.broken_site_report, org_mozilla_firefox_beta.broken_site_report, org_mozilla_fenix.broken_site_report, org_mozilla_fenix_nightly.broken_site_report, org_mozilla_fennec_aurora.broken_site_report).
+
+  It is used by Looker.
 owners: []
 labels: {}
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml	2025-10-28 19:02:08.000000000 +0000
@@ -1,6 +1,10 @@
-friendly_name: Crash
+friendly_name: App-specific view for Glean ping "crash"
 description: |-
-  Please provide a description for the query
+  This a view that UNIONs the stable ping tables
+  across all channels of the Glean application "Firefox for Android"
+  (org_mozilla_firefox.crash, org_mozilla_firefox_beta.crash, org_mozilla_fenix.crash, org_mozilla_fenix_nightly.crash, org_mozilla_fennec_aurora.crash).
+
+  It is used by Looker.
 owners: []
 labels: {}
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-10-28 18:54:37.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-10-28 18:54:28.000000000 +0000
@@ -48,6 +48,9 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml	2025-10-28 18:59:14.000000000 +0000
@@ -12674,7 +12674,8 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Failure occurs when initializing the audio stream.
+      description: 'Failure occurs when initializing the audio stream. (Migrated from
+        the geckoview metric of the same name).
 
         '
     - name: glean_validation_pings_submitted
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/migration/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/migration/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/migration/schema.yaml	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/migration/schema.yaml	2025-10-28 18:59:16.000000000 +0000
@@ -22,86 +22,54 @@
   - name: android_sdk_version
     type: STRING
     mode: NULLABLE
-    description: The optional Android specific SDK version of the software running
-      on this hardware device.
   - name: app_build
     type: STRING
     mode: NULLABLE
-    description: The build identifier generated by the CI system (e.g. "1234/A").
-      For language bindings that provide automatic detection for this value, (e.g.
-      Android/Kotlin), in the unlikely event that the build identifier can not be
-      retrieved from the OS, it is set to "inaccessible". For other language bindings,
-      if the value was not provided through configuration, this metric gets set to
-      `Unknown`.
   - name: app_channel
     type: STRING
     mode: NULLABLE
-    description: The channel the application is being distributed on.
   - name: app_display_version
     type: STRING
     mode: NULLABLE
-    description: The user visible version string (e.g. "1.0.3").  In the unlikely
-      event that the display version can not be retrieved, it is set to "inaccessible".
   - name: architecture
     type: STRING
     mode: NULLABLE
-    description: The architecture of the device, (e.g. "arm", "x86").
   - name: client_id
     type: STRING
     mode: NULLABLE
-    description: A UUID uniquely identifying the client.
   - name: device_manufacturer
     type: STRING
     mode: NULLABLE
-    description: The manufacturer of the device the application is running on. Not
-      set if the device manufacturer can't be determined (e.g. on Desktop).
   - name: device_model
     type: STRING
     mode: NULLABLE
-    description: The model of the device the application is running on. On Android,
-      this is Build.MODEL, the user-visible marketing name, like "Pixel 2 XL". Not
-      set if the device model can't be determined (e.g. on Desktop).
   - name: first_run_date
     type: STRING
     mode: NULLABLE
-    description: The date of the first run of the application.
   - name: locale
     type: STRING
     mode: NULLABLE
-    description: The locale of the application during initialization (e.g. "es-ES").
-      If the locale can't be determined on the system, the value is ["und"](https://unicode.org/reports/tr35/#Unknown_or_Invalid_Identifiers),
-      to indicate "undetermined".
   - name: os
     type: STRING
     mode: NULLABLE
-    description: 'The name of the operating system. Possible values: Android, iOS,
-      Linux, Darwin, Windows, FreeBSD, NetBSD, OpenBSD, Solaris, unknown'
   - name: os_version
     type: STRING
     mode: NULLABLE
-    description: The user-visible version of the operating system (e.g. "1.2.3").
-      If the version detection fails, this metric gets set to `Unknown`.
   - name: telemetry_sdk_build
     type: STRING
     mode: NULLABLE
-    description: The version of the Glean SDK
   - name: build_date
     type: STRING
     mode: NULLABLE
-    description: The date & time the application was built
   - name: windows_build_number
     type: INTEGER
     mode: NULLABLE
-    description: The optional Windows build number, reported by Windows (e.g. 22000)
-      and not set for other platforms
   - name: session_count
     type: INTEGER
     mode: NULLABLE
-    description: An optional running counter of the number of sessions for a client.
   - name: session_id
     type: STRING
     mode: NULLABLE
-    description: An optional UUID uniquely identifying the client's current session.
   - name: attribution
     type: RECORD
     mode: NULLABLE
@@ -109,23 +77,18 @@
     - name: campaign
       type: STRING
       mode: NULLABLE
-      description: The attribution campaign (e.g. 'mozilla-org').
     - name: content
       type: STRING
       mode: NULLABLE
-      description: The attribution content (e.g. 'firefoxview').
     - name: medium
       type: STRING
       mode: NULLABLE
-      description: The attribution medium (e.g. 'organic' for a search engine).
     - name: source
       type: STRING
       mode: NULLABLE
-      description: The attribution source (e.g. 'google-play').
     - name: term
       type: STRING
       mode: NULLABLE
-      description: The attribution term (e.g. 'browser with developer tools for android').
     - name: ext
       type: JSON
       mode: NULLABLE
@@ -136,7 +99,6 @@
     - name: name
       type: STRING
       mode: NULLABLE
-      description: The distribution name (e.g. 'MozillaOnline').
     - name: ext
       type: JSON
       mode: NULLABLE
@@ -181,21 +143,15 @@
     - name: country
       type: STRING
       mode: NULLABLE
-      description: An ISO 3166-1 alpha-2 country code
     - name: db_version
       type: STRING
       mode: NULLABLE
-      description: The specific geo database version used for this lookup
     - name: subdivision1
       type: STRING
       mode: NULLABLE
-      description: First major country subdivision, typically a state, province, or
-        county
     - name: subdivision2
       type: STRING
       mode: NULLABLE
-      description: Second major country subdivision; not applicable for most countries
-    description: Results of a geographic lookup based on the client's IP address
   - name: header
     type: RECORD
     mode: NULLABLE
@@ -203,35 +159,27 @@
     - name: date
       type: STRING
       mode: NULLABLE
-      description: Date HTTP header
     - name: dnt
       type: STRING
       mode: NULLABLE
-      description: DNT (Do Not Track) HTTP header
     - name: x_debug_id
       type: STRING
       mode: NULLABLE
-      description: X-Debug-Id HTTP header
     - name: x_pingsender_version
       type: STRING
       mode: NULLABLE
-      description: X-PingSender-Version HTTP header
     - name: x_source_tags
       type: STRING
       mode: NULLABLE
-      description: X-Source-Tags HTTP header
     - name: x_telemetry_agent
       type: STRING
       mode: NULLABLE
-      description: X-Telemetry-Agent HTTP header
     - name: x_foxsec_ip_reputation
       type: STRING
       mode: NULLABLE
-      description: X-Foxsec-IP-Reputation header
     - name: x_lb_tags
       type: STRING
       mode: NULLABLE
-      description: X-LB-Tags HTTP header
     - name: parsed_date
       type: TIMESTAMP
       mode: NULLABLE
@@ -248,7 +196,6 @@
       - name: tls_cipher_hex
         type: STRING
         mode: NULLABLE
-    description: Headers included in the client's HTTP request
   - name: user_agent
     type: RECORD
     mode: NULLABLE
@@ -262,7 +209,6 @@
     - name: version
       type: STRING
       mode: NULLABLE
-    description: Parsed components of the client's user agent string
   - name: isp
     type: RECORD
     mode: NULLABLE
@@ -270,17 +216,12 @@
     - name: db_version
       type: STRING
       mode: NULLABLE
-      description: The specific geo ISP database version used for this lookup
     - name: name
       type: STRING
       mode: NULLABLE
-      description: The name of the ISP associated with the client's IP address
     - name: organization
       type: STRING
       mode: NULLABLE
-      description: The name of a specific business entity associated with the client's
-        IP address when available; otherwise the ISP name
-    description: Results of ISP lookup based on the client's IP address
 - name: metrics
   type: RECORD
   mode: NULLABLE
@@ -292,87 +233,45 @@
     - name: migration_addons_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
     - name: migration_bookmarks_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
     - name: migration_fxa_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
     - name: migration_fxa_has_custom_idp_server
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Is custom idp server configured?
-
-        '
     - name: migration_fxa_has_custom_token_server
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Is custom token server configured?
-
-        '
     - name: migration_gecko_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
     - name: migration_history_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
     - name: migration_logins_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
     - name: migration_open_tabs_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
     - name: migration_settings_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
     - name: migration_settings_telemetry_enabled
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Is telemetry enabled after this migration?
-
-        '
     - name: migration_telemetry_identifiers_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
     - name: migration_search_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
     - name: migration_pinned_sites_any_failures
       type: BOOLEAN
       mode: NULLABLE
-      description: 'Did this migration encounter any failures (exceptions)?
-
-        '
   - name: counter
     type: RECORD
     mode: NULLABLE
@@ -380,196 +279,99 @@
     - name: migration_addons_failed_addons
       type: INTEGER
       mode: NULLABLE
-      description: 'How many addons failed to migrate?
-
-        '
     - name: migration_addons_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_addons_migrated_addons
       type: INTEGER
       mode: NULLABLE
-      description: 'How many addons were migrated?
-
-        '
     - name: migration_addons_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_bookmarks_detected
       type: INTEGER
       mode: NULLABLE
-      description: 'Total number of detected bookmarks prior to a migration.
-
-        '
     - name: migration_fxa_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_fxa_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_history_detected
       type: INTEGER
       mode: NULLABLE
-      description: 'Total number of detected history items prior to a migration.
-
-        '
     - name: migration_logins_detected
       type: INTEGER
       mode: NULLABLE
-      description: 'How many logins were detected?
-
-        '
     - name: migration_logins_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_logins_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_logins_unsupported_db_version
       type: INTEGER
       mode: NULLABLE
-      description: 'If we couldn''t migrate due to an unsupported db version, what
-        was it?
-
-        '
     - name: migration_open_tabs_detected
       type: INTEGER
       mode: NULLABLE
-      description: 'Number of detected open tabs.
-
-        '
     - name: migration_open_tabs_migrated
       type: INTEGER
       mode: NULLABLE
-      description: 'Number of migrated open tabs.
-
-        '
     - name: migration_settings_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_settings_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_bookmarks_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_bookmarks_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_gecko_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_gecko_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_history_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_history_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_open_tabs_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_open_tabs_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_telemetry_identifiers_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_telemetry_identifiers_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_search_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_search_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_pinned_sites_detected_pinned_sites
       type: INTEGER
       mode: NULLABLE
-      description: 'How many pinned sites were detected?
-
-        '
     - name: migration_pinned_sites_failure_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration failed. See codes in TelemetryHelpers.kt
-
-        '
     - name: migration_pinned_sites_migrated_pinned_sites
       type: INTEGER
       mode: NULLABLE
-      description: 'How many pinned sites were migrated?
-
-        '
     - name: migration_pinned_sites_success_reason
       type: INTEGER
       mode: NULLABLE
-      description: 'Why this migration succeeded. See codes in TelemetryHelpers.kt
-
-        '
   - name: datetime
     type: RECORD
     mode: NULLABLE
@@ -577,9 +379,6 @@
     - name: migration_telemetry_identifiers_fennec_profile_creation_date
       type: STRING
       mode: NULLABLE
-      description: 'Records when the migrated Fennec profile was created.
-
-        '
     - name: raw_migration_telemetry_identifiers_fennec_profile_creation_date
       type: STRING
       mode: NULLABLE
@@ -712,41 +511,21 @@
     - name: migration_fxa_bad_auth_state
       type: STRING
       mode: NULLABLE
-      description: 'If we couldn''t migrate due to a bad auth state, what was it?
-
-        '
     - name: migration_fxa_failure_reason_rust
       type: STRING
       mode: NULLABLE
-      description: 'Why this migration failed (rust exception).
-
-        '
     - name: migration_fxa_unsupported_account_version
       type: STRING
       mode: NULLABLE
-      description: 'What is the detected unsupported account version?
-
-        '
     - name: migration_fxa_unsupported_pickle_version
       type: STRING
       mode: NULLABLE
-      description: 'What is the detected unsupported pickle version?
-
-        '
     - name: migration_fxa_unsupported_state_version
       type: STRING
       mode: NULLABLE
-      description: 'What is the detected unsupported state version?
-
-        '
     - name: glean_client_annotation_experimentation_id
       type: STRING
       mode: NULLABLE
-      description: 'An experimentation identifier derived and provided by the application
-
-        for the purpose of experimentation enrollment.
-
-        '
   - name: timespan
     type: RECORD
     mode: NULLABLE
@@ -761,9 +540,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Duration of the bookmarks migration (Rust operations).
-
-        '
     - name: migration_history_duration
       type: RECORD
       mode: NULLABLE
@@ -774,9 +550,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Duration of the history migration (Rust operations).
-
-        '
     - name: migration_addons_total_duration
       type: RECORD
       mode: NULLABLE
@@ -787,9 +560,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
     - name: migration_bookmarks_total_duration
       type: RECORD
       mode: NULLABLE
@@ -800,9 +570,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
     - name: migration_fxa_total_duration
       type: RECORD
       mode: NULLABLE
@@ -813,9 +580,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
     - name: migration_gecko_total_duration
       type: RECORD
       mode: NULLABLE
@@ -826,9 +590,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
     - name: migration_history_total_duration
       type: RECORD
       mode: NULLABLE
@@ -839,9 +600,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
     - name: migration_logins_total_duration
       type: RECORD
       mode: NULLABLE
@@ -852,9 +610,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
     - name: migration_open_tabs_total_duration
       type: RECORD
       mode: NULLABLE
@@ -865,9 +620,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
     - name: migration_pinned_sites_total_duration
       type: RECORD
       mode: NULLABLE
@@ -878,9 +630,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
     - name: migration_search_total_duration
       type: RECORD
       mode: NULLABLE
@@ -891,9 +640,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
     - name: migration_settings_total_duration
       type: RECORD
       mode: NULLABLE
@@ -904,9 +650,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
     - name: migration_telemetry_identifiers_total_duration
       type: RECORD
       mode: NULLABLE
@@ -917,9 +660,6 @@
       - name: value
         type: INTEGER
         mode: NULLABLE
-      description: 'Total duration of this migration.
-
-        '
   - name: uuid
     type: RECORD
     mode: NULLABLE
@@ -927,9 +667,6 @@
     - name: migration_telemetry_identifiers_fennec_client_id
       type: STRING
       mode: NULLABLE
-      description: 'Records clientID of the migrated Fennec profile.
-
-        '
   - name: jwe
     type: RECORD
     mode: REPEATED
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/migration/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/migration/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/migration/view.sql	2025-10-28 18:49:25.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/migration/view.sql	2025-10-28 18:53:24.000000000 +0000
@@ -41,13 +41,7 @@
   document_id,
   events,
   STRUCT(
-    STRUCT(
-      metadata.geo.city,
-      metadata.geo.country,
-      metadata.geo.db_version,
-      metadata.geo.subdivision1,
-      metadata.geo.subdivision2
-    ) AS `geo`,
+    metadata.geo,
     STRUCT(
       metadata.header.date,
       metadata.header.dnt,
@@ -62,7 +56,7 @@
       CAST(NULL AS STRUCT<`tls_version` STRING, `tls_cipher_hex` STRING>) AS `parsed_x_lb_tags`
     ) AS `header`,
     metadata.user_agent,
-    STRUCT(metadata.isp.db_version, metadata.isp.name, metadata.isp.organization) AS `isp`
+    metadata.isp
   ) AS `metadata`,
   STRUCT(
     STRUCT(
@@ -195,10 +189,79 @@
   ).channel AS normalized_channel,
   CAST(NULL AS DATE) AS `submission_date`,
   additional_properties,
-  client_info,
+  STRUCT(
+    client_info.android_sdk_version,
+    client_info.app_build,
+    client_info.app_channel,
+    client_info.app_display_version,
+    client_info.architecture,
+    client_info.client_id,
+    client_info.device_manufacturer,
+    client_info.device_model,
+    client_info.first_run_date,
+    client_info.locale,
+    client_info.os,
+    client_info.os_version,
+    client_info.telemetry_sdk_build,
+    client_info.build_date,
+    client_info.windows_build_number,
+    client_info.session_count,
+    client_info.session_id,
+    STRUCT(
+      client_info.attribution.campaign,
+      client_info.attribution.content,
+      client_info.attribution.medium,
+      client_info.attribution.source,
+      client_info.attribution.term,
+      client_info.attribution.ext
+    ) AS `attribution`,
+    STRUCT(client_info.distribution.name, client_info.distribution.ext) AS `distribution`
+  ) AS `client_info`,
   document_id,
-  events,
-  metadata,
+  ARRAY(
+    SELECT
+      STRUCT(
+        events.category,
+        ARRAY(
+          SELECT
+            STRUCT(extra.key, extra.value)
+          FROM
+            UNNEST(events.extra) AS `extra`
+        ) AS `extra`,
+        events.name,
+        events.timestamp
+      )
+    FROM
+      UNNEST(events) AS `events`
+  ) AS `events`,
+  STRUCT(
+    STRUCT(
+      metadata.geo.city,
+      metadata.geo.country,
+      metadata.geo.db_version,
+      metadata.geo.subdivision1,
+      metadata.geo.subdivision2
+    ) AS `geo`,
+    STRUCT(
+      metadata.header.date,
+      metadata.header.dnt,
+      metadata.header.x_debug_id,
+      metadata.header.x_pingsender_version,
+      metadata.header.x_source_tags,
+      metadata.header.x_telemetry_agent,
+      metadata.header.x_foxsec_ip_reputation,
+      metadata.header.x_lb_tags,
+      metadata.header.parsed_date,
+      metadata.header.parsed_x_source_tags,
+      metadata.header.parsed_x_lb_tags
+    ) AS `header`,
+    STRUCT(
+      metadata.user_agent.browser,
+      metadata.user_agent.os,
+      metadata.user_agent.version
+    ) AS `user_agent`,
+    STRUCT(metadata.isp.db_version, metadata.isp.name, metadata.isp.organization) AS `isp`
+  ) AS `metadata`,
   STRUCT(
     STRUCT(
       metrics.boolean.migration_addons_any_failures,
@@ -255,15 +318,65 @@
       metrics.datetime.raw_migration_telemetry_identifiers_fennec_profile_creation_date
     ) AS `datetime`,
     STRUCT(
-      metrics.labeled_counter.glean_error_invalid_label,
-      metrics.labeled_counter.glean_error_invalid_overflow,
-      metrics.labeled_counter.glean_error_invalid_state,
-      metrics.labeled_counter.glean_error_invalid_value,
-      metrics.labeled_counter.migration_bookmarks_migrated,
-      metrics.labeled_counter.migration_history_migrated,
+      ARRAY(
+        SELECT
+          STRUCT(glean_error_invalid_label.key, glean_error_invalid_label.value)
+        FROM
+          UNNEST(metrics.labeled_counter.glean_error_invalid_label) AS `glean_error_invalid_label`
+      ) AS `glean_error_invalid_label`,
+      ARRAY(
+        SELECT
+          STRUCT(glean_error_invalid_overflow.key, glean_error_invalid_overflow.value)
+        FROM
+          UNNEST(
+            metrics.labeled_counter.glean_error_invalid_overflow
+          ) AS `glean_error_invalid_overflow`
+      ) AS `glean_error_invalid_overflow`,
+      ARRAY(
+        SELECT
+          STRUCT(glean_error_invalid_state.key, glean_error_invalid_state.value)
+        FROM
+          UNNEST(metrics.labeled_counter.glean_error_invalid_state) AS `glean_error_invalid_state`
+      ) AS `glean_error_invalid_state`,
+      ARRAY(
+        SELECT
+          STRUCT(glean_error_invalid_value.key, glean_error_invalid_value.value)
+        FROM
+          UNNEST(metrics.labeled_counter.glean_error_invalid_value) AS `glean_error_invalid_value`
+      ) AS `glean_error_invalid_value`,
+      ARRAY(
+        SELECT
+          STRUCT(migration_bookmarks_migrated.key, migration_bookmarks_migrated.value)
+        FROM
+          UNNEST(
+            metrics.labeled_counter.migration_bookmarks_migrated
+          ) AS `migration_bookmarks_migrated`
+      ) AS `migration_bookmarks_migrated`,
+      ARRAY(
+        SELECT
+          STRUCT(migration_history_migrated.key, migration_history_migrated.value)
+        FROM
+          UNNEST(metrics.labeled_counter.migration_history_migrated) AS `migration_history_migrated`
+      ) AS `migration_history_migrated`,
+      ARRAY(
+        SELECT
+          STRUCT(migration_logins_failure_counts.key, migration_logins_failure_counts.value)
+        FROM
+          UNNEST(
       metrics.labeled_counter.migration_logins_failure_counts
+          ) AS `migration_logins_failure_counts`
+      ) AS `migration_logins_failure_counts`
     ) AS `labeled_counter`,
-    STRUCT(metrics.labeled_string.migration_migration_versions) AS `labeled_string`,
+    STRUCT(
+      ARRAY(
+        SELECT
+          STRUCT(migration_migration_versions.key, migration_migration_versions.value)
+        FROM
+          UNNEST(
+            metrics.labeled_string.migration_migration_versions
+          ) AS `migration_migration_versions`
+      ) AS `migration_migration_versions`
+    ) AS `labeled_string`,
     STRUCT(
       metrics.string.migration_fxa_bad_auth_state,
       metrics.string.migration_fxa_failure_reason_rust,
@@ -273,19 +386,58 @@
       metrics.string.glean_client_annotation_experimentation_id
     ) AS `string`,
     STRUCT(
-      metrics.timespan.migration_bookmarks_duration,
-      metrics.timespan.migration_history_duration,
-      metrics.timespan.migration_addons_total_duration,
-      metrics.timespan.migration_bookmarks_total_duration,
-      metrics.timespan.migration_fxa_total_duration,
-      metrics.timespan.migration_gecko_total_duration,
-      metrics.timespan.migration_history_total_duration,
-      metrics.timespan.migration_logins_total_duration,
-      metrics.timespan.migration_open_tabs_total_duration,
-      metrics.timespan.migration_pinned_sites_total_duration,
-      metrics.timespan.migration_search_total_duration,
-      metrics.timespan.migration_settings_total_duration,
-      metrics.timespan.migration_telemetry_identifiers_total_duration
+      STRUCT(
+        metrics.timespan.migration_bookmarks_duration.time_unit,
+        metrics.timespan.migration_bookmarks_duration.value
+      ) AS `migration_bookmarks_duration`,
+      STRUCT(
+        metrics.timespan.migration_history_duration.time_unit,
+        metrics.timespan.migration_history_duration.value
+      ) AS `migration_history_duration`,
+      STRUCT(
+        metrics.timespan.migration_addons_total_duration.time_unit,
+        metrics.timespan.migration_addons_total_duration.value
+      ) AS `migration_addons_total_duration`,
+      STRUCT(
+        metrics.timespan.migration_bookmarks_total_duration.time_unit,
+        metrics.timespan.migration_bookmarks_total_duration.value
+      ) AS `migration_bookmarks_total_duration`,
+      STRUCT(
+        metrics.timespan.migration_fxa_total_duration.time_unit,
+        metrics.timespan.migration_fxa_total_duration.value
+      ) AS `migration_fxa_total_duration`,
+      STRUCT(
+        metrics.timespan.migration_gecko_total_duration.time_unit,
+        metrics.timespan.migration_gecko_total_duration.value
+      ) AS `migration_gecko_total_duration`,
+      STRUCT(
+        metrics.timespan.migration_history_total_duration.time_unit,
+        metrics.timespan.migration_history_total_duration.value
+      ) AS `migration_history_total_duration`,
+      STRUCT(
+        metrics.timespan.migration_logins_total_duration.time_unit,
+        metrics.timespan.migration_logins_total_duration.value
+      ) AS `migration_logins_total_duration`,
+      STRUCT(
+        metrics.timespan.migration_open_tabs_total_duration.time_unit,
+        metrics.timespan.migration_open_tabs_total_duration.value
+      ) AS `migration_open_tabs_total_duration`,
+      STRUCT(
+        metrics.timespan.migration_pinned_sites_total_duration.time_unit,
+        metrics.timespan.migration_pinned_sites_total_duration.value
+      ) AS `migration_pinned_sites_total_duration`,
+      STRUCT(
+        metrics.timespan.migration_search_total_duration.time_unit,
+        metrics.timespan.migration_search_total_duration.value
+      ) AS `migration_search_total_duration`,
+      STRUCT(
+        metrics.timespan.migration_settings_total_duration.time_unit,
+        metrics.timespan.migration_settings_total_duration.value
+      ) AS `migration_settings_total_duration`,
+      STRUCT(
+        metrics.timespan.migration_telemetry_identifiers_total_duration.time_unit,
+        metrics.timespan.migration_telemetry_identifiers_total_duration.value
+      ) AS `migration_telemetry_identifiers_total_duration`
     ) AS `timespan`,
     STRUCT(metrics.uuid.migration_telemetry_identifiers_fennec_client_id) AS `uuid`,
     CAST(NULL AS ARRAY<STRUCT<`key` STRING, `value` STRING>>) AS `jwe`,
@@ -309,7 +461,27 @@
   normalized_country_code,
   normalized_os,
   normalized_os_version,
-  ping_info,
+  STRUCT(
+    ping_info.end_time,
+    ARRAY(
+      SELECT
+        STRUCT(
+          experiments.key,
+          STRUCT(
+            experiments.value.branch,
+            STRUCT(experiments.value.extra.type, experiments.value.extra.enrollment_id) AS `extra`
+          ) AS `value`
+        )
+      FROM
+        UNNEST(ping_info.experiments) AS `experiments`
+    ) AS `experiments`,
+    ping_info.ping_type,
+    ping_info.reason,
+    ping_info.seq,
+    ping_info.start_time,
+    ping_info.parsed_start_time,
+    ping_info.parsed_end_time
+  ) AS `ping_info`,
   sample_id,
   submission_timestamp,
   app_version_major,
@@ -327,10 +499,79 @@
   ).channel AS normalized_channel,
   CAST(NULL AS DATE) AS `submission_date`,
   additional_properties,
-  client_info,
+  STRUCT(
+    client_info.android_sdk_version,
+    client_info.app_build,
+    client_info.app_channel,
+    client_info.app_display_version,
+    client_info.architecture,
+    client_info.client_id,
+    client_info.device_manufacturer,
+    client_info.device_model,
+    client_info.first_run_date,
+    client_info.locale,
+    client_info.os,
+    client_info.os_version,
+    client_info.telemetry_sdk_build,
+    client_info.build_date,
+    client_info.windows_build_number,
+    client_info.session_count,
+    client_info.session_id,
+    STRUCT(
+      client_info.attribution.campaign,
+      client_info.attribution.content,
+      client_info.attribution.medium,
+      client_info.attribution.source,
+      client_info.attribution.term,
+      client_info.attribution.ext
+    ) AS `attribution`,
+    STRUCT(client_info.distribution.name, client_info.distribution.ext) AS `distribution`
+  ) AS `client_info`,
   document_id,
-  events,
-  metadata,
+  ARRAY(
+    SELECT
+      STRUCT(
+        events.category,
+        ARRAY(
+          SELECT
+            STRUCT(extra.key, extra.value)
+          FROM
+            UNNEST(events.extra) AS `extra`
+        ) AS `extra`,
+        events.name,
+        events.timestamp
+      )
+    FROM
+      UNNEST(events) AS `events`
+  ) AS `events`,
+  STRUCT(
+    STRUCT(
+      metadata.geo.city,
+      metadata.geo.country,
+      metadata.geo.db_version,
+      metadata.geo.subdivision1,
+      metadata.geo.subdivision2
+    ) AS `geo`,
+    STRUCT(
+      metadata.header.date,
+      metadata.header.dnt,
+      metadata.header.x_debug_id,
+      metadata.header.x_pingsender_version,
+      metadata.header.x_source_tags,
+      metadata.header.x_telemetry_agent,
+      metadata.header.x_foxsec_ip_reputation,
+      metadata.header.x_lb_tags,
+      metadata.header.parsed_date,
+      metadata.header.parsed_x_source_tags,
+      metadata.header.parsed_x_lb_tags
+    ) AS `header`,
+    STRUCT(
+      metadata.user_agent.browser,
+      metadata.user_agent.os,
+      metadata.user_agent.version
+    ) AS `user_agent`,
+    STRUCT(metadata.isp.db_version, metadata.isp.name, metadata.isp.organization) AS `isp`
+  ) AS `metadata`,
   STRUCT(
     STRUCT(
       metrics.boolean.migration_addons_any_failures,
@@ -387,15 +628,65 @@
       metrics.datetime.raw_migration_telemetry_identifiers_fennec_profile_creation_date
     ) AS `datetime`,
     STRUCT(
-      metrics.labeled_counter.glean_error_invalid_label,
-      metrics.labeled_counter.glean_error_invalid_overflow,
-      metrics.labeled_counter.glean_error_invalid_state,
-      metrics.labeled_counter.glean_error_invalid_value,
-      metrics.labeled_counter.migration_bookmarks_migrated,
-      metrics.labeled_counter.migration_history_migrated,
+      ARRAY(
+        SELECT
+          STRUCT(glean_error_invalid_label.key, glean_error_invalid_label.value)
+        FROM
+          UNNEST(metrics.labeled_counter.glean_error_invalid_label) AS `glean_error_invalid_label`
+      ) AS `glean_error_invalid_label`,
+      ARRAY(
+        SELECT
+          STRUCT(glean_error_invalid_overflow.key, glean_error_invalid_overflow.value)
+        FROM
+          UNNEST(
+            metrics.labeled_counter.glean_error_invalid_overflow
+          ) AS `glean_error_invalid_overflow`
+      ) AS `glean_error_invalid_overflow`,
+      ARRAY(
+        SELECT
+          STRUCT(glean_error_invalid_state.key, glean_error_invalid_state.value)
+        FROM
+          UNNEST(metrics.labeled_counter.glean_error_invalid_state) AS `glean_error_invalid_state`
+      ) AS `glean_error_invalid_state`,
+      ARRAY(
+        SELECT
+          STRUCT(glean_error_invalid_value.key, glean_error_invalid_value.value)
+        FROM
+          UNNEST(metrics.labeled_counter.glean_error_invalid_value) AS `glean_error_invalid_value`
+      ) AS `glean_error_invalid_value`,
+      ARRAY(
+        SELECT
+          STRUCT(migration_bookmarks_migrated.key, migration_bookmarks_migrated.value)
+        FROM
+          UNNEST(
+            metrics.labeled_counter.migration_bookmarks_migrated
+          ) AS `migration_bookmarks_migrated`
+      ) AS `migration_bookmarks_migrated`,
+      ARRAY(
+        SELECT
+          STRUCT(m

⚠️ Only part of the diff is displayed.

Link to full diff

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants