Orchestrate BigQuery Workloads with Dataform

Orchestrate BigQuery Workloads with DataformOBWDGOGoogleGO-OBWD1.0<ul> <li>Understand the components of Dataform core.</li><li>Create tables and views in BigQuery using Dataform.</li><li>Document BigQuery tables and views.</li><li>Understand BigQuery security settings using Dataform.</li><li>Use assertions to validate data in Dataform workflows.</li><li>Execute Dataform SQL workflows in an automated fashion.</li></ul>Knowledge of SQL data analysis and BigQuery as discussed in BigQuery for Data Analysis.<h4>Module 1 - Dataform Core Components</h4> Topics: <ul> <li>SQL workflow</li><li>Repositories and workspaces</li><li>Default files and folders</li><li>Compiled graphs</li></ul>Objectives: <ul> <li>Understand the components of Dataflow core.</li></ul><h4>Module 2 - Table Definitions and Dependencies</h4> Topics: <ul> <li>Declare a data source.</li><li>Create a table.</li><li>Create an incremental table.</li><li>Set partitioning and clustering options.</li><li>Create an empty table.</li><li>Create an external BigLake table.</li><li>Create views and materialized views.</li><li>Define dependencies.</li></ul>Objectives: <ul> <li>Create tables and views in BigQuery using Dataform</li></ul><h4>Module 3 - Document BigQuery Tables and Views</h4> Topics: <ul> <li>Use column descriptions.</li><li>Use globally defined JavaScript constants.</li><li>Add labels.</li></ul>Objectives: <ul> <li>Document BigQuery tables and views.</li></ul>Activities: <ul> <li>Lab: Build SQL Workflows with Dependencies in Dataform</li></ul><h4>Module 4 - BigQuery Security Settings</h4> Topics: <ul> <li>IAM dataset and table/view access</li><li>Column-level security</li><li>Row-level security</li></ul>Objectives: <ul> <li>Understand BigQuery security settings using Dataform</li></ul><h4>Module 5 - Assertions</h4> Topics: <ul> <li>Use built-in assertions.</li><li>Create manual assertions.</li></ul>Objectives: <ul> <li>Use assertions to validate data in Dataform workflows.</li></ul>Activities: <ul> <li>Lab: Work with Assertions and BigQuery Security Settings in Dataform.</li></ul><h4>Module 6 - SQL Workflow Executions</h4> Topics: <ul> <li>Dataform code lifecycle.</li><li>What happens during compilation.</li><li>Customize and schedule compilation results.</li><li>Execute workflows (UI, Cloud Scheduler, Cloud Composer).</li><li>Logging and monitoring.</li></ul>Objectives: <ul> <li>Execute Dataform SQL workflows in an automated fashion.</li></ul>Activities: <ul> <li>Lab: Automate and Monitor SQL Workflow Executions in Dataform</li></ul><h4>Module 7 - Advanced Use Cases</h4> Topics: <ul> <li>Create a BigLake table after file upload using Cloud Run functions.</li><li>Build a Machine Learning pipeline with BigQuery ML.</li><li>Work with Slowly Changing Dimensions Type 2.</li></ul>Objectives: <ul> <li>Explore additional use cases for Dataform.</li></ul>Activities: <ul> <li>Lab: Create a BigLake Table with Dataform Using Cloud Run Functions.</li></ul>- Understand the components of Dataform core. - Create tables and views in BigQuery using Dataform. - Document BigQuery tables and views. - Understand BigQuery security settings using Dataform. - Use assertions to validate data in Dataform workflows. - Execute Dataform SQL workflows in an automated fashion.Knowledge of SQL data analysis and BigQuery as discussed in BigQuery for Data Analysis.Module 1 - Dataform Core Components Topics: - SQL workflow - Repositories and workspaces - Default files and folders - Compiled graphs Objectives: - Understand the components of Dataflow core. Module 2 - Table Definitions and Dependencies Topics: - Declare a data source. - Create a table. - Create an incremental table. - Set partitioning and clustering options. - Create an empty table. - Create an external BigLake table. - Create views and materialized views. - Define dependencies. Objectives: - Create tables and views in BigQuery using Dataform Module 3 - Document BigQuery Tables and Views Topics: - Use column descriptions. - Use globally defined JavaScript constants. - Add labels. Objectives: - Document BigQuery tables and views. Activities: - Lab: Build SQL Workflows with Dependencies in Dataform Module 4 - BigQuery Security Settings Topics: - IAM dataset and table/view access - Column-level security - Row-level security Objectives: - Understand BigQuery security settings using Dataform Module 5 - Assertions Topics: - Use built-in assertions. - Create manual assertions. Objectives: - Use assertions to validate data in Dataform workflows. Activities: - Lab: Work with Assertions and BigQuery Security Settings in Dataform. Module 6 - SQL Workflow Executions Topics: - Dataform code lifecycle. - What happens during compilation. - Customize and schedule compilation results. - Execute workflows (UI, Cloud Scheduler, Cloud Composer). - Logging and monitoring. Objectives: - Execute Dataform SQL workflows in an automated fashion. Activities: - Lab: Automate and Monitor SQL Workflow Executions in Dataform Module 7 - Advanced Use Cases Topics: - Create a BigLake table after file upload using Cloud Run functions. - Build a Machine Learning pipeline with BigQuery ML. - Work with Slowly Changing Dimensions Type 2. Objectives: - Explore additional use cases for Dataform. Activities: - Lab: Create a BigLake Table with Dataform Using Cloud Run Functions.2 days1495.001300.001300.001300.001300.001320.002065.001550.001300.00