DWBQ-SDQAGOGoogleGO-DWBQ-SDQA1.0<ul> <li>Describe BigQuery architecture fundamentals.</li><li>Implement storage and schema design patterns to improve performance.</li><li>Use DML and schedule data transfers to ingest data.</li><li>Apply best practices to improve read efficiency and optimize query performance.</li><li>Manage capacity and automate workloads.</li><li>Understand patterns versus anti-patterns to optimize queries and improve read performance.</li><li>Use logging and monitoring tools to understand and optimize usage patterns.</li><li>Apply security best practices to govern data and resources.</li><li>Build and deploy several categories of machine learning models with BigQuery ML.</li></ul><p><span class="cms-link-marked"><a class="fl-href-prod" href="/swisscom/fr/course/google-dwbq-sdqa"><svg role="img" aria-hidden="true" focusable="false" data-nosnippet class="cms-linkmark"><use xlink:href="/css/img/icnset-linkmarks.svg#linkmark"></use></svg>Data Warehousing with BigQuery: Storage Design, Query Optimization, and Administration <span class="fl-prod-pcode">(DWBQ-SDQA)</span></a></span></p><p>Data analysts, data scientists, data engineers, and developers who perform work on a scale that requires advanced BigQuery internals knowledge to optimize performance.</p><h5>Module 1 - BigQuery Architecture Fundamentals</h5><h6>Topics</h6><ul> <li>Introduction</li><li>BigQuery Core Infrastructure</li><li>BigQuery Storage</li><li>BigQuery Query Processing</li><li>BigQuery Data Shuffling</li></ul><h6>Objectives</h6><ul> <li>Explain the benefits of columnar storage.</li><li>Understand how BigQuery processes data.</li><li>Explore the basics of BigQuery&rsquo;s shuffling service to improve query efficiency.</li></ul><h6>Activities</h6><ul> <li>Labs and demos</li></ul><h5>Module 2 - Storage and Schema Optimizations</h5><h6>Topics</h6><ul> <li>BigQuery Storage</li><li>Partitioning and Clustering</li><li>Nested and Repeated Fields</li><li>ARRAY and STRUCT syntax</li><li>Best Practices</li></ul><h6>Objectives</h6><ul> <li>Compare the performance of different schemas (snowflake, denormalized, and nested and repeated fields).</li><li>Partition and cluster data for better performance</li><li>Improve schema design using nested and repeated fields.</li><li>Describe additional best practices such as table and partition expiration</li></ul><h6>Activities</h6><ul> <li>Labs and demos</li></ul><h5>Module 3 - Ingesting Data</h5><h6>Topics</h6><ul> <li>Data Ingestion Options</li><li>Batch Ingestion</li><li>Streaming Ingestion</li><li>Legacy Streaming API</li><li>BigQuery Storage Write API</li><li>Query Materialization</li><li>Query External Data Sources</li><li>Data Transfer Service</li></ul><h6>Objectives</h6><ul> <li>Ingest batch and streaming data.</li><li>Query external data sources.</li><li>Schedule data transfers.</li><li>Understand how to use Storage Write API.</li></ul><h6>Activities</h6><ul> <li>Labs and demos</li></ul><h5>Module 4 - Changing Data</h5><h6>Topics</h6><ul> <li>Managing Change in Data Warehouses</li><li>Handling Slowly Changing Dimensions (SCD)</li><li>DML statements</li><li>DML Best Practices and Common Issues</li></ul><h6>Objectives</h6><ul> <li>Write DML statements.</li><li>Address common DML performance problems and bottlenecks.</li><li>Identify slowly changing dimensions (SCD) in your data and make updates.</li></ul><h5>Module 5 - Improving Read Performance</h5><h6>Topics</h6><ul> <li>BigQuery&rsquo;s Cache</li><li>Materialized Views</li><li>BI Engine</li><li>High Throughput Reads</li><li>BigQuery Storage Read API</li></ul><h6>Objectives</h6><ul> <li>Explore BigQuery&rsquo;s cache.</li><li>Create materialized views.</li><li>Work with BI Engine to accelerate your SQL queries.</li><li>Use the Storage Read API for fast access to BigQuery-managed storage.</li><li>Explain the caveats of using external data sources.</li></ul><h6>Activities</h6><ul> <li>Labs and demos</li></ul><h5>Module 6 - Optimizing and Troubleshooting Queries</h5><h6>Topics</h6><ul> <li>Simple Query Execution</li><li>SELECTs and Aggregation</li><li>JOINs and Skewed JOINs</li><li>Filtering and Ordering</li><li>Best Practices for Functions</li></ul><h6>Objectives</h6><ul> <li>Interpret BigQuery execution details and the query plan.</li><li>Optimize query performance by using suggested methods for SQL statements and clauses.</li><li>Demonstrate best practices for functions in business use cases.</li></ul><h6>Activities</h6><ul> <li>Labs and demos</li></ul><h5>Module 7 - Workload Management and Pricing</h5><h6>Topics</h6><ul> <li>BigQuery Slots</li><li>Pricing Models and Estimates</li><li>Slot Reservations</li><li>Controlling Costs</li></ul><h6>Objectives</h6><ul> <li>Define a BigQuery slot.</li><li>Explain pricing models and pricing estimations (BigQuery UI, bq dry_run, jobs API).</li><li>Understand slot reservations, commitments, and assignments.</li><li>Identify best practices to control costs.</li></ul><h6>Activities</h6><ul> <li>Demos</li></ul><h5>Module 8 - Logging and Monitoring</h5><h6>Topics</h6><ul> <li>Cloud Monitoring</li><li>BigQuery Admin Panel</li><li>Cloud Audit Logs</li><li>INFORMATION_SCHEMA</li><li>Query Path and Common Errors</li></ul><h6>Objectives</h6><ul> <li>Use Cloud Monitoring to view BigQuery metrics.</li><li>Explore the BigQuery admin panel.</li><li>Use Cloud Audit logs.</li><li>Work with INFORMATION_SCHEMA tables to get insights for your BigQuery entities.</li></ul><h6>Activities</h6><ul> <li>Labs and demos</li></ul><h5>Module 9 - Security in BigQuery</h5><h6>Topics</h6><ul> <li>Secure Resources with IAM</li><li>Authorized Views</li><li>Secure Data with Classification</li><li>Encryption</li><li>Data Discovery and Governance</li></ul><h6>Objectives</h6><ul> <li>Explore data discovery using Data Catalog.</li><li>Discuss data governance using DLP API and Data Catalog.</li><li>Create IAM policies (e.g., authorized views) to secure resources.</li><li>Secure data with classifications (e.g., row-level policies).</li><li>Understand how BigQuery uses encryption.</li></ul><h6>Activities</h6><ul> <li>Labs and demos</li></ul><h5>Module 10 - Automating Workloads</h5><h6>Topics</h6><ul> <li>Scheduling Queries</li><li>Scripting</li><li>Stored Procedures</li><li>Integration with Big Data Products</li></ul><h6>Objectives</h6><ul> <li>Schedule queries.</li><li>Use scripting and stored procedures to build custom transformations.</li><li>Describe how to integrate BigQuery workloads with other Google Cloud big data products.</li></ul><h6>Activities</h6><ul> <li>Demos</li></ul><h5>Module 11 - Machine Learning in BigQuery</h5><h6>Topics</h6><ul> <li>Introduction to BigQuery ML</li><li>How to Make Predictions with BigQuery ML</li><li>How to Build and Deploy a Recommendation System with BigQuery ML</li><li>How to Build and Deploy a Demand Forecasting Solution with BigQuery ML</li><li>Time-Series Models with BigQuery ML</li><li>BigQuery ML Explainability</li></ul><h6>Objectives</h6><ul> <li>Describe some of the different applications of BigQuery ML.</li><li>Build and deploy several categories of machine learning models with BigQuery ML.</li><li>Use AutoML Tables to solve high-value business problems.</li></ul><h6>Activities</h6><ul> <li>Labs and demos</li></ul>- Describe BigQuery architecture fundamentals. - Implement storage and schema design patterns to improve performance. - Use DML and schedule data transfers to ingest data. - Apply best practices to improve read efficiency and optimize query performance. - Manage capacity and automate workloads. - Understand patterns versus anti-patterns to optimize queries and improve read performance. - Use logging and monitoring tools to understand and optimize usage patterns. - Apply security best practices to govern data and resources. - Build and deploy several categories of machine learning models with BigQuery ML.Data Warehousing with BigQuery: Storage Design, Query Optimization, and Administration (DWBQ-SDQA)Data analysts, data scientists, data engineers, and developers who perform work on a scale that requires advanced BigQuery internals knowledge to optimize performance.Module 1 - BigQuery Architecture Fundamentals Topics - Introduction - BigQuery Core Infrastructure - BigQuery Storage - BigQuery Query Processing - BigQuery Data Shuffling Objectives - Explain the benefits of columnar storage. - Understand how BigQuery processes data. - Explore the basics of BigQuery’s shuffling service to improve query efficiency. Activities - Labs and demos Module 2 - Storage and Schema Optimizations Topics - BigQuery Storage - Partitioning and Clustering - Nested and Repeated Fields - ARRAY and STRUCT syntax - Best Practices Objectives - Compare the performance of different schemas (snowflake, denormalized, and nested and repeated fields). - Partition and cluster data for better performance - Improve schema design using nested and repeated fields. - Describe additional best practices such as table and partition expiration Activities - Labs and demos Module 3 - Ingesting Data Topics - Data Ingestion Options - Batch Ingestion - Streaming Ingestion - Legacy Streaming API - BigQuery Storage Write API - Query Materialization - Query External Data Sources - Data Transfer Service Objectives - Ingest batch and streaming data. - Query external data sources. - Schedule data transfers. - Understand how to use Storage Write API. Activities - Labs and demos Module 4 - Changing Data Topics - Managing Change in Data Warehouses - Handling Slowly Changing Dimensions (SCD) - DML statements - DML Best Practices and Common Issues Objectives - Write DML statements. - Address common DML performance problems and bottlenecks. - Identify slowly changing dimensions (SCD) in your data and make updates. Module 5 - Improving Read Performance Topics - BigQuery’s Cache - Materialized Views - BI Engine - High Throughput Reads - BigQuery Storage Read API Objectives - Explore BigQuery’s cache. - Create materialized views. - Work with BI Engine to accelerate your SQL queries. - Use the Storage Read API for fast access to BigQuery-managed storage. - Explain the caveats of using external data sources. Activities - Labs and demos Module 6 - Optimizing and Troubleshooting Queries Topics - Simple Query Execution - SELECTs and Aggregation - JOINs and Skewed JOINs - Filtering and Ordering - Best Practices for Functions Objectives - Interpret BigQuery execution details and the query plan. - Optimize query performance by using suggested methods for SQL statements and clauses. - Demonstrate best practices for functions in business use cases. Activities - Labs and demos Module 7 - Workload Management and Pricing Topics - BigQuery Slots - Pricing Models and Estimates - Slot Reservations - Controlling Costs Objectives - Define a BigQuery slot. - Explain pricing models and pricing estimations (BigQuery UI, bq dry_run, jobs API). - Understand slot reservations, commitments, and assignments. - Identify best practices to control costs. Activities - Demos Module 8 - Logging and Monitoring Topics - Cloud Monitoring - BigQuery Admin Panel - Cloud Audit Logs - INFORMATION_SCHEMA - Query Path and Common Errors Objectives - Use Cloud Monitoring to view BigQuery metrics. - Explore the BigQuery admin panel. - Use Cloud Audit logs. - Work with INFORMATION_SCHEMA tables to get insights for your BigQuery entities. Activities - Labs and demos Module 9 - Security in BigQuery Topics - Secure Resources with IAM - Authorized Views - Secure Data with Classification - Encryption - Data Discovery and Governance Objectives - Explore data discovery using Data Catalog. - Discuss data governance using DLP API and Data Catalog. - Create IAM policies (e.g., authorized views) to secure resources. - Secure data with classifications (e.g., row-level policies). - Understand how BigQuery uses encryption. Activities - Labs and demos Module 10 - Automating Workloads Topics - Scheduling Queries - Scripting - Stored Procedures - Integration with Big Data Products Objectives - Schedule queries. - Use scripting and stored procedures to build custom transformations. - Describe how to integrate BigQuery workloads with other Google Cloud big data products. Activities - Demos Module 11 - Machine Learning in BigQuery Topics - Introduction to BigQuery ML - How to Make Predictions with BigQuery ML - How to Build and Deploy a Recommendation System with BigQuery ML - How to Build and Deploy a Demand Forecasting Solution with BigQuery ML - Time-Series Models with BigQuery ML - BigQuery ML Explainability Objectives - Describe some of the different applications of BigQuery ML. - Build and deploy several categories of machine learning models with BigQuery ML. - Use AutoML Tables to solve high-value business problems. Activities - Labs and demos3 jours1950.001950.001995.002095.002095.002220.001950.001950.001980.006770.001950.002755.002450.00