CTAKCRClouderaCR-CTAK1.0<p>During this course, you learn how to:</p> <ul> <li>Plan, deploy, and operate Kafka clusters</li><li>Create and manage topics</li><li>Develop producers and consumers</li><li>Use replication to improve fault tolerance</li><li>Use partitioning to improve scalability</li><li>Troubleshoot common problems and performance issues</li></ul><p>This course is designed for system administrators, data engineers, and developers. All students are expected to have basic Linux experience, and basic proficiency with the Java programming language is recommended. No prior experience with Apache Kafka is necessary.</p><ul> <li>Kafka Overview</li><li>Deploying Apache Kafka</li><li>Kafka Command Line Basics</li><li>Using Streams Messaging Manager (SMM)</li><li>Kafka Java API Basics</li><li>Improving Availability through Replication</li><li>Improving Application Scalability</li><li>Improving Application Reliability</li><li>Monitoring Kafka</li><li>Managing Kafka</li><li>Message Structure, Format, and Versioning</li><li>Improving Application Performance</li><li>Improving Kafka Service Performance</li><li>Securing the Kafka Cluster</li></ul><h5>Kafka Overview</h5><ul> <li>High-Level Architecture</li><li>Common Use Cases</li><li>Cloudera&#039;s Distribution of Apache Kafka</li></ul><h5>Deploying Apache Kafka</h5><ul> <li>System Requirements and Dependencies</li><li>Service Roles</li><li>Planning Your Deployment Deploying Kafka Services</li><li>Exercise: Preparing the Exercise Environment</li><li>Exercise: Installing the Kafka Service with Cloudera Manager</li><li>Exercise (optional): Create Metrics Dashboards</li><li>Exercise (optional): Using the CM API</li></ul><h5>Kafka Command Line Basics</h5><ul> <li>Create and Manage Topics</li><li>Running Producers and Consumers</li></ul><h5>Using Streams Messaging Manager (SMM)</h5><ul> <li>Streams Messaging Manager Overview</li><li>Producers, Topics, and Consumers</li><li>Data Explorer</li><li>Brokers</li><li>Topic Management</li><li>Exercise: Managing Topics using the CLI</li><li>Exercise: Connecting Producers and Consumers from the Command Line</li></ul><h5>Kafka Java API Basics</h5><ul> <li>Overview of Kafka&#039;s APIs</li><li>Topic Management from the Java API</li><li>Exercise (optional): Managing Kafka Topics Using the Java API</li><li>Using Producers and Consumers from the Java API</li><li>Exercise: Developing Producers and Consumers with the Java API</li></ul><h5>Improving Availability through Replication</h5><ul> <li>Replication</li><li>Exercise: Observing Downtime Due to Broker Failure</li><li>Considerations for the Replication Factor</li><li>Exercise: Adding Replicas to Improve Availability</li></ul><h5>Improving Application Scalability</h5><ul> <li>Partitioning</li><li>How Messages are Partitioned</li><li>Exercise: Observing How Partitioning Affects Performance</li><li>Consumer Groups</li><li>Exercise: Implementing Consumer Groups</li><li>Consumer Rebalancing</li><li>Exercise: Using a Key to Control Partition Assignment</li></ul><h5>Improving Application Reliability</h5><ul> <li>Delivery Semantics</li><li>Demonstration (optional): ISRs vs. ACKs</li><li>Producer Delivery</li><li>Exercise: Idempotent Producer</li><li>Transactions</li><li>Exercise: Transactional Producers and Consumers</li><li>Handling Consumer Failure</li><li>Offset Management</li><li>Exercise: Detecting and Suppressing Duplicate Messages</li><li>Exercise: Handling Invalid Records</li><li>Handling Producer Failure</li><li>Analyzing Kafka Clusters with SMM</li><li>End-to-End Latency</li><li>Notifiers</li><li>Alert Policies</li><li>Use Cases</li></ul><h5>Monitoring Kafka</h5><ul> <li>Monitoring Overview</li><li>Monitoring using Cloudera Manager</li><li>Charts and Reports in CM</li><li>Monitoring Recommendations</li><li>Metrics for Troubleshooting</li><li>Diagnosing Service Failure</li><li>Exercise: Monitoring Kafka</li></ul><h5>Managing Kafka</h5><ul> <li>Managing Kafka Topic Storage</li><li>Demonstration (optional): Message Retention Period</li><li>Log Cleanup and Collection</li><li>Rebalancing Partitions</li><li>Cruise Control</li><li>Exercise: Installing Cruise Control</li><li>Exercise: Troubleshooting Kafka Topics</li><li>Unclean Leader Election</li><li>Exercise: Unclean Leader Election</li><li>Adding and Removing Brokers</li><li>Exercise: Adding and Removing Brokers</li><li>Best Practices</li></ul><h5>Message Structure, Format, and Versioning</h5><ul> <li>Message Structure</li><li>Schema Registry</li><li>Defining Schemas</li><li>Schema Evolution and Versioning</li><li>Schema Registry Client</li><li>Exercise: Using an Avro Schema</li></ul><h5>Improving Application Performance</h5><ul> <li>Message Size</li><li>Batching</li><li>Compression</li><li>Exercise: Observing How Compression Affects Performance</li></ul><h5>Improving Kafka Service Performance</h5><ul> <li>Performance Tuning Strategies for the Administrator</li><li>Cluster Sizing</li><li>Exercise: Planning Capacity Needed for a Use Case</li></ul><h5>Securing the Kafka Cluster</h5><ul> <li>Encryption</li><li>Authentication</li><li>Authorization</li><li>Auditing</li></ul>During this course, you learn how to: - Plan, deploy, and operate Kafka clusters - Create and manage topics - Develop producers and consumers - Use replication to improve fault tolerance - Use partitioning to improve scalability - Troubleshoot common problems and performance issuesThis course is designed for system administrators, data engineers, and developers. All students are expected to have basic Linux experience, and basic proficiency with the Java programming language is recommended. No prior experience with Apache Kafka is necessary.- Kafka Overview - Deploying Apache Kafka - Kafka Command Line Basics - Using Streams Messaging Manager (SMM) - Kafka Java API Basics - Improving Availability through Replication - Improving Application Scalability - Improving Application Reliability - Monitoring Kafka - Managing Kafka - Message Structure, Format, and Versioning - Improving Application Performance - Improving Kafka Service Performance - Securing the Kafka ClusterKafka Overview - High-Level Architecture - Common Use Cases - Cloudera's Distribution of Apache Kafka Deploying Apache Kafka - System Requirements and Dependencies - Service Roles - Planning Your Deployment Deploying Kafka Services - Exercise: Preparing the Exercise Environment - Exercise: Installing the Kafka Service with Cloudera Manager - Exercise (optional): Create Metrics Dashboards - Exercise (optional): Using the CM API Kafka Command Line Basics - Create and Manage Topics - Running Producers and Consumers Using Streams Messaging Manager (SMM) - Streams Messaging Manager Overview - Producers, Topics, and Consumers - Data Explorer - Brokers - Topic Management - Exercise: Managing Topics using the CLI - Exercise: Connecting Producers and Consumers from the Command Line Kafka Java API Basics - Overview of Kafka's APIs - Topic Management from the Java API - Exercise (optional): Managing Kafka Topics Using the Java API - Using Producers and Consumers from the Java API - Exercise: Developing Producers and Consumers with the Java API Improving Availability through Replication - Replication - Exercise: Observing Downtime Due to Broker Failure - Considerations for the Replication Factor - Exercise: Adding Replicas to Improve Availability Improving Application Scalability - Partitioning - How Messages are Partitioned - Exercise: Observing How Partitioning Affects Performance - Consumer Groups - Exercise: Implementing Consumer Groups - Consumer Rebalancing - Exercise: Using a Key to Control Partition Assignment Improving Application Reliability - Delivery Semantics - Demonstration (optional): ISRs vs. ACKs - Producer Delivery - Exercise: Idempotent Producer - Transactions - Exercise: Transactional Producers and Consumers - Handling Consumer Failure - Offset Management - Exercise: Detecting and Suppressing Duplicate Messages - Exercise: Handling Invalid Records - Handling Producer Failure - Analyzing Kafka Clusters with SMM - End-to-End Latency - Notifiers - Alert Policies - Use Cases Monitoring Kafka - Monitoring Overview - Monitoring using Cloudera Manager - Charts and Reports in CM - Monitoring Recommendations - Metrics for Troubleshooting - Diagnosing Service Failure - Exercise: Monitoring Kafka Managing Kafka - Managing Kafka Topic Storage - Demonstration (optional): Message Retention Period - Log Cleanup and Collection - Rebalancing Partitions - Cruise Control - Exercise: Installing Cruise Control - Exercise: Troubleshooting Kafka Topics - Unclean Leader Election - Exercise: Unclean Leader Election - Adding and Removing Brokers - Exercise: Adding and Removing Brokers - Best Practices Message Structure, Format, and Versioning - Message Structure - Schema Registry - Defining Schemas - Schema Evolution and Versioning - Schema Registry Client - Exercise: Using an Avro Schema Improving Application Performance - Message Size - Batching - Compression - Exercise: Observing How Compression Affects Performance Improving Kafka Service Performance - Performance Tuning Strategies for the Administrator - Cluster Sizing - Exercise: Planning Capacity Needed for a Use Case Securing the Kafka Cluster - Encryption - Authentication - Authorization - Auditing4 Tage2970.00