Course Overview
Log messages are created on every system and in every application. They contain important events such as errors, warnings or business transactions. Log messages are an essential tool for troubleshooting, ensuring security, and documenting business processes. In distributed systems, there are numerous log sources that are constantly changing due to scaling and updates. For most issues, logs from multiple sources, such as multiple microservices, need to be analyzed together. Loki promises to be a “Prometheus for logs” that collects log messages and makes them accessible across various sources. In this course, the management and analysis of log messages with Loki is explained and practiced through numerous practical examples. This course also covers how to use Loki in Kubernetes and how to link log messages in Loki with metrics in Prometheus.
Who should attend
The training is aimed at developers and administrators as well as architects and decision-makers with a high level of technical interest. The participants have to master the basics of Linux system administration. They need to be able to proficiently use the command line, edit files on the command line, and manage system services. Basic knowledge of Prometheus and Kubernetes as well as regular expressions is required in parts of the course.
Course Objectives
Participants learn to use Loki to centrally store log messages and analyze them across multiple sources. Special emphasis is placed on log data collection and preprocessing to ensure efficient searching of the collected data. Participants use LogQL to analyze log messages and learn how to graphically display log data in Grafana. In addition, the course addresses linking log content in Loki with monitoring metrics in Prometheus and collecting log messages from applications in Kubernetes.
Course Content
Introduction to log management with Loki
- Principles of log management
- Architecture of Loki
- Streams, labels, indexes and log messages
Collecting log messages with Promtail
- Log messages from log files
- Log messages from the systemd journal
- Overview of other log sources (syslog, Windows Event Logs, batch processing, …)
- Relabeling, especially in Kubernetes
Promtail pipelines
- Parsing log messages (logfmt, JSON, regular expressions)
- Changing timestamps, labels and log messages
- Processing multiline messages
- Filtering log messages
- Different handling of specific messages
Accessing log data
- logcli
- Grafana
LogQL
- Stream selection, log message filtering and full text search
- Extracting and comparing information from log messages
- Changing labels and log messages
- Metrics based on log messages
- Functions and operators for metric queries
Integration of Loki and Prometheus
- Labels in Loki and Prometheus
- Common display of logs and metrics in Grafana
- Provision of Prometheus metrics in Promtail
- Recording and alerting rules and alerting with AlertManager
Logging in Kubernetes
- Promtail for container logs
- Promtail as sidecar container
Overview of advanced topics
- Configuration of storage and retention
- Clustering and high availability
- Multi Tenancy, authentication and limits