Skip to content
KlagKlag

Introduction

Klag is a Kafka consumer lag exporter built with Vert.x and Micrometer. It continuously monitors consumer lag and consumer-group state across your cluster and exposes the data to Prometheus, Datadog, or any OTLP-compatible backend.

It is inspired by kafka-lag-exporter (archived in 2024), rebuilt on a modern reactive stack.

Why consumer lag matters

Section titled “Why consumer lag matters”

Consumer lag is the gap between a partition’s log-end offset and the consumer group’s committed offset. It is a useful proxy for work not yet acknowledged by the group, not proof that every record before the commit completed business processing. Left unmonitored, growing lag leads to:

  • Stale data in downstream systems.
  • Memory pressure as consumers struggle to catch up.
  • Silent failures when consumer groups die without alerts.

Klag surfaces these problems early, with enough signal to act before users notice. Monitoring lag and group health is an essential operational practice for production Kafka; Klag is one exporter that can supply those signals to your existing observability stack.

Key features

Section titled “Key features”
Feature Why it matters
Lag velocity Know if lag is growing or shrinking, to catch problems before they escalate.
Time-based lag estimation See lag in seconds/minutes, beyond raw message counts.
Hot partition detection Find partitions with uneven load causing bottlenecks.
Consumer group state tracking Alert on preparing_rebalance, completing_rebalance, dead, or empty states.
Request batching Safely monitor large clusters without overwhelming brokers.
Stale group cleanup Automatically stops reporting deleted/inactive groups.
Data loss prevention Catch the case where lag exceeds retention and data is lost.

Control collection load

Section titled “Control collection load”

Request batching with configurable delays lets you spread offset requests across a collection cycle and control broker load on larger clusters. See Group Filtering and the KAFKA_CHUNK_COUNT / KAFKA_CHUNK_DELAY_MS settings in the Configuration Reference.

Read-only by design

Section titled “Read-only by design”

Klag requires read-only (DESCRIBE) access to Kafka, no write or alter permissions. See ACL Permissions for the exact grants on self-managed Kafka and Confluent Cloud.

Next steps

Section titled “Next steps”