$ command-helper
enru

Apache Kafka

Topics, console producer and consumer, consumer groups, lag and offset resets.

Requires Google sign-in

Topics

List topics
kafka-topics.sh --bootstrap-server <broker> --list
Describe a topic (partitions, replicas, ISR)
kafka-topics.sh --bootstrap-server <broker> --describe --topic <topic>
Create a topic
kafka-topics.sh --bootstrap-server <broker> --create --topic <topic> --partitions 3 --replication-factor 3
Increase partition countadvanced
kafka-topics.sh --bootstrap-server <broker> --alter --topic <topic> --partitions 6

Partitions can't be decreased. Per-key ordering breaks after the change.

Delete a topic
kafka-topics.sh --bootstrap-server <broker> --delete --topic <topic>
Under-replicated partitionsadvanced
kafka-topics.sh --bootstrap-server <broker> --describe --under-replicated-partitions

Producer and consumer

Read a topic from the beginning
kafka-console-consumer.sh --bootstrap-server <broker> --topic <topic> --from-beginning
Read with keys, partitions and timestamps
kafka-console-consumer.sh --bootstrap-server <broker> --topic <topic> --property print.key=true --property print.partition=true --property print.timestamp=true
Read N messages and exit
kafka-console-consumer.sh --bootstrap-server <broker> --topic <topic> --from-beginning --max-messages 10
Send messages (one per line)
kafka-console-producer.sh --bootstrap-server <broker> --topic <topic>
Send messages with a key
kafka-console-producer.sh --bootstrap-server <broker> --topic <topic> --property parse.key=true --property key.separator=:

Consumer groups

List groups
kafka-consumer-groups.sh --bootstrap-server <broker> --list
Group lag per partition
kafka-consumer-groups.sh --bootstrap-server <broker> --describe --group <group>
Group members
kafka-consumer-groups.sh --bootstrap-server <broker> --describe --group <group> --members
Reset offsets to earliest (dry run)advanced
kafka-consumer-groups.sh --bootstrap-server <broker> --group <group> --topic <topic> --reset-offsets --to-earliest --dry-run

Replace --dry-run with --execute to apply. The group must be stopped.

Reset offsets to a point in timeadvanced
kafka-consumer-groups.sh --bootstrap-server <broker> --group <group> --topic <topic> --reset-offsets --to-datetime 2024-01-01T00:00:00.000 --execute

Configuration

Topic config
kafka-configs.sh --bootstrap-server <broker> --entity-type topics --entity-name <topic> --describe
Change topic retention (7 days)advanced
kafka-configs.sh --bootstrap-server <broker> --entity-type topics --entity-name <topic> --alter --add-config retention.ms=604800000
Latest offsets per partitionadvanced
kafka-get-offsets.sh --bootstrap-server <broker> --topic <topic>

For Kafka before 3.0 use kafka-run-class.sh kafka.tools.GetOffsetShell.