Basic Kafka CLI Tools/Commands

Kafka provides list of CLI and UI based utilities those are useful while managing/debugging/monitoring various Kafka entities like: –

Cluster
Broker
Producer
Consumer
Topics
Partition
Payload

CLI Tools

zookeeper-server-start.bat

Start Zookeeper

zookeeper-server-start.bat ....\config\zookeeper.properties

kafka-server-start.bat

Start Multiple Kafka Server

kafka-server-start.bat ..\..\config\server1.properties
kafka-server-start.bat ..\..\config\server2.properties
kafka-server-start.bat ..\..\config\server3.properties

kafka-console-producer.bat

kafka-console-producer.bat --bootstrap-server localhost:9092 --topic topic1

kafka-console-producer.bat --bootstrap-server localhost:9093 --topic topic2

kafka-console-producer.bat --bootstrap-server localhost:9094 --topic topic3

kafka-console-consumer.bat

kafka-console-consumer.bat --bootstrap-server localhost:9092 --from-beginning --topic topic1

kafka-console-consumer.bat --bootstrap-server localhost:9093 --from-beginning --topic topic2

kafka-console-consumer.bat --bootstrap-server localhost:9094 --from-beginning --topic topic3

kafka-topics.bat

Create Topic’s

kafka-topics.bat --create --zookeeper localhost:2182 --replication-factor 1 --partitions 1 --topic topic1

kafka-topics.bat --create --zookeeper localhost:2182 --replication-factor 3 --partitions 1 --topic topic2

kafka-topics.bat --create --zookeeper localhost:2182 --replication-factor 3 --partitions 3 --topic topic3

Update Partitions

kafka-topics.bat --zookeeper localhost:2182 --alter --topic topic2 --partitions 3

Listing

kafka-topics.bat --zookeeper localhost:2182 --list

Listing Partitions Which are not in-sync with leader

kafka-topics.bat --describe --zookeeper localhost:2182 --under-replicated-partitions

Listing Partitions Which are Not Having Leader

kafka-topics.bat --describe --zookeeper localhost:2182 --unavailable-partitions

Describe Topics

kafka-topics.bat --describe --zookeeper localhost:2182

kafka-topics.bat --describe --zookeeper localhost:2182 --topic topic1

leader
is the node responsible for all reads and writes for the given partition. Each node will be the leader for a randomly selected portion of the partitions.

replicas
is the list of nodes that replicate the log for this partition regardless of whether they are the leader or even if they are currently alive.

isr
is the set of “in-sync” replicas. This is the subset of the replicas list that is currently alive and caught-up to the leader.

Delete Topic

Make sure below properties is set in server.properties.
delete.topic.enable=true

kafka-topics.bat --delete --zookeeper localhost:2182 --topic topic1

kafka-configs.bat

Kafka config syntax
kafka-configs.bat –zookeeper localhost:2182 –alter –entity-type –entity-name –add-config =[,=…]

For example retention for the topic named “topic2” to 1 hour (3,600,000 ms)

kafka-configs.bat --zookeeper localhost:2182 --alter --entity-type topics --entity-name topic2 --add-config retention.ms=3600000

kafka-configs.bat --zookeeper localhost:2182 --describe --entity-type topics --entity-name topic2

View all config

kafka-configs.bat --bootstrap-server localhost:9092 --describe --all --entity-type topics --entity-name topic2

Other tools

In order to trigger manually leadership for any partition after broker restarted/Stopped

kafka-preferred-replica-election.bat --zookeeper localhost:2182

Offset

kafka-run-class.bat kafka.tools.ExportZkOffsets --zkconnect localhost:2182 --group testgroup --output-file offsets

Dumping Log Segments

kafka-run-class.bat kafka.tools.DumpLogSegments --files ..\..\data\kafka-server1\topic1-0\00000000000000000000.log

kafka-run-class.bat kafka.tools.DumpLogSegments --files ..\..\data\kafka-server1\topic1-0\00000000000000000000.log --print-data-log

Manually deleting topics (not recommended)

Shutdown Brokers
Go to C:\zworkdrive\kafka\data\kafka-server1
Delete topic specific folders