kafka運維常用指令

kafka運維常用指令

列出所有的topic
./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --list

檢視 topic資訊
./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --describe --topic topicName

./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --describe --topic __consumer_offsets
./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --describe --topic kafka_demo_test_topic

檢視所有的group
./bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 localhost:2181 --list 

檢視消費情況
./bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 --describe --group kafkademo_test_group


kafka-consumer-groups.sh
老版本 消費中繼資料存在zk上 使用 --zookeeper localhost:2181
新版本 使用 --bootstrap-server 127.0.0.1:9092


./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --describe --topic kafka_demo_test_topic
Topic:kafka_demo_test_topic	PartitionCount:6	ReplicationFactor:1	Configs:
	Topic: kafka_demo_test_topic	Partition: 0	Leader: 0	Replicas: 0	Isr: 0
	Topic: kafka_demo_test_topic	Partition: 1	Leader: 0	Replicas: 0	Isr: 0
	Topic: kafka_demo_test_topic	Partition: 2	Leader: 0	Replicas: 0	Isr: 0
	Topic: kafka_demo_test_topic	Partition: 3	Leader: 0	Replicas: 0	Isr: 0
	Topic: kafka_demo_test_topic	Partition: 4	Leader: 0	Replicas: 0	Isr: 0
	Topic: kafka_demo_test_topic	Partition: 5	Leader: 0	Replicas: 0	Isr: 0
	
./bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 localhost:2181 --list 
Note: This will not show information about old Zookeeper-based consumers.
kafka_demo_test_topic
kafkademo_test_group

./bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 --describe --group kafkademo_test_group
Note: This will not show information about old Zookeeper-based consumers.

TOPIC                 PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                     HOST            CLIENT-ID
kafka_demo_test_topic 1          114             114             0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1
kafka_demo_test_topic 2          98              98              0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1
kafka_demo_test_topic 5          95              95              0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1
kafka_demo_test_topic 0          99              99              0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1
kafka_demo_test_topic 4          84              84              0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1
kafka_demo_test_topic 3          100             100             0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1

CURRENT_OFFSET 目前消費offset
LOG_ENG_OFFSET  最新的offset
LAG 落後多少


增加分區數  隻能加不能減
./bin/kafka-topics.sh  --zookeeper 127.0.0.1:2181 --alter --partitions 8 --topic kafka_demo_test_topic