kafka运维常用命令

kafka运维常用命令

列出所有的topic
./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --list

查看 topic信息
./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --describe --topic topicName

./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --describe --topic __consumer_offsets
./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --describe --topic kafka_demo_test_topic

查看所有的group
./bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 localhost:2181 --list 

查看消费情况
./bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 --describe --group kafkademo_test_group


kafka-consumer-groups.sh
老版本 消费元数据存在zk上 使用 --zookeeper localhost:2181
新版本 使用 --bootstrap-server 127.0.0.1:9092


./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --describe --topic kafka_demo_test_topic
Topic:kafka_demo_test_topic	PartitionCount:6	ReplicationFactor:1	Configs:
	Topic: kafka_demo_test_topic	Partition: 0	Leader: 0	Replicas: 0	Isr: 0
	Topic: kafka_demo_test_topic	Partition: 1	Leader: 0	Replicas: 0	Isr: 0
	Topic: kafka_demo_test_topic	Partition: 2	Leader: 0	Replicas: 0	Isr: 0
	Topic: kafka_demo_test_topic	Partition: 3	Leader: 0	Replicas: 0	Isr: 0
	Topic: kafka_demo_test_topic	Partition: 4	Leader: 0	Replicas: 0	Isr: 0
	Topic: kafka_demo_test_topic	Partition: 5	Leader: 0	Replicas: 0	Isr: 0
	
./bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 localhost:2181 --list 
Note: This will not show information about old Zookeeper-based consumers.
kafka_demo_test_topic
kafkademo_test_group

./bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 --describe --group kafkademo_test_group
Note: This will not show information about old Zookeeper-based consumers.

TOPIC                 PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                     HOST            CLIENT-ID
kafka_demo_test_topic 1          114             114             0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1
kafka_demo_test_topic 2          98              98              0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1
kafka_demo_test_topic 5          95              95              0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1
kafka_demo_test_topic 0          99              99              0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1
kafka_demo_test_topic 4          84              84              0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1
kafka_demo_test_topic 3          100             100             0               consumer-1-e989cb0d-d9b5-4473-8317-feb897e2e686 /127.0.0.1      consumer-1

CURRENT_OFFSET 当前消费offset
LOG_ENG_OFFSET  最新的offset
LAG 落后多少


增加分区数  只能加不能减
./bin/kafka-topics.sh  --zookeeper 127.0.0.1:2181 --alter --partitions 8 --topic kafka_demo_test_topic