kafka修改分區和副本數
檢視現在副本配置設定情況
../bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --describe --topic test1
Topic:test1 PartitionCount:3 ReplicationFactor:2 Configs:
Topic: test1 Partition: 0 Leader: 2 Replicas: 2,4 Isr: 2,4
Topic: test1 Partition: 1 Leader: 3 Replicas: 3,5 Isr: 3,5
Topic: test1 Partition: 2 Leader: 4 Replicas: 4,1 Isr: 4,1
topic 分區擴容
./kafka-topics.sh --zookeeper 127.0.0.1:2181 -alter --partitions 4 --topic test1
修改備份數量
這個檔案自己建立 格式按照下面的格式就可以了
根據topic的分區情況自行修改 partitions-topic.json 檔案配置
{
"partitions":
[
{
"topic": "test1",
"partition": 0,
"replicas": [1,2]
},
{
"topic": "test1",
"partition": 1,
"replicas": [0,3]
},
{
"topic": "test1",
"partition": 2,
"replicas": [4,5]
}
],
"version":1
}
執行副本搬遷
../bin/kafka-reassign-partitions.sh --zookeeper 127.0.0.1:2181 --reassignment-json-file partitions-topic.json --execute
檢視遷移情況：
../bin/kafka-reassign-partitions.sh --zookeeper 127.0.0.1:2181 --reassignment-json-file partitions-topic.json --verify
Status of partition reassignment:
Reassignment of partition [mx_prd_nginx_access,0] is still in progress
Reassignment of partition [mx_prd_nginx_access,1] completed successfully
Reassignment of partition [mx_prd_nginx_access,2] is still in progress
注釋
kafka-reassign-partitions.sh工具來重新分布分區。該工具有三種使用模式：
generate模式，給定需要重新配置設定的Topic，自動生成reassign plan(并不執行)
execute模式，根據指定的reassign plan重新配置設定Partition
verify模式，驗證重新配置設定Partition是否成功