修改Kafka topic的replication-factor

叢集資訊

kafka叢集有三台broker，id分别為：0，1，2

檢視topic的replication-factor資訊：

[root@ali- bin]# ./kafka-topics.sh --zookeeper localhost:2182 --describe --topic connect-configs
Topic:connect-configs   PartitionCount:    ReplicationFactor: Configs:cleanup.policy=compact
    Topic: connect-configs  Partition:     Leader:    Replicas:  Isr: 
[root@ali- bin]# 
           

可以看到

ReplicationFactor

為1，并且Leader、Replicas、Isr都在broker id=2的機器上。

編輯修改資訊json檔案

[root@ali- kafka_2.-.]# cat increase-replication-factor.json 
{"version":,
"partitions":[{"topic":"connect-configs","partition":,"replicas":[,,]}]
}
[root@ali- kafka_2.-.]# 
           

由于connect-configs這個topic隻有一個partition，檔案中指定了其partition 0的replicas在broker id = 0,1,2這三台broker上。如果有多個partition，可以在partitions後面放一個數組，針對每一個partition進行修改副本數。

執行修改指令

[[email protected] kafka_2-]# bin/kafka-reassign-partitions.sh --zookeeper localhost:2182 --reassignment-json-file increase-replication-factor.json --execute
Current partition replica assignment

{"version":,"partitions":[{"topic":"connect-configs","partition":,"replicas":[],"log_dirs":["any"]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions.
[[email protected] kafka_2-]# 
           

看到控制台輸出成功開始partitions的再配置設定，但是配置設定結果如何，需要用下面的指令驗證一下。

驗證修改是否成功

[root@ali- kafka_2-]# bin/kafka-reassign-partitions.sh --zookeeper localhost:2182 --reassignment-json-file increase-replication-factor.json --verify
Status of partition reassignment: 
Reassignment of partition connect-configs- completed successfully
[root@ali- kafka_2-]# 
           

注意這兒驗證的json檔案和上一步的是同一個檔案，隻是換成了

--verify

選項。可以看到再配置設定的狀态是成功的。

再檢視topic資訊

[root@ali- kafka_2.-.]# bin/kafka-topics.sh --zookeeper localhost:2182 --topic connect-configs --describe
Topic:connect-configs   PartitionCount:    ReplicationFactor: Configs:cleanup.policy=compact
    Topic: connect-configs  Partition:     Leader:    Replicas: ,, Isr: ,,
[root@ali- kafka_2.-.]# 
           

可以看到

ReplicationFactor

已經變成了3，Leader在broker id = 2的機器上，副本和同步副本都在broker id = 0,1,2這三台機器上。整個修改過程完成。

參考Kafka官方網站：http://kafka.apache.org/documentation/#basic_ops_increase_replication_factor