天天看點

20190426 部署Zookeeper叢集

20190426 部署Zookeeper叢集

kafka是制作消息隊列的,與redis相似,但是優于redis,支援消息并列與通知,

實驗:部署zookeeper叢集:

安裝思路及環境:先安裝Java環境,再安裝zookeeper,後安裝kafka,:安裝思路及環境:先安裝Java環境,再安裝zookeeper,後安裝kafka,

zookeeper版本下載下傳網址: http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz

http://zookeeper.apache.org/doc/current/zookeeperStarted.html中有:

server.1=172.18.9.203:2888:3888 server.2=172.18.9.204:2888:3888 server.3=172.18.9.205:2888:3888

制作zookeeper叢集需三台伺服器:172.18.9.203,172.18.9.204,172.18.9.205

三台伺服器分别配置hosts檔案:

#cat /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4

::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

172.18.9.203 linux-host1.exmaple.com

172.18.9.204 linux-host2.exmaple.com

172.18.9.205 linux-host3.exmaple.com

一、安裝Java環境:[root@203/204/205 src]#yum -y install jdk-8u191-linux-x64.rpm
二、安裝zookeeper:[root@203/204/205 src]#wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz

[root@203/204/205 src]#tar xf zookeeper-3.4.14.tar.gz

三、制作zookeeper配置檔案:

[root@203/204/205 src]#cd zookeeper-3.4.14/conf

[root@203/204/205 conf]#ll

total 12

-rw-rw-r--. 1 2002 2002 535 Mar 7 00:50 configuration.xsl

-rw-rw-r--. 1 2002 2002 2161 Mar 7 00:50 log4j.properties

-rw-rw-r--. 1 2002 2002 922 Mar 7 00:50 zoo_sample.cfg

[root@203/204/205 conf]#cp zoo_sample.cfg zoo.cfg #把模闆cp成配置檔案

四、修改配置檔案并做軟連接配接并建立資料存放目錄:

[root@203/204/205 conf]#vim zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

dataDir=/usr/local/zookeeper/data

clientPort=2181

maxClientCnxns=4096

server.1=172.18.9.203:2888:3888

server.2=172.18.9.204:2888:3888

server.3=172.18.9.205:2888:3888

[root@203/204/205 conf]#mkdir /usr/local/zookeeper/data

[root@203/204/205 conf]#ln -s /usr/local/src/zookeeper-3.4.14 /usr/local/zookeeper

五、開啟并檢視狀态:[root@205 conf]#/usr/local/zookeeper/bin/zkServer.sh start

ZooKeeper JMX enabled by default

Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg

Starting zookeeper ... STARTED

[root@205 conf]#/usr/local/zookeeper/bin/zkServer.sh status

Error contacting service. It is probably not running.

出現此錯的原因是zookeeper的pid沒有生成!

六、将server的編号寫入myid檔案:

[root@203/204/205 zookeeper-3.4.14]#echo 1 2 3 > /usr/local/zookeeper/data/myid

[root@203/204/205 zookeeper-3.4.14]#

七、重新開啟服務并檢視端口:

[root@203/204/205 zookeeper-3.4.14]#/usr/local/zookeeper/bin/zkServer.sh start

[root@203/204/205 zookeeper-3.4.14]#ss -ntl

State Recv-Q Send-Q Local Address:Port Peer Address:Port

LISTEN 0 50 :::2181 :::*

八、檢視叢集狀态:leader/follower 一目了然:

[root@203 zookeeper-3.4.14]#/usr/local/zookeeper/bin/zkServer.sh status

Mode: follower

[root@204 conf]#/usr/local/zookeeper/bin/zkServer.sh status

Mode: leader

九、将開啟服務指令寫入腳本、令其開機自啟:

[root@203/204/205 zookeeper-3.4.14]#vim /etc/rc.d/rc.local

/usr/local/zookeeper/bin/zkServer.sh start

此時,zookeeper已完成安裝,它沒有賬号認證,任何普戶都能用!

十、安裝kafka:

[root@203/204/205 src]#tar xf kafka_2.12-2.2.0.tar.gz

[root@203/204/205 src]#ln -s /usr/local/src/kafka_2.12-2.2.0 /usr/local/kafka

[root@203/204/205 src]#vim /usr/local/kafka/config/server.properties

21 broker.id=1 2 3 #設定每個代理全局唯一的整數ID

31 listeners=PLAINTEXT://172.18.9.203:9092

103 log.retention.hours=24 #保留指定小時的日志内容

123 zookeeper.connect=172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 #所有的zookeeper位址

十一、啟動kafka,驗證程序“jps”:

[root@203/204/205 ~]#/usr/local/kafka/bin/kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties

[root@203 src]#jps

26632 Jps

26602 Kafka

23453 QuorumPeerMain

[root@204 src]#jps

41969 Jps

41939 Kafka

34490 QuorumPeerMain

[root@205 src]#jps

19785 QuorumPeerMain

23066 Kafka

23147 Jps

十二、下載下傳安裝kafka用戶端插件:點選并安裝:

http://www.kafkatool.com/download.html Windows 64-Bit download

20190426 部署Zookeeper叢集
20190426 部署Zookeeper叢集
十三、one:測試建立topic:建立名為logstashtest,partitions(分區)為3,replication(複制)為3的topic(主題):在任意kafaka伺服器操作:

[root@204 src]#/usr/local/kafka/bin/kafka-topics.sh --create --zookeeper 172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 --partitions 3 --replication-factor 3 --topic logstashtest

Created topic logstashtest.