
kafka是制作消息隊列的,與redis相似,但是優于redis,支援消息并列與通知,
實驗:部署zookeeper叢集:
安裝思路及環境:先安裝Java環境,再安裝zookeeper,後安裝kafka,:安裝思路及環境:先安裝Java環境,再安裝zookeeper,後安裝kafka,
zookeeper版本下載下傳網址: http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
http://zookeeper.apache.org/doc/current/zookeeperStarted.html中有:
server.1=172.18.9.203:2888:3888 server.2=172.18.9.204:2888:3888 server.3=172.18.9.205:2888:3888
制作zookeeper叢集需三台伺服器:172.18.9.203,172.18.9.204,172.18.9.205
三台伺服器分别配置hosts檔案:
#cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.18.9.203 linux-host1.exmaple.com
172.18.9.204 linux-host2.exmaple.com
172.18.9.205 linux-host3.exmaple.com
一、安裝Java環境:[root@203/204/205 src]#yum -y install jdk-8u191-linux-x64.rpm
二、安裝zookeeper:[root@203/204/205 src]#wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
[root@203/204/205 src]#tar xf zookeeper-3.4.14.tar.gz
三、制作zookeeper配置檔案:
[root@203/204/205 src]#cd zookeeper-3.4.14/conf
[root@203/204/205 conf]#ll
total 12
-rw-rw-r--. 1 2002 2002 535 Mar 7 00:50 configuration.xsl
-rw-rw-r--. 1 2002 2002 2161 Mar 7 00:50 log4j.properties
-rw-rw-r--. 1 2002 2002 922 Mar 7 00:50 zoo_sample.cfg
[root@203/204/205 conf]#cp zoo_sample.cfg zoo.cfg #把模闆cp成配置檔案
四、修改配置檔案并做軟連接配接并建立資料存放目錄:
[root@203/204/205 conf]#vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/data
clientPort=2181
maxClientCnxns=4096
server.1=172.18.9.203:2888:3888
server.2=172.18.9.204:2888:3888
server.3=172.18.9.205:2888:3888
[root@203/204/205 conf]#mkdir /usr/local/zookeeper/data
[root@203/204/205 conf]#ln -s /usr/local/src/zookeeper-3.4.14 /usr/local/zookeeper
五、開啟并檢視狀态:[root@205 conf]#/usr/local/zookeeper/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@205 conf]#/usr/local/zookeeper/bin/zkServer.sh status
Error contacting service. It is probably not running.
出現此錯的原因是zookeeper的pid沒有生成!
六、将server的編号寫入myid檔案:
[root@203/204/205 zookeeper-3.4.14]#echo 1 2 3 > /usr/local/zookeeper/data/myid
[root@203/204/205 zookeeper-3.4.14]#
七、重新開啟服務并檢視端口:
[root@203/204/205 zookeeper-3.4.14]#/usr/local/zookeeper/bin/zkServer.sh start
[root@203/204/205 zookeeper-3.4.14]#ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 50 :::2181 :::*
八、檢視叢集狀态:leader/follower 一目了然:
[root@203 zookeeper-3.4.14]#/usr/local/zookeeper/bin/zkServer.sh status
Mode: follower
[root@204 conf]#/usr/local/zookeeper/bin/zkServer.sh status
Mode: leader
九、将開啟服務指令寫入腳本、令其開機自啟:
[root@203/204/205 zookeeper-3.4.14]#vim /etc/rc.d/rc.local
/usr/local/zookeeper/bin/zkServer.sh start
此時,zookeeper已完成安裝,它沒有賬号認證,任何普戶都能用!
十、安裝kafka:
[root@203/204/205 src]#tar xf kafka_2.12-2.2.0.tar.gz
[root@203/204/205 src]#ln -s /usr/local/src/kafka_2.12-2.2.0 /usr/local/kafka
[root@203/204/205 src]#vim /usr/local/kafka/config/server.properties
21 broker.id=1 2 3 #設定每個代理全局唯一的整數ID
31 listeners=PLAINTEXT://172.18.9.203:9092
103 log.retention.hours=24 #保留指定小時的日志内容
123 zookeeper.connect=172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 #所有的zookeeper位址
十一、啟動kafka,驗證程序“jps”:
[root@203/204/205 ~]#/usr/local/kafka/bin/kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties
[root@203 src]#jps
26632 Jps
26602 Kafka
23453 QuorumPeerMain
[root@204 src]#jps
41969 Jps
41939 Kafka
34490 QuorumPeerMain
[root@205 src]#jps
19785 QuorumPeerMain
23066 Kafka
23147 Jps
十二、下載下傳安裝kafka用戶端插件:點選并安裝:
http://www.kafkatool.com/download.html Windows 64-Bit download
十三、one:測試建立topic:建立名為logstashtest,partitions(分區)為3,replication(複制)為3的topic(主題):在任意kafaka伺服器操作:
[root@204 src]#/usr/local/kafka/bin/kafka-topics.sh --create --zookeeper 172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 --partitions 3 --replication-factor 3 --topic logstashtest
Created topic logstashtest.