天天看點

在Windows環境與Linux環境下搭建Zookeeper單機環境與叢集環境

作者:馬士兵教育CTO

下載下傳與安裝

1.通路位址: https://zookeeper.apache.org/releases.html#download下載下傳需要的ZK版本,下載下傳到本地後解壓。

2.在解壓目錄下面建立一個空的 data 檔案夾和 log 檔案夾

在Windows環境與Linux環境下搭建Zookeeper單機環境與叢集環境

配置

将 conf 目錄下的 zoo_sample.cfg 檔案,複制一份,重命名為 zoo.cfg

在Windows環境與Linux環境下搭建Zookeeper單機環境與叢集環境

修改 zoo.cfg 配置檔案,将 dataDir=/tmp/zookeeper 修改成 zookeeper 安裝目錄所在的 data 檔案夾,再添加一條添加資料日志的配置

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=D:\Development\apache-zookeeper-3.6.3-bin\data
dataLogDir=D:\Development\apache-zookeeper-3.6.3-bin\log
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

## Metrics Providers
#
# https://prometheus.io Metrics Exporter
#metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
#metricsProvider.httpPort=7000
#metricsProvider.exportJvmInfo=true
複制代碼           

啟動

輕按兩下bin目錄下的zkServer.cmd啟動即可

在Windows環境與Linux環境下搭建Zookeeper單機環境與叢集環境

zookeeper新版本中有個内嵌的管理控制台是通過jetty啟動,會占用8080 端口

在Windows環境與Linux環境下搭建Zookeeper單機環境與叢集環境

解決方法

1.修改端口

在zoo.cfg中增加admin.serverPort=沒有被占用的端口号
複制代碼           
admin.serverPort=8088
複制代碼           

2.修改啟動腳本

在啟動腳本中增加"-Dzookeeper.admin.enableServer=false"
複制代碼           
"-Dzookeeper.admin.enableServer=false"
複制代碼           

Linux環境下的安裝

下載下傳與安裝

通路位址: zookeeper.apache.org/releases.ht…

wget https://downloads.apache.org/zookeeper/zookeeper-3.7.0/apache-zookeeper-3.7.0-bin.tar.gz
複制代碼           

解壓zookeeper

[root@administrator conf]#	tar -zxvf apache-zookeeper-3.7.0-bin.tar.gz
複制代碼           

重命名

[root@administrator conf]#	mv tar -zxvf apache-zookeeper-3.7.0-bin zookeeper
複制代碼           

移動zookeeper位置

[root@administrator conf]# cp zookeeper  /usr/local/
複制代碼           

配置

修改zoo_sample.cfg檔案

[root@administrator conf]# cd /usr/local/zookeeper/conf
[root@administrator conf]# mv zoo_sample.cfg zoo.cfg
複制代碼           

修改dataDir目錄,改成真實輸出目錄

# The number of milliseconds of each tick
# 基本事件單元,以毫秒為機關。它用來控制心跳和逾時,預設情況下最小的會話逾時時間為兩個心跳時間
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
# 用于限制follower跟随者伺服器必須連接配接到leader上司者伺服器的時限
# 多少個心跳時間内,允許其他server連接配接并初始化資料,如果ZooKeeper管理的資料較大,則應相應增大這個值
# 它以tickTime的倍數來表示。當超過設定倍數的tickTime時間,則連接配接失敗
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
# leader與follower之間發送消息,請求和應答時間長度。
# 如果follower在設定的時間内不能與leader進行通信,那麼此follower将被丢棄。
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
#dataDir=/tmp/zookeeper
#存儲記憶體資料庫快照的位置
dataDir=/usr/local/zookeeper/data
# the port at which the clients will connect
# 監聽用戶端連接配接的端口,預設是2181
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

## Metrics Providers
#
# https://prometheus.io Metrics Exporter
#metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
#metricsProvider.httpPort=7000
#metricsProvider.exportJvmInfo=true
複制代碼           

建立data目錄

[root@administrator conf]# cd /usr/local/zookeeper/
[root@administrator zookeeper]# mkdir data
複制代碼           

啟動

注意:zookeeper需要依賴jdk環境

[root@administrator bin]# cd /usr/local/zookeeper/bin
[root@administrator bin]# ./zkServer.sh start
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@administrator bin]# 
複制代碼           

檢視啟動狀态

[root@administrator bin]# ./zkServer.sh status
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: standalone
複制代碼           

停止

[root@administrator bin]# ./zkServer.sh stop
複制代碼           

測試

[root@administrator bin]# ./zkCli.sh -server 127.0.0.1:2181

[zk: 127.0.0.1:2181(CONNECTED) 8] ls
ls [-s] [-w] [-R] path
複制代碼           

退出用戶端

[zk: 127.0.0.1:2181(CONNECTED) 0] quit
複制代碼           

搭建Zookeeper叢集環境

搭建Zookeeper叢集至少需要三台伺服器,且伺服器之間系統時間保持一緻,三台伺服器中一個leader和兩個follower

下載下傳與安裝

[root@node001 zookeeper]# wget https://downloads.apache.org/zookeeper/zookeeper-3.7.0/apache-zookeeper-3.7.0-bin.tar.gz
複制代碼           

解壓并重命名

[root@node001 zookeeper]# tar -zxvf apache-zookeeper-3.7.0-bin.tar.gz

[root@node001 zookeeper]# mv apache-zookeeper-3.7.0-bin zookeeper
複制代碼           

配置

建立data目錄

[root@node001 zookeeper]# cd zookeeper

[root@node001 zookeeper]# mkdir data
複制代碼           

建立myid檔案,填寫每個節點編号(node001 ==> 1),需唯一。

[root@node001 zookeeper]# cd data
[root@node001 zookeeper]# vim myid

1
複制代碼           

配置zoo.cfg

[root@node001 zookeeper]# cd conf/

[root@node001 zookeeper]# mv zoo_sample.cfg zoo.cfg

[root@node001 zookeeper]# vim zoo.cfg
複制代碼           
# The number of milliseconds of each tick
# 心跳時間
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
# 用于限制follower跟随者伺服器必須連接配接到leader上司者伺服器的時限
# 多少個心跳時間内,允許其他server連接配接并初始化資料,如果ZooKeeper管理的資料較大,則應相應增大這個值
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
# 叢集中Leader與Follower之間的最大響應時間機關
# 多少個tickTime内,允許follower同步,如果follower落後太多,則會被丢棄。
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
# 修改資料存儲路徑
# 存儲記憶體資料庫快照的位置
dataDir=/usr/local/program/zookeeper/data
# the port at which the clients will connect
# 用于偵聽用戶端連接配接的端口
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

## Metrics Providers
#
# https://prometheus.io Metrics Exporter
#metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
#metricsProvider.httpPort=7000
#metricsProvider.exportJvmInfo=true

#3台伺服器的位址
# server.A=B:C:D
# 添加叢集配置
server.1=node001:2888:3888
server.2=node002:2888:3888
server.3=node003:2888:3888
複制代碼           
A :一個數字,表示是第幾号伺服器

B:伺服器的位址

C:伺服器Follower與叢集中的Leader伺服器交換資訊的端口

D:萬一叢集中的Leader伺服器挂了,需要一個端口來重新進行選舉一個新的Leader,而這個端口就是用來執行選舉時伺服器互相通信的端口。
複制代碼           

分發到其他節點

scp -r zookeeper node002:/usr/local/program/zookeeper

scp -r zookeeper node003:/usr/local/program/zookeeper
複制代碼           

修改分發Zookeeper機器的myid編号,分别為2,3

[root@node002 zookeeper]# vim data/myid 
2

[root@node003 zookeeper]# vim data/myid 
3
複制代碼           

啟動叢集

[root@node001 zookeeper]# bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/program/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[root@node002 zookeeper]# bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/program/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED


[root@node003 zookeeper]# bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/program/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
複制代碼           

檢視叢集狀态

[root@node001 zookeeper]# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/program/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leader

[root@node002 zookeeper]# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/program/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

[root@node003 zookeeper]# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/program/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower
複制代碼           

檢視程序

[root@node001 hadoop]# jps
18946 QuorumPeerMain
19675 Jps

[root@node002 hadoop]# jps
21860 QuorumPeerMain
22095 Jps

[root@node003 hadoop]# jps
25607 Jps
25547 JournalNode
複制代碼           

常用指令

ZK操作

指令 作用
zkServer.sh start/stop 啟動/關閉服務
zkCli.sh start/stop 啟動/關閉用戶端
zkServer.sh status 檢視狀态(leader/follower)
bin/zkCli.sh 連接配接本機zk
zkCli.sh -server node001:2181 連接配接其他zk伺服器
status /zk 檢視/zk節點的狀态資訊

節點ZNode的建立

指令 作用
create /zk mydata 建立znode節點,永久+不帶序号
create -e /zk mydata 建立臨時znode節點
create -s /zk mydata 建立順序znode節點
create -e -s /zk mydata 建立臨時的順序znode節點

節點ZNode的查詢

指令 作用
ls /zk 檢視znode子節點清單
ls /zk watch 對一個節點的子節點變化事件注冊了監聽
get /zk 擷取znode資料
get /zk watch 對一個節點的資料内容變化事件注冊了監聽

節點ZNode的修改與删除

指令 作用
set /zk value 設定znode資料
delete /zk 隻能删除沒有子znode的znode
rmr /zk 删除znode的znode,遞歸删除

資料資訊字段

znode資料資訊字段的解釋

cZxid = 0x1000fe0597c0002 節點建立的時候的zxid
# The zxid of the change that caused this znode to be created.

ctime = Fri Dec 02 16:41:50 PST 2022 節點建立的時間
# The time in milliseconds from epoch when this znode was created.

mZxid = 0x1000fe0597c0002 節點修改的時候的zxid,與子節點的修改無關
# The zxid of the change that last modified this znode.

mtime = Fri Dec 02 16:41:50 PST 2022 節點的修改的時間
# The time in milliseconds from epoch when this znode was last modified.

pZxid = 0x1000fe0597c0002 和子節點的建立/删除對應的zxid,和修改無關,和孫子節點無關
# The zxid of the change that last modified children of this znode.

cversion = 0 子節點的更新次數
# The number of changes to the children of this znode.

dataVersion = 0 節點資料的更新次數
# The number of changes to the data of this znode.

aclVersion = 0 節點(ACL)的更新次數
# The number of changes to the ACL of this znode.

ephemeralOwner = 0x0 如果該節點為ephemeral節點, ephemeralOwner值表示與該節點綁定的
session id. 如果該節點不是ephemeral節點, ephemeralOwner值為0
# The session id of the owner of this znode if the znode is an ephemeral node.
If it is not an ephemeral node, it will be zero.

dataLength = 6 節點資料的位元組數
# The length of the data field of this znode.

numChildren = 0 子節點個數,不包含孫子節點
# The number of children of this znode.           

繼續閱讀