簡介
Redis Cluster是Redis官方的一個高可用分布式解決方案,其優點是高可用,缺點是不能保證資料強一緻。在這裡使用docker容器來搭建一套6節點(3主,3從)Redis-Cluster叢集環境。
環境準備
作業系統版本:CentOS Linux release 7.6.1810 (Core)
docker版本:19.03.13
伺服器IP位址:192.168.112.136
redis版本:5.0.9
redis執行個體端口:8001--8006
docker安裝
首先需要在CentOS系統上安裝好docker軟體,目前docker軟體分為社群版ce和企業版ee,企業版是需要收費的,是以在這裡使用社群版,要安裝docker軟體,CentOS系統的核心版本高于 3.10。
下面是docker安裝步驟
yum install -y yum-utils device-mapper-persistent-data lvm2yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repoyum install docker-ce
systemctl start dockersystemctl enable docker
由于docker預設是從國外拉取鏡像,速度會比較慢,是以需要配置docker從國内的鏡像網站拉取鏡像,其配置步驟如下所示
![](https://img.laitimes.com/img/__Qf2AjLwojIjJCLyojI0JCLicmbw5SMhFTZzEjNyYzNjJ2Y4QmZ3EWNzEGNyMWNzEDMiFGO48CX0JXZ252bj91Ztl2Lc52YucWbp5GZzNmLn9Gbi1yZtl2Lc9CX6MHc0RHaiojIsJye.png)
将國内鏡像網站添加到daemon.json配置檔案
>/etc/docker/daemon.json{ "registry-mirrors": [ "https://registry.docker-cn.com", "https://docker.mirrors.ustc.edu.cn", "https://reg-mirror.qiniu.com", "https://mirror.ccs.tencentyun.com" ]}
添加完成之後,需要重新開機docker伺服器,使配置生效
systemctl daemon-reloadsystemctl restart docker
docker伺服器重新開機之後,使用docker info指令,檢查配置是否生效
redis鏡像拉取
在docker-hub官網,有很多版本的redis鏡像可供選擇,在這裡,選擇redis:5.0.9-buster進行拉取
[[email protected] ~]# docker pull redis:5.0.9-buster[[email protected] ~]# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEredis 5.0.9-buster c440306287f4 7 days ago 98.3MB
建立redis目錄和配置檔案
由于這裡需要配置6個redis執行個體的配置檔案,是以在這裡,可以先建立一個模闆配置檔案,之後使用代碼批量替換的方式來建立每個redis的配置檔案。
建立模闆檔案
mkdir -p /data/docker_redis/編寫redis_cluster.conf.template檔案vi redis_cluster.conf.templateport ${PORT}requirepass 1234masterauth 1234protected-mode nodaemonize noappendonly yescluster-enabled yescluster-config-file nodes.confcluster-node-timeout 5000cluster-announce-ip 192.168.112.136cluster-announce-port ${PORT}cluster-announce-bus-port 1${PORT}
在這裡需要講解2個參數
cluster-announce-port:此端口為redis提供服務端口,用于應用用戶端連接配接
cluster-announce-bus-port:此端口用于redis叢集進行故障檢測、配置更新、故障轉移授權和内部通訊使用,不用于應用用戶端連接配接使用。
建立6個redis執行個體的資料目錄,配置檔案目錄和配置檔案
for port in `seq 8001 8006`; do \ mkdir -p /data/redis_data/${port}/conf \ && PORT=${port} envsubst < /data/docker_redis/redis_cluster.conf.template > /data/redis_data/${port}/conf/redis.conf \ && mkdir -p /data/redis_data/${port}/data;\done
使用tree指令檢查建立的目錄和檔案
[[email protected] data]$ tree redis_data/redis_data/|-- 8001| |-- conf| | `-- redis.conf| `-- data|-- 8002| |-- conf| | `-- redis.conf| `-- data|-- 8003| |-- conf| | `-- redis.conf| `-- data|-- 8004| |-- conf| | `-- redis.conf| `-- data|-- 8005| |-- conf| | `-- redis.conf| `-- data`-- 8006 |-- conf | `-- redis.conf `-- data18 directories, 6 files
批量建立容器
在這裡使用for循環批量建立redis容器,代碼如下所示
for port in $(seq 8001 8006); do \ docker run -d -it -p ${port}:${port} -p 1${port}:1${port} --restart always --name redis-${port} \ -v /data/redis_data/${port}/conf/redis.conf:/usr/local/etc/redis/redis.conf \ -v /data/redis_data/${port}/data:/data \ redis:5.0.9-buster redis-server /usr/local/etc/redis/redis.conf; \done
在這裡使用-v參數,讓容器的redis配置參數和資料全部使用主控端的檔案目錄。這樣做的好處是,保證redis的資料不丢失。
檢視剛剛建立的redis容器
[[email protected] data]# docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESf4c971ce2d84 redis:5.0.9-buster "docker-entrypoint.s鈥 6 seconds ago Up 5 seconds 0.0.0.0:8006->8006/tcp, 6379/tcp, 0.0.0.0:18006->18006/tcp redis-80061b5a8b0a1c93 redis:5.0.9-buster "docker-entrypoint.s鈥 7 seconds ago Up 6 seconds 0.0.0.0:8005->8005/tcp, 6379/tcp, 0.0.0.0:18005->18005/tcp redis-800535d8fe01fc71 redis:5.0.9-buster "docker-entrypoint.s鈥 7 seconds ago Up 6 seconds 0.0.0.0:8004->8004/tcp, 6379/tcp, 0.0.0.0:18004->18004/tcp redis-8004408e5302378a redis:5.0.9-buster "docker-entrypoint.s鈥 8 seconds ago Up 7 seconds 0.0.0.0:8003->8003/tcp, 6379/tcp, 0.0.0.0:18003->18003/tcp redis-8003c1c23c543233 redis:5.0.9-buster "docker-entrypoint.s鈥 8 seconds ago Up 7 seconds 0.0.0.0:8002->8002/tcp, 6379/tcp, 0.0.0.0:18002->18002/tcp redis-8002c752fab6c6b8 redis:5.0.9-buster "docker-entrypoint.s鈥 9 seconds ago Up 8 seconds 0.0.0.0:8001->8001/tcp, 6379/tcp, 0.0.0.0:18001->18001/tcp redis-8001
建立redis-cluster叢集
6個redis容器建立好之後,選擇其中的一個容器進入,進行redis-cluster叢集建立
[[email protected] bin]# docker-enter [email protected]:~# redis-cli -a 1234 --cluster create 192.168.112.136:8001 192.168.112.136:8002 192.168.112.136:8003 192.168.112.136:8004 192.168.112.136:8005 192.168.112.136:8006 --cluster-replicas 1Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.>>> Performing hash slots allocation on 6 nodes...Master[0] -> Slots 0 - 5460Master[1] -> Slots 5461 - 10922Master[2] -> Slots 10923 - 16383Adding replica 192.168.112.136:8005 to 192.168.112.136:8001Adding replica 192.168.112.136:8006 to 192.168.112.136:8002Adding replica 192.168.112.136:8004 to 192.168.112.136:8003>>> Trying to optimize slaves allocation for anti-affinity[WARNING] Some slaves are in the same host as their masterM: c9e549f4d04d4466d2550d1e027412304099b1f2 192.168.112.136:8001 slots:[0-5460] (5461 slots) masterM: 0956927ee74737d5ff91a1885e77f94d531dab76 192.168.112.136:8002 slots:[5461-10922] (5462 slots) masterM: d6321788b2717b4142390c158ba70ca758f70964 192.168.112.136:8003 slots:[10923-16383] (5461 slots) masterS: 96d2cb55f163ecc13a714ba01d90348c1c3ad02f 192.168.112.136:8004 replicates c9e549f4d04d4466d2550d1e027412304099b1f2S: 8a771c26a95e82d9a6818e372d7c0226937670ac 192.168.112.136:8005 replicates 0956927ee74737d5ff91a1885e77f94d531dab76S: aa7c2f6173904973f041b35efc5200359188eb0f 192.168.112.136:8006 replicates d6321788b2717b4142390c158ba70ca758f70964Can I set the above configuration? (type 'yes' to accept): yes>>> Nodes configuration updated>>> Assign a different config epoch to each node>>> Sending CLUSTER MEET messages to join the clusterWaiting for the cluster to join.>>> Performing Cluster Check (using node 192.168.112.136:8001)M: c9e549f4d04d4466d2550d1e027412304099b1f2 192.168.112.136:8001 slots:[0-5460] (5461 slots) master 1 additional replica(s)M: 0956927ee74737d5ff91a1885e77f94d531dab76 192.168.112.136:8002 slots:[5461-10922] (5462 slots) master 1 additional replica(s)S: 96d2cb55f163ecc13a714ba01d90348c1c3ad02f 192.168.112.136:8004 slots: (0 slots) slave replicates c9e549f4d04d4466d2550d1e027412304099b1f2S: aa7c2f6173904973f041b35efc5200359188eb0f 192.168.112.136:8006 slots: (0 slots) slave replicates d6321788b2717b4142390c158ba70ca758f70964S: 8a771c26a95e82d9a6818e372d7c0226937670ac 192.168.112.136:8005 slots: (0 slots) slave replicates 0956927ee74737d5ff91a1885e77f94d531dab76M: d6321788b2717b4142390c158ba70ca758f70964 192.168.112.136:8003 slots:[10923-16383] (5461 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.
上述指令執行之後,redis-cluster叢集就搭建好了,下面是整個redis-cluster叢集的架構圖
redis-cluster叢集狀态檢查
檢查redis-cluster叢集狀态
[email protected]:~# redis-cli -a 1234 --cluster check 192.168.112.136:8001Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.192.168.112.136:8001 (c9e549f4...) -> 0 keys | 5461 slots | 1 slaves.192.168.112.136:8002 (0956927e...) -> 0 keys | 5462 slots | 1 slaves.192.168.112.136:8003 (d6321788...) -> 0 keys | 5461 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.>>> Performing Cluster Check (using node 192.168.112.136:8001)M: c9e549f4d04d4466d2550d1e027412304099b1f2 192.168.112.136:8001 slots:[0-5460] (5461 slots) master 1 additional replica(s)M: 0956927ee74737d5ff91a1885e77f94d531dab76 192.168.112.136:8002 slots:[5461-10922] (5462 slots) master 1 additional replica(s)S: 96d2cb55f163ecc13a714ba01d90348c1c3ad02f 192.168.112.136:8004 slots: (0 slots) slave replicates c9e549f4d04d4466d2550d1e027412304099b1f2S: aa7c2f6173904973f041b35efc5200359188eb0f 192.168.112.136:8006 slots: (0 slots) slave replicates d6321788b2717b4142390c158ba70ca758f70964S: 8a771c26a95e82d9a6818e372d7c0226937670ac 192.168.112.136:8005 slots: (0 slots) slave replicates 0956927ee74737d5ff91a1885e77f94d531dab76M: d6321788b2717b4142390c158ba70ca758f70964 192.168.112.136:8003 slots:[10923-16383] (5461 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.
檢查redis叢集節點資訊
[email protected]:~# redis-cli -c -a 1234 -h 192.168.112.136 -p 8001192.168.112.136:8001> cluster nodes0956927ee74737d5ff91a1885e77f94d531dab76 192.168.112.136:[email protected] master - 0 1603265855374 2 connected 5461-1092296d2cb55f163ecc13a714ba01d90348c1c3ad02f 192.168.112.136:[email protected] slave c9e549f4d04d4466d2550d1e027412304099b1f2 0 1603265856080 4 connectedaa7c2f6173904973f041b35efc5200359188eb0f 192.168.112.136:[email protected] slave d6321788b2717b4142390c158ba70ca758f70964 0 1603265855575 6 connected8a771c26a95e82d9a6818e372d7c0226937670ac 192.168.112.136:[email protected] slave 0956927ee74737d5ff91a1885e77f94d531dab76 0 1603265855575 5 connectedd6321788b2717b4142390c158ba70ca758f70964 192.168.112.136:[email protected] master - 0 1603265856382 3 connected 10923-16383c9e549f4d04d4466d2550d1e027412304099b1f2 192.168.112.136:[email protected] myself,master - 0 1603265855000 1 connected 0-5460
在這裡可以很清晰明了的看到有3個master節點和3個slave節點的IP位址和端口資訊。
redis-cluster連接配接測試
redis-cluster搭建好之後,随意連接配接一個redis節點,無論是主節點還是從節點都可以,在這裡選擇連接配接一個從節點。
192.168.112.136:8001> cluster nodes0956927ee74737d5ff91a1885e77f94d531dab76 192.168.112.136:[email protected] master - 0 1603265855374 2 connected 5461-1092296d2cb55f163ecc13a714ba01d90348c1c3ad02f 192.168.112.136:[email protected] slave c9e549f4d04d4466d2550d1e027412304099b1f2 0 1603265856080 4 connectedaa7c2f6173904973f041b35efc5200359188eb0f 192.168.112.136:[email protected] slave d6321788b2717b4142390c158ba70ca758f70964 0 1603265855575 6 connected8a771c26a95e82d9a6818e372d7c0226937670ac 192.168.112.136:[email protected] slave 0956927ee74737d5ff91a1885e77f94d531dab76 0 1603265855575 5 connectedd6321788b2717b4142390c158ba70ca758f70964 192.168.112.136:[email protected] master - 0 1603265856382 3 connected 10923-16383c9e549f4d04d4466d2550d1e027412304099b1f2 192.168.112.136:[email protected] myself,master - 0 1603265855000 1 connected [email protected]:~# redis-cli -c -a 1234 -h 192.168.112.136 -p 8004Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.192.168.112.136:8004> 192.168.112.136:8004> set 5 test5-> Redirected to slot [9974] located at 192.168.112.136:8002192.168.112.136:8002>
仔細看一下,當在slave節點寫入資料時,redis-cluster将你的寫請求重定向到另一個master節點了,至于原因,在這裡就不說了,下次在講解redis-cluster叢集原理篇再細講。
測試查詢
[email protected]:~# redis-cli -c -a 1234 -h 192.168.112.136 -p 8006Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.192.168.112.136:8006> 192.168.112.136:8006> get 5-> Redirected to slot [9974] located at 192.168.112.136:8002"test5"192.168.112.136:8002>
在這裡查詢也被路由重定向了。