天天看點
傳回

手把手一步一步的搭建Redis叢集(超級詳細)

環境 

  • 叢集需要多台伺服器操作，該例僅為個人練手，是以在一台上啟動多個節點做的僞叢集  
[[email protected] redis]# uname -a
Linux hadoop-master 3.10.0-1062.el7.x86_64 #1 SMP Wed Aug 7 18:08:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

 Redis版本  

redis-6.0.5

安裝步驟 

#建立操作目錄 
[[email protected] mnt]# mkdir reids
 
[[email protected] redis]# cd redis
 
#擷取資源
[[email protected] redis]# wget http://download.redis.io/releases/redis-6.0.5.tar.gz
 
#解壓壓縮包
[[email protected] redis# tar xzf redis-6.0.5.tar.gz
 
[[email protected] redis]# cd redis-6.0.5

#編譯
[[email protected] redis]# make

# 安裝到 /usr/local/redis 目錄中 安裝的檔案隻有一個bin目錄
[[email protected] redis-6.0.5]# make install PREFIX=/mnt/redis

# 建立配置檔案和data存放目錄
[[email protected] redis-6.0.5]# mkdir /mnt/redis/conf/cluster /mnt/redis/data

配置檔案  

# 配置檔案進行了精簡，完整配置可自行和官方提供的完整conf檔案進行對照。端口号自行對應修改
#背景啟動的意思
daemonize yes
 #端口号
port 6381
# IP綁定，redis不建議對公網開放，直接綁定0.0.0.0
bind 0.0.0.0
# redis資料檔案存放的目錄
dir /mnt/redis/data
# 開啟AOF
appendonly yes
 # 開啟叢集
cluster-enabled yes
# 會自動生成在上面配置的dir目錄下
cluster-config-file nodes-6381.conf
cluster-node-timeout 5000
# 這個檔案會自動生成
pidfile /var/run/redis_6381.pid
#日志級别
loglevel notice
#日志儲存路徑
logfile "/mnt/redis/logs/cluster/redis_6381.log"
 根據端口号建立多個conf檔案 
-rw-r--r-- 1 root root 696 7月  29 16:22 redis-6381.conf
-rw-r--r-- 1 root root 696 7月  29 16:35 redis-6382.conf
-rw-r--r-- 1 root root 696 7月  29 16:35 redis-6383.conf
-rw-r--r-- 1 root root 696 7月  29 16:35 redis-6384.conf
-rw-r--r-- 1 root root 695 7月  29 16:35 redis-6385.conf
-rw-r--r-- 1 root root 695 7月  29 16:36 redis-6386.conf
-rw-r--r-- 1 root root 695 7月  29 16:36 redis-6387.conf

 啟動6個Redis執行個體 

# 使用redis-server指令，并指定配置檔案
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6381.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6382.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6383.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6384.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6385.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6386.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6387.conf

檢視Redis服務  

# ps -fe | grep -v grep | grep redis
root     13428     1  0 16:36 ?        00:00:03 /mnt/redis/bin/redis-server 0.0.0.0:6381 [cluster]
root     13434     1  0 16:36 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6382 [cluster]
root     13439     1  0 16:36 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6383 [cluster]
root     13444     1  0 16:36 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6384 [cluster]
root     13450     1  0 16:37 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6385 [cluster]
root     13455     1  0 16:37 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6386 [cluster]
root     13462     1  0 16:37 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6387 [cluster]

 檢視叢集情況 

[[email protected] cluster]# /mnt/redis/bin/redis-cli -c -h 192.168.16.40 -p 6381 cluster nodes
49584c05aaa490e95bb4b01bc3f5661d78092360 :[email protected] myself,master - 0 0 0 connected

 建立叢集 

# 建立指令
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster create 192.168.16.40:6381 192.168.16.40:6382 192.168.16.40:6383 192.168.16.40:6384 192.168.16.40:6385 192.168.16.40:6386 192.168.16.40:6387 --cluster-replicas 1

#執行結果
>>> Performing hash slots allocation on 7 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.16.40:6385 to 192.168.16.40:6381
Adding replica 192.168.16.40:6386 to 192.168.16.40:6382
Adding replica 192.168.16.40:6387 to 192.168.16.40:6383
Adding extra replicas...
Adding replica 192.168.16.40:6384 to 192.168.16.40:6381
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 49584c05aaa490e95bb4b01bc3f5661d78092360 192.168.16.40:6381
   slots:[0-5460] (5461 slots) master
M: e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 192.168.16.40:6382
   slots:[5461-10922] (5462 slots) master
M: 668ee334baca38e30867d6ed13193ca3c0b4d631 192.168.16.40:6383
   slots:[10923-16383] (5461 slots) master
S: f26bc921a701a3ce7e5186468226afc8790dc6ab 192.168.16.40:6384
   replicates 668ee334baca38e30867d6ed13193ca3c0b4d631
S: c2f8e626724a621655a436f362c5023cd3dfde38 192.168.16.40:6385
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
S: 4748cd9cf310f4ed67331f5266b19efe628f36d6 192.168.16.40:6386
   replicates e4e8acfef08ea118fda4d8efc20cb70ad86b74e1
S: 311b841adb9adb14a1024d40bc26bf16a1b08bb1 192.168.16.40:6387
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
# 自動設定主從，而且會提示你，是否運作使用自動的配置
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 192.168.16.40:6381)
M: 49584c05aaa490e95bb4b01bc3f5661d78092360 192.168.16.40:6381
   slots:[0-5460] (5461 slots) master
   2 additional replica(s)
M: e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 192.168.16.40:6382
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 311b841adb9adb14a1024d40bc26bf16a1b08bb1 192.168.16.40:6387
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
S: 4748cd9cf310f4ed67331f5266b19efe628f36d6 192.168.16.40:6386
   slots: (0 slots) slave
   replicates e4e8acfef08ea118fda4d8efc20cb70ad86b74e1
S: f26bc921a701a3ce7e5186468226afc8790dc6ab 192.168.16.40:6384
   slots: (0 slots) slave
   replicates 668ee334baca38e30867d6ed13193ca3c0b4d631
S: c2f8e626724a621655a436f362c5023cd3dfde38 192.168.16.40:6385
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
M: 668ee334baca38e30867d6ed13193ca3c0b4d631 192.168.16.40:6383
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

叢集檢驗和測試 

# 再次檢查叢集情況
[[email protected] cluster]#  /mnt/redis/bin/redis-cli -c -h 192.168.16.40 -p 6381 cluster nodes
# 節點id ip+端口 角色 masterid 處理的ping數量 最後一個pong時間 節點配置版本 節點連接配接狀态 slot槽配置設定情況
e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 192.168.16.40:[email protected] master - 0 1596014067006 2 connected 5461-10922
311b841adb9adb14a1024d40bc26bf16a1b08bb1 192.168.16.40:[email protected] slave 49584c05aaa490e95bb4b01bc3f5661d78092360 0 1596014067509 7 connected
4748cd9cf310f4ed67331f5266b19efe628f36d6 192.168.16.40:[email protected] slave e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 0 1596014068511 6 connected
f26bc921a701a3ce7e5186468226afc8790dc6ab 192.168.16.40:[email protected] slave 668ee334baca38e30867d6ed13193ca3c0b4d631 0 1596014067509 3 connected
c2f8e626724a621655a436f362c5023cd3dfde38 192.168.16.40:[email protected] slave 49584c05aaa490e95bb4b01bc3f5661d78092360 0 1596014068011 5 connected
49584c05aaa490e95bb4b01bc3f5661d78092360 192.168.16.40:[email protected] myself,master - 0 1596014068000 1 connected 0-5460
668ee334baca38e30867d6ed13193ca3c0b4d631 192.168.16.40:[email protected] master - 0 1596014068000 3 connected 10923-16383


# 測試Redis Cluster的一種簡單方法是使用redis-cli指令行實用程式
# -c 是支援cluster重定向
[[email protected] cluster]# /mnt/redis/bin/redis-cli -c -h 192.168.16.40 -p 6381
192.168.16.40:6381> set a 1
-> Redirected to slot [15495] located at 192.168.16.40:6383
OK
192.168.16.40:6383> get a
"1"
192.168.16.40:6383> set hello test
-> Redirected to slot [866] located at 192.168.16.40:6381
OK
192.168.16.40:6381> get hello
"test"
192.168.16.40:6381> cluster keyslot hello
(integer) 866

 叢集slot數量整理 reshard 

#  ../bin/redis-cli --cluster help 可以檢視所有這個指令和子指令的幫助資訊
# 預設是master平均分了0-16383的所有虛拟slot
# 可以進行調整，部分節點放多一點slot(槽或者位置)
#               指令                          主機    端口                  Redis執行個體id             Redis執行個體id                數量               
/mnt/redis/bin/redis-cli --cluster reshard  <host>:<port> --cluster-from <node-id> --cluster-to <node-id> --cluster-slots <number of slots> --cluster-yes
# 執行具體指令
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster reshard 192.168.16.40:6381 --cluster-from 49584c05aaa490e95bb4b01bc3f5661d78092360 --cluster-to 668ee334baca38e30867d6ed13193ca3c0b4d631 --cluster-slots 888 of slots --cluster-yes


# 執行的結果

Moving slot 0 from 192.168.16.40:6381 to 192.168.16.40:6383:
Moving slot 1 from 192.168.16.40:6381 to 192.168.16.40:6383:
Moving slot 2 from 192.168.16.40:6381 to 192.168.16.40:6383:
Moving slot 3 from 192.168.16.40:6381 to 192.168.16.40:6383:
                .
                .
                .
Moving slot 885 from 192.168.16.40:6381 to 192.168.16.40:6383:
Moving slot 886 from 192.168.16.40:6381 to 192.168.16.40:6383:
Moving slot 887 from 192.168.16.40:6381 to 192.168.16.40:6383:

 再次檢查叢集 

# 也可以使用該指令檢視叢集資訊
# 或者  /mnt/redis/bin/redis-cli -c -h 192.168.16.40 -p 6381 cluster nodes
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster check 192.168.16.40:6381
192.168.16.40:6381 (49584c05...) -> 0 keys | 4573 slots | 2 slaves.
192.168.16.40:6382 (e4e8acfe...) -> 0 keys | 5462 slots | 1 slaves.
192.168.16.40:6383 (668ee334...) -> 2 keys | 6349 slots | 1 slaves.
[OK] 2 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.16.40:6381)
M: 49584c05aaa490e95bb4b01bc3f5661d78092360 192.168.16.40:6381
   slots:[888-5460] (4573 slots) master  可以看到該位置的區塊已經發生了變化
   2 additional replica(s)
M: e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 192.168.16.40:6382
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 311b841adb9adb14a1024d40bc26bf16a1b08bb1 192.168.16.40:6387
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
S: 4748cd9cf310f4ed67331f5266b19efe628f36d6 192.168.16.40:6386
   slots: (0 slots) slave
   replicates e4e8acfef08ea118fda4d8efc20cb70ad86b74e1
S: f26bc921a701a3ce7e5186468226afc8790dc6ab 192.168.16.40:6384
   slots: (0 slots) slave
   replicates 668ee334baca38e30867d6ed13193ca3c0b4d631
S: c2f8e626724a621655a436f362c5023cd3dfde38 192.168.16.40:6385
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
M: 668ee334baca38e30867d6ed13193ca3c0b4d631 192.168.16.40:6383
   slots:[0-887],[10923-16383] (6349 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

 手動故障轉移 

# 可能某個節點需要維護（機器下線、硬體更新、系統版本調整等等場景），需要手動的實作轉移
# 在slave節點上執行指令
CLUSTER FAILOVER 

# 注：CLUSTER  help 可以看到幫助文檔和簡介。 相對安全的做法

擴容 (增加節點)

  •  實質就是增加節點執行個體- 增加節點的時候需要指定為master或者是slave 
# 1、增加配置檔案

[[email protected] cluster]# redis-6388.conf

# 2、啟動節點
[[email protected] cluster]# /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6388.conf

# 3、加入已經存在的叢集作為master(隻能添加一次)
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster add-node 192.168.16.40:6388192.168.16.40:6381 

# 3、加入已經存在的叢集作為slave(隻能添加一次)
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster add-node 192.168.16.40:6388192.168.16.40:6381 --cluster-slave


# 3、可以手工指定master，否則就是選擇一個slave數量較少的master 
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster add-node 192.168.16.40:6388192.168.16.40:6381 --cluster-slave --cluster-master-id  668ee334baca38e30867d6ed13193ca3c0b4d631



# 檢視叢集資訊
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster check 192.168.16.40:6381
192.168.16.40:6381 (49584c05...) -> 0 keys | 4573 slots | 2 slaves.
192.168.16.40:6388 (07d0b433...) -> 0 keys | 0 slots | 0 slaves.
192.168.16.40:6382 (e4e8acfe...) -> 0 keys | 5462 slots | 1 slaves.
192.168.16.40:6383 (668ee334...) -> 2 keys | 6349 slots | 1 slaves.
[OK] 2 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.16.40:6381)
M: 49584c05aaa490e95bb4b01bc3f5661d78092360 192.168.16.40:6381
   slots:[888-5460] (4573 slots) master
   2 additional replica(s)
M: 07d0b433bd15b9e34b6400ad297bf9f81da1964d 192.168.16.40:6388 //增加了一個master節點
   slots: (0 slots) master
M: e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 192.168.16.40:6382
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 311b841adb9adb14a1024d40bc26bf16a1b08bb1 192.168.16.40:6387
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
S: 4748cd9cf310f4ed67331f5266b19efe628f36d6 192.168.16.40:6386
   slots: (0 slots) slave
   replicates e4e8acfef08ea118fda4d8efc20cb70ad86b74e1
S: f26bc921a701a3ce7e5186468226afc8790dc6ab 192.168.16.40:6384
   slots: (0 slots) slave
   replicates 668ee334baca38e30867d6ed13193ca3c0b4d631
S: c2f8e626724a621655a436f362c5023cd3dfde38 192.168.16.40:6385
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
M: 668ee334baca38e30867d6ed13193ca3c0b4d631 192.168.16.40:6383
   slots:[0-887],[10923-16383] (6349 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

縮容 (删除節點) 

# 注意：删除master的時候要把資料清空或者配置設定給其他主節點
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster del-node 192.168.16.40:6381 668ee334baca38e30867d6ed13193ca3c0b4d631

# 需要指定節點的id

 拓展問題 

# 1、 增加了slot槽的計算，是不是比單機性能差？
共16384個槽，slots槽計算方式公開的，java用戶端中就使用了：HASH_SLOT = CRC16(key) mod 16384
為了避免每次都需要伺服器計算重定向，優秀的java用戶端都實作了本地計算，和伺服器slots配置設定進行映射，有變動時再更新本地内容。

# 2、 redis叢集大小
理論是可以做到16384個槽，但是redis官方建議是最大1000個執行個體

# 3、cluster meet指令中的bus-port是什麼？

MEET <ip> <port> [bus-port]

# 4、叢集節點間的通信方式

每個Redis群集節點都有一個額外的TCP端口，用于接收來自其他Redis群集節點的傳入連接配接，每個節點使用TCP連接配接與每個其他節點連接配接。

# 5、ask和moved重定向的差別

重定向包括兩種情況
如果是确定slot不屬于目前節點，redis會傳回moved
如果目前redis節點正在處理slot遷移，則代表此處請求對應的key暫時不在此節點，傳回ask，告訴用戶端本次請求重定向

# 6、資料傾斜和通路傾斜的問題
解決辦法 調整key的政策 + slot遷移
遷移過程如下，完整的遷移流程：
在遷移目的節點執行cluster setslot <slot> IMPORTING <node ID>指令，指明需要遷移的slot和遷移源節點。
在遷移源節點執行cluster setslot <slot> MIGRATING <node ID>指令，指明需要遷移的slot和遷移目的節點。
在遷移源節點執行cluster getkeysinslot擷取該slot的key清單。
在遷移源節點執行對每個key執行migrate指令，該指令會同步把該key遷移到目的節點。
在遷移源節點反複執行cluster getkeysinslot指令，直到該slot的清單為空。
在遷移源節點和目的節點執行cluster setslot <slot> NODE <node ID>，完成遷移操作。

# 7、Pub/SUb釋出訂閱機制
# 注意： 對叢集内任意一個節點執行publish釋出資訊，這個資訊會在叢集中進行傳播，其他節點會接收到釋出的消息

Java代碼連接配接叢集  

public class ClusterConfig {
    @Bean
    public JedisConnectionFactory jedisConnectionFactory() {
        RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration(Arrays.asList(
                "192.168.16.40:6381",
                "192.168.16.40:6382",
                "192.168.16.40:6383",
                "192.168.16.40:6384",
                "192.168.16.40:6385",
                "192.168.16.40:6386"
        ));
        return new JedisConnectionFactory(redisClusterConfiguration);
    }

    @Bean
    public LettuceConnectionFactory lettuceConnectionFactory() {
        RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration(Arrays.asList(
                "192.168.16.40:6381",
                "192.168.16.40:6382",
                "192.168.16.40:6383",
                "192.168.16.40:6384",
                "192.168.16.40:6385",
                "192.168.16.40:6386"
        ));
        return new LettuceConnectionFactory(redisClusterConfiguration);
    }
}
高性能程式設計 Redis Reids叢集 redis叢集 Redis高可用
上一篇: 記錄一下自己對原碼、反碼、補碼的了解
下一篇: 【Redis】手把手教你Windows中redis的下載下傳，安裝，設定及啟動

繼續閱讀