天天看点
返回

手把手一步一步的搭建Redis集群(超级详细)

环境 

  • 集群需要多台服务器操作，该例仅为个人练手，所以在一台上启动多个节点做的伪集群  
[[email protected] redis]# uname -a
Linux hadoop-master 3.10.0-1062.el7.x86_64 #1 SMP Wed Aug 7 18:08:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

 Redis版本  

redis-6.0.5

安装步骤 

#创建操作目录 
[[email protected] mnt]# mkdir reids
 
[[email protected] redis]# cd redis
 
#获取资源
[[email protected] redis]# wget http://download.redis.io/releases/redis-6.0.5.tar.gz
 
#解压压缩包
[[email protected] redis# tar xzf redis-6.0.5.tar.gz
 
[[email protected] redis]# cd redis-6.0.5

#编译
[[email protected] redis]# make

# 安装到 /usr/local/redis 目录中 安装的文件只有一个bin目录
[[email protected] redis-6.0.5]# make install PREFIX=/mnt/redis

# 创建配置文件和data存放目录
[[email protected] redis-6.0.5]# mkdir /mnt/redis/conf/cluster /mnt/redis/data

配置文件  

# 配置文件进行了精简，完整配置可自行和官方提供的完整conf文件进行对照。端口号自行对应修改
#后台启动的意思
daemonize yes
 #端口号
port 6381
# IP绑定，redis不建议对公网开放，直接绑定0.0.0.0
bind 0.0.0.0
# redis数据文件存放的目录
dir /mnt/redis/data
# 开启AOF
appendonly yes
 # 开启集群
cluster-enabled yes
# 会自动生成在上面配置的dir目录下
cluster-config-file nodes-6381.conf
cluster-node-timeout 5000
# 这个文件会自动生成
pidfile /var/run/redis_6381.pid
#日志级别
loglevel notice
#日志保存路径
logfile "/mnt/redis/logs/cluster/redis_6381.log"
 根据端口号创建多个conf文件 
-rw-r--r-- 1 root root 696 7月  29 16:22 redis-6381.conf
-rw-r--r-- 1 root root 696 7月  29 16:35 redis-6382.conf
-rw-r--r-- 1 root root 696 7月  29 16:35 redis-6383.conf
-rw-r--r-- 1 root root 696 7月  29 16:35 redis-6384.conf
-rw-r--r-- 1 root root 695 7月  29 16:35 redis-6385.conf
-rw-r--r-- 1 root root 695 7月  29 16:36 redis-6386.conf
-rw-r--r-- 1 root root 695 7月  29 16:36 redis-6387.conf

 启动6个Redis实例 

# 使用redis-server命令，并指定配置文件
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6381.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6382.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6383.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6384.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6385.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6386.conf
  /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6387.conf

查看Redis服务  

# ps -fe | grep -v grep | grep redis
root     13428     1  0 16:36 ?        00:00:03 /mnt/redis/bin/redis-server 0.0.0.0:6381 [cluster]
root     13434     1  0 16:36 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6382 [cluster]
root     13439     1  0 16:36 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6383 [cluster]
root     13444     1  0 16:36 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6384 [cluster]
root     13450     1  0 16:37 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6385 [cluster]
root     13455     1  0 16:37 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6386 [cluster]
root     13462     1  0 16:37 ?        00:00:02 /mnt/redis/bin/redis-server 0.0.0.0:6387 [cluster]

 查看集群情况 

[[email protected] cluster]# /mnt/redis/bin/redis-cli -c -h 192.168.16.40 -p 6381 cluster nodes
49584c05aaa490e95bb4b01bc3f5661d78092360 :[email protected] myself,master - 0 0 0 connected

 创建集群 

# 创建命令
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster create 192.168.16.40:6381 192.168.16.40:6382 192.168.16.40:6383 192.168.16.40:6384 192.168.16.40:6385 192.168.16.40:6386 192.168.16.40:6387 --cluster-replicas 1

#执行结果
>>> Performing hash slots allocation on 7 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.16.40:6385 to 192.168.16.40:6381
Adding replica 192.168.16.40:6386 to 192.168.16.40:6382
Adding replica 192.168.16.40:6387 to 192.168.16.40:6383
Adding extra replicas...
Adding replica 192.168.16.40:6384 to 192.168.16.40:6381
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 49584c05aaa490e95bb4b01bc3f5661d78092360 192.168.16.40:6381
   slots:[0-5460] (5461 slots) master
M: e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 192.168.16.40:6382
   slots:[5461-10922] (5462 slots) master
M: 668ee334baca38e30867d6ed13193ca3c0b4d631 192.168.16.40:6383
   slots:[10923-16383] (5461 slots) master
S: f26bc921a701a3ce7e5186468226afc8790dc6ab 192.168.16.40:6384
   replicates 668ee334baca38e30867d6ed13193ca3c0b4d631
S: c2f8e626724a621655a436f362c5023cd3dfde38 192.168.16.40:6385
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
S: 4748cd9cf310f4ed67331f5266b19efe628f36d6 192.168.16.40:6386
   replicates e4e8acfef08ea118fda4d8efc20cb70ad86b74e1
S: 311b841adb9adb14a1024d40bc26bf16a1b08bb1 192.168.16.40:6387
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
# 自动设置主从，而且会提示你，是否运行使用自动的配置
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 192.168.16.40:6381)
M: 49584c05aaa490e95bb4b01bc3f5661d78092360 192.168.16.40:6381
   slots:[0-5460] (5461 slots) master
   2 additional replica(s)
M: e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 192.168.16.40:6382
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 311b841adb9adb14a1024d40bc26bf16a1b08bb1 192.168.16.40:6387
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
S: 4748cd9cf310f4ed67331f5266b19efe628f36d6 192.168.16.40:6386
   slots: (0 slots) slave
   replicates e4e8acfef08ea118fda4d8efc20cb70ad86b74e1
S: f26bc921a701a3ce7e5186468226afc8790dc6ab 192.168.16.40:6384
   slots: (0 slots) slave
   replicates 668ee334baca38e30867d6ed13193ca3c0b4d631
S: c2f8e626724a621655a436f362c5023cd3dfde38 192.168.16.40:6385
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
M: 668ee334baca38e30867d6ed13193ca3c0b4d631 192.168.16.40:6383
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

集群检验和测试 

# 再次检查集群情况
[[email protected] cluster]#  /mnt/redis/bin/redis-cli -c -h 192.168.16.40 -p 6381 cluster nodes
# 节点id ip+端口 角色 masterid 处理的ping数量 最后一个pong时间 节点配置版本 节点连接状态 slot槽分配情况
e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 192.168.16.40:[email protected] master - 0 1596014067006 2 connected 5461-10922
311b841adb9adb14a1024d40bc26bf16a1b08bb1 192.168.16.40:[email protected] slave 49584c05aaa490e95bb4b01bc3f5661d78092360 0 1596014067509 7 connected
4748cd9cf310f4ed67331f5266b19efe628f36d6 192.168.16.40:[email protected] slave e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 0 1596014068511 6 connected
f26bc921a701a3ce7e5186468226afc8790dc6ab 192.168.16.40:[email protected] slave 668ee334baca38e30867d6ed13193ca3c0b4d631 0 1596014067509 3 connected
c2f8e626724a621655a436f362c5023cd3dfde38 192.168.16.40:[email protected] slave 49584c05aaa490e95bb4b01bc3f5661d78092360 0 1596014068011 5 connected
49584c05aaa490e95bb4b01bc3f5661d78092360 192.168.16.40:[email protected] myself,master - 0 1596014068000 1 connected 0-5460
668ee334baca38e30867d6ed13193ca3c0b4d631 192.168.16.40:[email protected] master - 0 1596014068000 3 connected 10923-16383


# 测试Redis Cluster的一种简单方法是使用redis-cli命令行实用程序
# -c 是支持cluster重定向
[[email protected] cluster]# /mnt/redis/bin/redis-cli -c -h 192.168.16.40 -p 6381
192.168.16.40:6381> set a 1
-> Redirected to slot [15495] located at 192.168.16.40:6383
OK
192.168.16.40:6383> get a
"1"
192.168.16.40:6383> set hello test
-> Redirected to slot [866] located at 192.168.16.40:6381
OK
192.168.16.40:6381> get hello
"test"
192.168.16.40:6381> cluster keyslot hello
(integer) 866

 集群slot数量整理 reshard 

#  ../bin/redis-cli --cluster help 可以查看所有这个命令和子命令的帮助信息
# 默认是master平均分了0-16383的所有虚拟slot
# 可以进行调整，部分节点放多一点slot(槽或者位置)
#               命令                          主机    端口                  Redis实例id             Redis实例id                数量               
/mnt/redis/bin/redis-cli --cluster reshard  <host>:<port> --cluster-from <node-id> --cluster-to <node-id> --cluster-slots <number of slots> --cluster-yes
# 执行具体命令
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster reshard 192.168.16.40:6381 --cluster-from 49584c05aaa490e95bb4b01bc3f5661d78092360 --cluster-to 668ee334baca38e30867d6ed13193ca3c0b4d631 --cluster-slots 888 of slots --cluster-yes


# 执行的结果

Moving slot 0 from 192.168.16.40:6381 to 192.168.16.40:6383:
Moving slot 1 from 192.168.16.40:6381 to 192.168.16.40:6383:
Moving slot 2 from 192.168.16.40:6381 to 192.168.16.40:6383:
Moving slot 3 from 192.168.16.40:6381 to 192.168.16.40:6383:
                .
                .
                .
Moving slot 885 from 192.168.16.40:6381 to 192.168.16.40:6383:
Moving slot 886 from 192.168.16.40:6381 to 192.168.16.40:6383:
Moving slot 887 from 192.168.16.40:6381 to 192.168.16.40:6383:

 再次检查集群 

# 也可以使用该命令查看集群信息
# 或者  /mnt/redis/bin/redis-cli -c -h 192.168.16.40 -p 6381 cluster nodes
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster check 192.168.16.40:6381
192.168.16.40:6381 (49584c05...) -> 0 keys | 4573 slots | 2 slaves.
192.168.16.40:6382 (e4e8acfe...) -> 0 keys | 5462 slots | 1 slaves.
192.168.16.40:6383 (668ee334...) -> 2 keys | 6349 slots | 1 slaves.
[OK] 2 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.16.40:6381)
M: 49584c05aaa490e95bb4b01bc3f5661d78092360 192.168.16.40:6381
   slots:[888-5460] (4573 slots) master  可以看到该位置的区块已经发生了变化
   2 additional replica(s)
M: e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 192.168.16.40:6382
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 311b841adb9adb14a1024d40bc26bf16a1b08bb1 192.168.16.40:6387
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
S: 4748cd9cf310f4ed67331f5266b19efe628f36d6 192.168.16.40:6386
   slots: (0 slots) slave
   replicates e4e8acfef08ea118fda4d8efc20cb70ad86b74e1
S: f26bc921a701a3ce7e5186468226afc8790dc6ab 192.168.16.40:6384
   slots: (0 slots) slave
   replicates 668ee334baca38e30867d6ed13193ca3c0b4d631
S: c2f8e626724a621655a436f362c5023cd3dfde38 192.168.16.40:6385
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
M: 668ee334baca38e30867d6ed13193ca3c0b4d631 192.168.16.40:6383
   slots:[0-887],[10923-16383] (6349 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

 手动故障转移 

# 可能某个节点需要维护（机器下线、硬件升级、系统版本调整等等场景），需要手动的实现转移
# 在slave节点上执行命令
CLUSTER FAILOVER 

# 注：CLUSTER  help 可以看到帮助文档和简介。 相对安全的做法

扩容 (增加节点)

  •  实质就是增加节点实例- 增加节点的时候需要指定为master或者是slave 
# 1、增加配置文件

[[email protected] cluster]# redis-6388.conf

# 2、启动节点
[[email protected] cluster]# /mnt/redis/bin/redis-server /mnt/redis/conf/cluster/redis-6388.conf

# 3、加入已经存在的集群作为master(只能添加一次)
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster add-node 192.168.16.40:6388192.168.16.40:6381 

# 3、加入已经存在的集群作为slave(只能添加一次)
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster add-node 192.168.16.40:6388192.168.16.40:6381 --cluster-slave


# 3、可以手工指定master，否则就是选择一个slave数量较少的master 
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster add-node 192.168.16.40:6388192.168.16.40:6381 --cluster-slave --cluster-master-id  668ee334baca38e30867d6ed13193ca3c0b4d631



# 查看集群信息
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster check 192.168.16.40:6381
192.168.16.40:6381 (49584c05...) -> 0 keys | 4573 slots | 2 slaves.
192.168.16.40:6388 (07d0b433...) -> 0 keys | 0 slots | 0 slaves.
192.168.16.40:6382 (e4e8acfe...) -> 0 keys | 5462 slots | 1 slaves.
192.168.16.40:6383 (668ee334...) -> 2 keys | 6349 slots | 1 slaves.
[OK] 2 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.16.40:6381)
M: 49584c05aaa490e95bb4b01bc3f5661d78092360 192.168.16.40:6381
   slots:[888-5460] (4573 slots) master
   2 additional replica(s)
M: 07d0b433bd15b9e34b6400ad297bf9f81da1964d 192.168.16.40:6388 //增加了一个master节点
   slots: (0 slots) master
M: e4e8acfef08ea118fda4d8efc20cb70ad86b74e1 192.168.16.40:6382
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 311b841adb9adb14a1024d40bc26bf16a1b08bb1 192.168.16.40:6387
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
S: 4748cd9cf310f4ed67331f5266b19efe628f36d6 192.168.16.40:6386
   slots: (0 slots) slave
   replicates e4e8acfef08ea118fda4d8efc20cb70ad86b74e1
S: f26bc921a701a3ce7e5186468226afc8790dc6ab 192.168.16.40:6384
   slots: (0 slots) slave
   replicates 668ee334baca38e30867d6ed13193ca3c0b4d631
S: c2f8e626724a621655a436f362c5023cd3dfde38 192.168.16.40:6385
   slots: (0 slots) slave
   replicates 49584c05aaa490e95bb4b01bc3f5661d78092360
M: 668ee334baca38e30867d6ed13193ca3c0b4d631 192.168.16.40:6383
   slots:[0-887],[10923-16383] (6349 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

缩容 (删除节点) 

# 注意：删除master的时候要把数据清空或者分配给其他主节点
[[email protected] cluster]# /mnt/redis/bin/redis-cli --cluster del-node 192.168.16.40:6381 668ee334baca38e30867d6ed13193ca3c0b4d631

# 需要指定节点的id

 拓展问题 

# 1、 增加了slot槽的计算，是不是比单机性能差？
共16384个槽，slots槽计算方式公开的，java客户端中就使用了：HASH_SLOT = CRC16(key) mod 16384
为了避免每次都需要服务器计算重定向，优秀的java客户端都实现了本地计算，和服务器slots分配进行映射，有变动时再更新本地内容。

# 2、 redis集群大小
理论是可以做到16384个槽，但是redis官方建议是最大1000个实例

# 3、cluster meet命令中的bus-port是什么？

MEET <ip> <port> [bus-port]

# 4、集群节点间的通信方式

每个Redis群集节点都有一个额外的TCP端口，用于接收来自其他Redis群集节点的传入连接，每个节点使用TCP连接与每个其他节点连接。

# 5、ask和moved重定向的区别

重定向包括两种情况
如果是确定slot不属于当前节点，redis会返回moved
如果当前redis节点正在处理slot迁移，则代表此处请求对应的key暂时不在此节点，返回ask，告诉客户端本次请求重定向

# 6、数据倾斜和访问倾斜的问题
解决办法 调整key的策略 + slot迁移
迁移过程如下，完整的迁移流程：
在迁移目的节点执行cluster setslot <slot> IMPORTING <node ID>命令，指明需要迁移的slot和迁移源节点。
在迁移源节点执行cluster setslot <slot> MIGRATING <node ID>命令，指明需要迁移的slot和迁移目的节点。
在迁移源节点执行cluster getkeysinslot获取该slot的key列表。
在迁移源节点执行对每个key执行migrate命令，该命令会同步把该key迁移到目的节点。
在迁移源节点反复执行cluster getkeysinslot命令，直到该slot的列表为空。
在迁移源节点和目的节点执行cluster setslot <slot> NODE <node ID>，完成迁移操作。

# 7、Pub/SUb发布订阅机制
# 注意： 对集群内任意一个节点执行publish发布信息，这个信息会在集群中进行传播，其他节点会接收到发布的消息

Java代码连接集群  

public class ClusterConfig {
    @Bean
    public JedisConnectionFactory jedisConnectionFactory() {
        RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration(Arrays.asList(
                "192.168.16.40:6381",
                "192.168.16.40:6382",
                "192.168.16.40:6383",
                "192.168.16.40:6384",
                "192.168.16.40:6385",
                "192.168.16.40:6386"
        ));
        return new JedisConnectionFactory(redisClusterConfiguration);
    }

    @Bean
    public LettuceConnectionFactory lettuceConnectionFactory() {
        RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration(Arrays.asList(
                "192.168.16.40:6381",
                "192.168.16.40:6382",
                "192.168.16.40:6383",
                "192.168.16.40:6384",
                "192.168.16.40:6385",
                "192.168.16.40:6386"
        ));
        return new LettuceConnectionFactory(redisClusterConfiguration);
    }
}
高性能编程 Redis Reids集群 redis集群 Redis高可用
上一篇: 记录一下自己对原码、反码、补码的理解
下一篇: 【Redis】手把手教你Windows中redis的下载，安装，设置及启动

继续阅读