使用docker-compose搭建redis叢集環境

主題：

主機隻有一台需要搭建三主三從的redis叢集

開盤：

**第一步：**建立目錄分發redis.conf; redis目錄裡建立data、conf目錄，将redis.conf檔案分别拷貝到conf目錄裡；修改redis.conf檔案的port（從7001-7006）

redis.conf

#Redis configuration for testing.

always-show-logo yes
notify-keyspace-events KEA
daemonize no
pidfile /var/run/redis.pid
port {按需添加端口}
timeout 0
bind 0.0.0.0
loglevel verbose
logfile ''
databases 16
latency-monitor-threshold 1

save 900 1
save 300 10
save 60 10000

rdbcompression yes
dbfilename dump.rdb
dir ./

# 持久化配置
slave-serve-stale-data yes
appendonly yes
appendfilename  appendonly.aof
appendfsync everysec
no-appendfsync-on-rewrite no
activerehashing yes

# 叢集配置
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
           

**第二步，**便攜docker-compose的yml檔案；

docker-compose.yml

version: '3.1'
services:
  redis-1:
    restart: always
    image: redis:latest
    container_name: redis-cluster-1
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7001/data:/data:rw
      - /data02/xxxx/middleware/redis/7001/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
  redis-2:
    restart: always
    image: redis:latest
    container_name: redis-cluster-2
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7002/data:/data:rw
      - /data02/xxxx/middleware/redis/7002/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
  redis-3:
    restart: always
    image: redis:latest
    container_name: redis-cluster-3
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7003/data:/data:rw
      - /data02/xxxx/middleware/redis/7003/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
  redis-4:
    restart: always
    image: redis:latest
    container_name: redis-cluster-4
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7004/data:/data:rw
      - /data02/xxxx/middleware/redis/7004/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
  redis-5:
    restart: always
    image: redis:latest
    container_name: redis-cluster-5
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7005/data:/data:rw
      - /data02/xxxx/middleware/redis/7005/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
  redis-6:
    restart: always
    image: redis:latest
    container_name: redis-cluster-6
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7006/data:/data:rw
      - /data02/xxxx/middleware/redis/7006/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
           

指令： docker-compose -f docker-compose.yml up -d #背景啟動所有的容器

[[email protected] redis]# docker-compose up -d 
Creating redis-1 ... done
Creating redis-2 ... done
Creating redis-3 ... done
Creating redis-4 ... done
Creating redis-6 ... done
Creating redis-5 ... done 
           

**第三步：**建立叢集

中途有個yes

[[email protected] redis]# docker exec -it xxxxxxxxxxxx /bin/bash redis-cli --cluster create 192.168.0.4:7001 192.168.0.4:7002 192.168.0.4:7003 192.168.0.4:7004 192.168.0.4:7005 192.168.0.4:7006 --cluster-replicas 1
.........
>>> Trying to optimize slaves allocation for anti-affinity
.........
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
....
.........
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
           

有個問題需要注意和讨論：

我在第一次安裝的時候，自建了網絡使用bridge的模式；暴露端口7001->6379,17001->16379; 這樣的話配置檔案都指向同一個redis.conf即可；啟動容器的時候可以正常啟動，

預設的情況redis是在業務端口+10000，再啟動一個端口用于redis叢集的底層gossip協定通訊，按照理論來講，按照主控端位址建立叢集即可；但是在添加的時候會卡在“Waiting for the cluster to join…”；網絡規劃指定了每個容器啟動的ip位址，後将建立叢集的ip位址改成容器位址，并制定6379端口；例如：

結果是叢集順利建立；但是在使用redis-ui測試的時候，出現了可以連接配接單機的redis，不能連接配接叢集模式；至此問題暴露，即bridge模式下的redis不能建立叢集模式；

經過一頓國内外搜尋有兩個觀點：

1、redis的官網的說明，docker建立redis叢集模式，需要配置網絡模式為host，即yml裡需要改成 network_mode: “host”

2、redis是通過gossip協定進行叢集管理的，所有節點資訊都将寫入到node.conf裡，如果配置了主控端位址；docker容器不經修改防火牆規則，不能通路到主控端的位址；即容器裡不能通路到 192.168.0.4:17001、17002…17006

還有存有疑慮的事情，請路過的高手大俠施以援手：

1、當叢集配置容器位址的時候，可以建立叢集，但是業務系統不能通路以叢集模式通路redis，這是為什麼？

2、容器為什麼不能通路到宿主控端，我了解的是從容器發出來的包通過docker0網橋未NET位址，相當于内網通路，為什麼不通呢 ？