天天看点
返回

Centos7.0上用docker下搭建codis集群

1.环境

参数 值
宿主机ip 10.210.234.203
宿主机内核版本 3.10.0-123.el7.x86_64
宿主机cpu 4个单核cpu
docker版本 1.9.1
网络模式 host
用户 root

2.运行方式

  • 1.在docker容器中运行codis服务程序
  • 2.在宿主机上存储配置文件，docker配置文件映射到宿主机上

3.开始搭建

1.在宿主机上创建以下目录： 
mkdir -p  /data1/codis/config;
mkdir -p /data1/codis/logs;
mkdir -p /data1/codis/pid;
mkdir -p /data1/codis/out;
mkdir -p /data1/codis/storage;
mkdir -p /data1/codis/data;
目录说明:
目录 说明
config 配置文件目录
logs 日志文件目录
pid pid文件目录
assets 存放FE文件(js,css)
stroge codis元信息存储目录
data 存储redis数据文件
2.启动docker容器 
docker run -it  --net=host --name=codis_container -v /data1:/data1 centos /bin/bash

(如果本地没有centos的镜像,会从远程拉取,稍微会慢一点)

 
说明：
      --name:           容器的名称
      --net:            容器与主机的连接方式
      -v /data1:/data1  目录映射，程序配置文件及日志都会在这个目录中，docker中访问/data1目录会直接访问宿主机的/data1目录
3.进入容器,下载安装codis二进制文件包(这里不使用源码安装的方式)

# 下载:

wget https://github.com/CodisLabs/codis/releases/download/3.2.1/codis3.2.1-go1.7.6-linux.tar.gz

# 解压

tar -zxvf codis3.2.1-go1.7.6-linux.tar.gz

# 拷贝目录到/usr/local/bin/codis目录

mv codis3.2.1-go1.7.6-linux /usr/local/bin/codis

# 拷贝assets目录到宿主机(这个是web界面,拷出来方便修改)在宿主机下执行

docker cp codis_container:/usr/local/bin/codis/assets /data1/codis/assets
4.启动FE

1.进入到容器:docker exec -it codis_container

2.执行命令：

/usr/local/bin/codis/codis-fe \
--assets-dir=/data1/codis/assets \
--filesystem=/data1/codis/storage \
--log=/data1/codis/logs/fe.log \
--pidfile=/data1/codis/pid/fe.pid \
--log-level=INFO  \
--listen=...: \
>/data1/codis/out/fe.log 2>/dev/null &
           
说明：
    --assets 指定web界面所在目录
    --filesystem 以文件的形式存储codis集群元信息
    --log  fe日志文件
    --pidfile pid文件
    --listen  web服务端口

3.访问：http://x.x.x.x:9090 出现以下界面则说明启动成功,如果失败则查看log日志

Centos7.0上用docker下搭建codis集群
5.启动dashboard

1.在宿主机/data1/codis/config目录新增dashboard.toml文件

2.内容如下

coordinator_name = "filesystem"
coordinator_addr = "/data1/codis/storage"
product_name = "codis-demo"
product_auth = ""
admin_addr = "0.0.0.0:18080"
migration_method = "semi-async"
migration_parallel_slots = 
migration_async_maxbulks = 
migration_async_maxbytes = "32mb"
migration_async_numkeys = 
migration_timeout = "30s"
sentinel_client_timeout = "10s"
sentinel_quorum = 
sentinel_parallel_syncs = 
sentinel_down_after = "30s"
sentinel_failover_timeout = "5m"
sentinel_notification_script = ""
sentinel_client_reconfig_script = ""

3.在codis_container中执行

/usr/local/bin/codis/codis-dashboard \
--config=/data1/codis/config/dashboard.toml \
--log=/data1/codis/logs/dashboard.log \
--log-level=INFO \
--pidfile=/data1/codis/pid/dashboard.pid &

4.成功web界面中看到如下，失败查看日志信息

Centos7.0上用docker下搭建codis集群
6.启动proxy

1.在宿主机/data1/codis/config目录新增proxy.toml文件

2.添加如下内容

product_name = "codis-demo"
product_auth = ""
session_auth = ""
admin_addr = "0.0.0.0:11080"
proto_type = "tcp4"
proxy_addr = "0.0.0.0:19000"
jodis_name = ""
jodis_addr = ""
jodis_auth = ""
jodis_timeout = "20s"
jodis_compatible = false
proxy_datacenter = ""
proxy_max_clients = 
proxy_max_offheap_size = "1024mb"
proxy_heap_placeholder = "256mb"
backend_ping_period = "5s"
backend_recv_bufsize = "128kb"
backend_recv_timeout = "30s"
backend_send_bufsize = "128kb"
backend_send_timeout = "30s"
backend_max_pipeline = 
backend_primary_only = false
backend_primary_parallel = 
backend_replica_parallel = 
backend_keepalive_period = "75s"
backend_number_databases = 
session_recv_bufsize = "128kb"
session_recv_timeout = "30m"
session_send_bufsize = "64kb"
session_send_timeout = "30s"
session_max_pipeline = 
session_keepalive_period = "75s"
session_break_on_failure = false
metrics_report_server = ""
metrics_report_period = "1s"
metrics_report_influxdb_server = ""
metrics_report_influxdb_period = "1s"
metrics_report_influxdb_username = ""
metrics_report_influxdb_password = ""
metrics_report_influxdb_database = ""
metrics_report_statsd_server = ""
metrics_report_statsd_period = "1s"
metrics_report_statsd_prefix = ""

3.执行命令

/usr/local/bin/codis/codis-proxy \
  --config=/data1/codis/config/proxy.toml \
  --dashboard=..:  \
  --log=/data1/codis/logs/proxy.log \
  --pidfile=/data1/codis/pid/proxy.pid \
  > /data1/codis/out/proxy.log 2>/dev/null &

4.成功看到如下信息

Centos7.0上用docker下搭建codis集群
7.启动redis服务组(这里启动两组)

1.在/data1/codis/config新增redis.conf文件

2.添加内容：

protected-mode no
tcp-backlog 
timeout 
tcp-keepalive 
daemonize yes
supervised no
loglevel notice
databases 
save  
save  
save  
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename "dump.rdb"
dir "/data1/codis/data"
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 
repl-disable-tcp-nodelay no
slave-priority 
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 
auto-aof-rewrite-min-size mb
aof-load-truncated yes
lua-time-limit 
slowlog-log-slower-than 
slowlog-max-len 
latency-monitor-threshold 
notify-keyspace-events ""
hash-max-ziplist-entries 
hash-max-ziplist-value 
list-max-ziplist-size -
list-compress-depth 
set-max-intset-entries 
zset-max-ziplist-entries 
zset-max-ziplist-value 
hll-sparse-max-bytes 
activerehashing yes
client-output-buffer-limit normal   
client-output-buffer-limit slave mb mb 
client-output-buffer-limit pubsub mb mb 
hz 
aof-rewrite-incremental-fsync yes

group1:

master

/usr/local/bin/codis/codis-server /data1/codis/config/redis.conf \
--port  \
--pidfile /data1/codis/pid/redis_6379.pid  \
--logfile /data1/codis/logs/redis_6379.log

slave1

/usr/local/bin/codis/codis-server \
/data1/codis/config/redis.conf \
--port  \
--slaveof ..  \
--pidfile /data1/codis/pid/redis_1637.pid  \
--logfile /data1/codis/logs/redis_16379.log

slave2

/usr/local/bin/codis/codis-server \
/data1/codis/config/redis.conf \
--port  \
--slaveof .  \
--pidfile /data1/codis/pid/redis_26379.pid  \
--logfile /data1/codis/logs/redis_26379.log

group2:

master:
/usr/local/bin/codis/codis-server \
/data1/codis/config/redis.conf \
--port  \
--pidfile /data1/codis/pid/redis_638.pid  \
--logfile /data1/codis/logs/redis_638.log
           
slave1:
/usr/local/bin/codis/codis-server \
/data1/codis/config/redis.conf \
--port   \
--slaveof ..  \
--pidfile /data1/codis/pid/redis_1638.pid  \
--logfile /data1/codis/logs/redis_1638.log
           
slave2
 /usr/local/bin/codis/codis-server \
 /data1/codis/config/redis.conf \
 --port   \
 --slaveof ..  \
 --pidfile /data1/codis/pid/redis_2638.pid  \
 --logfile /data1/codis/logs/redis_2638.log
7.添加分组添加到集群中
Centos7.0上用docker下搭建codis集群
8.添加sentinel

1.添加sentinel.conf.001到/data1/codis/config

protected-mode no
port 
dir /data1/codis/data
sentinel monitor mymaster1   
sentinel monitor mymaster2   
sentinel down-after-milliseconds mymaster1 
sentinel down-after-milliseconds mymaster2 
sentinel parallel-syncs mymaster1 
sentinel parallel-syncs mymaster2 
sentinel failover-timeout mymaster1 
sentinel failover-timeout mymaster2 
sentinel client-reconfig-script mymaster1 /data1/codis/sentinal.sh
sentinel client-reconfig-script mymaster2 /data1/codis/sentinal.sh

2.添加sentinel.conf.002到/data1/codis/config

protected-mode no
port 
dir /data1/codis/data
sentinel monitor mymaster1   
sentinel monitor mymaster2   
sentinel down-after-milliseconds mymaster1 
sentinel down-after-milliseconds mymaster2 
sentinel parallel-syncs mymaster1 
sentinel parallel-syncs mymaster2 
sentinel failover-timeout mymaster1 
sentinel failover-timeout mymaster2 
sentinel client-reconfig-script mymaster1 /data1/codis/sentinal.sh
sentinel client-reconfig-script mymaster2 /data1/codis/sentinal.sh

3.添加sentinel.conf.003到/data1/codis/config

protected-mode no
port 
dir /data1/codis/data
sentinel monitor mymaster1   
sentinel monitor mymaster2   
sentinel down-after-milliseconds mymaster1 
sentinel down-after-milliseconds mymaster2 
sentinel parallel-syncs mymaster1 
sentinel parallel-syncs mymaster2 
sentinel failover-timeout mymaster1 
sentinel failover-timeout mymaster2 
sentinel client-reconfig-script mymaster1 /data1/codis/sentinal.sh
sentinel client-reconfig-script mymaster2 /data1/codis/sentinal.sh

4.添加到哨兵通知脚本到/data1/codis

vi /data1/codis/sentinal.sh

添加如下内容：

#!/bin/bash
echo $1 $2 $3 $4 $5 $6 $7 >> ./notice.txt

当redis故障迁移时完成时会调用：

格式：/data1/codis/sentinal.sh master-name role state from-ip from-port to-ip to-port

脚本必须以 #!/bin/bash 开头，一般会用来报警,通知客户端更新主服务器ip什么的

chmod +x /data1/codis/sentinal.sh

5.启动哨兵：

/usr/local/bin/codis/redis-sentinel   /data1/codis/config/sentinel.conf \
>/dev/null >/dev/null &
           
/usr/local/bin/codis/redis-sentinel \
 /data1/codis/config/sentinel.conf \
 >/dev/null >/dev/null &
           
/usr/local/bin/codis/redis-sentinel \
/data1/codis/config/sentinel.conf \
>/dev/null >/dev/null &

5.添加到集群中

Centos7.0上用docker下搭建codis集群
9.启动域名服务(在宿主机上安装)

1.yum -y install bind bind-utils bind-chroot

2.vim /etc/named.conf 修改如下图所示

Centos7.0上用docker下搭建codis集群

3 .vi /etc/named.rfc1912.zones 添加一组域名服务

Centos7.0上用docker下搭建codis集群

4.vim /var/named/named.service.com 新增内容

$TTL D
@       IN SOA  @ test01.service. (
                                               ; serial
                                        D      ; refresh
                                        H      ; retry
                                        W      ; expire
                                        H )    ; minimum
@ IN NS test01.service.
test01.service. IN A 
redis19000.huiyuan.comm.service. IN A 
redis19001.huiyuan.comm.service. IN A 
redis19001.huiyuan.activity.service. IN A

5.重启named服务器 (如果service不存在，执行yum install initscripts )

service named restart

6.在/etc/resolv.conf文件中新增一行

nameserver 10.210.234.203

7.ping redis19000.huiyuan.comm.service 则说明配置成功

Centos7.0上用docker下搭建codis集群

8.此时redis服务信息：

主机： redis19000.huiyuan.comm.service

端口： 19000

9.登录redis服务

redis-cli -h redis19000.huiyuan.comm.service -p 19000
集群 centos codis Docker
上一篇: Codis源码解析——proxy监听redis请求
下一篇: Codis集群搭建安装部署

继续阅读