Redis目前高可用的架構非常多,比如keepalived+redis,redis cluster,twemproxy,codis,這些架構各有優劣,今天暫且不說這些架構,今天主要說說redis sentinel高可用架構。
它的主要功能有以下幾點
- 不時地監控redis是否按照預期良好地運作;
- 如果發現某個redis節點運作出現狀況,能夠通知另外一個程序(例如它的用戶端);
- 能夠進行自動切換。當一個master節點不可用時,能夠選舉出master的多個slave(如果有超過一個slave的話)中的一個來作為新的master,其它的slave節點會将它所追随的master的位址改為被提升為master的slave的新位址。
關于更加詳細的配置以及介紹推薦看完以下文章,我在這裡就不多說了,直接進行搭建:
http://segmentfault.com/a/1190000002680804
http://segmentfault.com/a/1190000002685515
redis sentinel的架構如下圖:
![](https://img.laitimes.com/img/__Qf2AjLwojIjJCLyojI0JCLicmbw5iN1MjMxkzNzETMtcTO5ETM0MTOwITMyETNxAjMtATM3kDM28CXyETNxAjMvwFMxcTOwYzLcd2bsJ2Lc12bj5ycn9Gbi52YuUTMwIzcldWYtl2Lc9CX6MHc0RHaiojIsJye.png)
當然Redis-Sentinel推薦使用3個或者3個以上節點,至于為什麼這麼做看完我上面給的文章連結。
環境介紹:
Redis Sentinel5台伺服器:
10.36.30.203
10.36.30.204
10.37.124.202
10.37.124.203
10.37.124.204
這裡不要覺得浪費,這樣做是為了更加安全高效的監控redis,且redis Sentinel可以進行複用,也就是可以監控多個Redis執行個體,是以伺服器不存在浪費。
Redis 伺服器2台,1主1從:
10.69.25.173 master
10.69.30.170 slave
5台Sentinel的配置檔案内容如下:
port 26379
dir "/data/redis/sentinel/26379"
daemonize yes
logfile "/data/redis/sentinel/26379/sentinel.log"
# 6379
sentinel monitor master-6379 10.69.25.173 6379 3
sentinel down-after-milliseconds master-6379 15000
sentinel parallel-syncs master-6379 1
sentinel failover-timeout master-6379 180000
sentinel client-reconfig-script master-6379 /sh/redis/notify.py
其中sentinel client-reconfig-script master-6379 /sh/redis/notify.py是在主從切換以後發送告警郵件。其他參數的意義參考我給的文章連結。相關目錄自己建立好。
notify.py腳本内容如下,5台伺服器上面都需要存在,因為你不知道哪個節點會被選舉為leader(網上還沒有人提到切換發送告警郵件問題):
#!/usr/bin/python
#coding:utf8
import sys
import time
import smtplib
import logging
from email.mime.text import MIMEText
from email.message import Message
from email.header import Header
alarm_mail =['[email protected]']
def main():
failover_time=time.strftime("%Y-%m-%d %H:%M:%S")
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s',
datefmt='%Y-%m-%d %H:%M:%S',
filename='/sh/redis/failover.log',
filemode='a')
console = logging.StreamHandler()
console.setLevel(logging.INFO)
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)
mail_host='xxxxx'
mail_port=25
mail_user='xxxxxxx'
mail_pass='xxxxxxxx'
mail_send_from = 'xxxxxxx'
def send_mail(to_list,sub,content):
me=mail_send_from
msg = MIMEText(content, _subtype='html', _charset='utf-8')
msg['Subject'] = Header(sub,'utf-8')
msg['From'] = Header(me,'utf-8')
msg['To'] = ";".join(to_list)
try:
smtp = smtplib.SMTP()
smtp.connect(mail_host,mail_port)
smtp.login(mail_user,mail_pass)
smtp.sendmail(me,to_list, msg.as_string())
smtp.close()
return True
except Exception as error:
logging.error("郵件發送失敗: %s" % (error))
return False
try:
master_name = sys.argv[1]
role = sys.argv[2]
from_ip = sys.argv[4]
from_port = sys.argv[5]
to_ip = sys.argv[6]
to_port = sys.argv[7]
except Exception as error:
logging.error('從 Sentinel 擷取參數錯誤: %s ' % (error))
sys.exit(1)
sub='redis %s faiover' % (master_name)
nodify_message = "%s %s is failover end. sentinel find redis master %s:%s is down. failover to slave %s:%s" % (failover_time,master_name,from_ip,from_port,to_ip,to_port)
if role == 'leader':
logging.info(nodify_message)
send_mail(alarm_mail,sub,nodify_message)
if __name__ == "__main__":
main()
View Code
10.69.25.173 master
10.69.30.170 slave
自己安裝完成redis,并且搭建好複制關系。
現在分别在5台Sentinel伺服器上面啟動Sentinel,有2種方式啟動。哪兩種自己看前面文章。
redis-sentinel sentinel.conf
啟動以後随便找一台伺服器檢視日志,輸出如下提示:
[18219] 12 Dec 09:56:47.161 # Sentinel runid is f3086fc39145cb3d832785899699050d2c7f3b08
[18219] 12 Dec 09:56:47.161 # +monitor master master-6379 10.69.25.173 6379 quorum 1
[18219] 12 Dec 09:56:47.183 * +slave slave 10.69.30.170:6379 10.69.30.170 6379 @ master-6379 10.69.25.173 6379
這裡的+slave就表示找到了一個從庫。
再看看其他sentinel伺服器的日志:
[1480] 12 Dec 09:58:37.250 # Sentinel runid is 812f9f8b860dcc73d4b587e3bdf85df13808a3cd
[1480] 12 Dec 09:58:37.250 # +monitor master master-6379 10.69.25.173 6379 quorum 1
[1480] 12 Dec 09:58:38.252 * +slave slave 10.69.30.170:6379 10.69.30.170 6379 @ master-6379 10.69.25.173 6379
[1480] 12 Dec 09:58:38.304 * +sentinel sentinel 10.36.30.204:26379 10.36.30.204 26379 @ master-6379 10.69.25.173 6379
[1480] 12 Dec 09:58:38.388 * +sentinel sentinel 10.37.124.202:26379 10.37.124.202 26379 @ master-6379 10.69.25.173 6379
[1480] 12 Dec 09:58:38.461 * +sentinel sentinel 10.37.124.203:26379 10.37.124.203 26379 @ master-6379 10.69.25.173 6379
[1480] 12 Dec 09:58:39.423 * +sentinel sentinel 10.37.124.204:26379 10.37.124.204 26379 @ master-6379 10.69.25.173 6379
+sentinel表示發現了其他的sentinel伺服器。現在整個叢集就已經工作了。
首先進入sentinel檢視現在的主節點是哪台伺服器(随便哪台sentinel都可以):
redis-cli -p 26379
127.0.0.1:26379> info Sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=master-6379,status=ok,address=10.69.25.173:6379,slaves=1,sentinels=5
127.0.0.1:26379>
可以看到現在的主庫是10.69.25.173:6379。現在我們把這台伺服器的redis程序kill掉,檢視是否會進行切換:
pkill -9 redis
再次檢視,發現主庫已經是原來的從庫了。
而且還會收到告警郵件,内容如下:
127.0.0.1:26379> info Sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=master-6379,status=ok,address=10.69.30.170:6379,slaves=1,sentinels=5
127.0.0.1:26379>
同樣的,如果把剛才kill掉的reids重新啟動,又會把啟動的redis設定為10.69.30.170的從庫。
[1480] 12 Dec 10:01:48.921 # +new-epoch 1
[1480] 12 Dec 10:01:48.933 # +vote-for-leader 92517289efcb4ae695eff3e064fde7f4e0e43a1f 1
[1480] 12 Dec 10:01:48.955 # +sdown master master-6379 10.69.25.173 6379
[1480] 12 Dec 10:01:48.955 # +odown master master-6379 10.69.25.173 6379 #quorum 1/1
[1480] 12 Dec 10:01:48.955 # Next failover delay: I will not start a failover before Sat Dec 12 10:07:49 2015
[1480] 12 Dec 10:01:50.067 # +config-update-from sentinel 10.37.124.203:26379 10.37.124.203 26379 @ master-6379 10.69.25.173 6379
[1480] 12 Dec 10:01:50.067 # +switch-master master-6379 10.69.25.173 6379 10.69.30.170 6379
[1480] 12 Dec 10:01:50.067 * +slave slave 10.69.25.173:6379 10.69.25.173 6379 @ master-6379 10.69.30.170 6379
[1480] 12 Dec 10:02:05.109 # +sdown slave 10.69.25.173:6379 10.69.25.173 6379 @ master-6379 10.69.30.170 6379
[1480] 12 Dec 10:03:19.241 # -sdown slave 10.69.25.173:6379 10.69.25.173 6379 @ master-6379 10.69.30.170 6379
[1480] 12 Dec 10:03:29.219 * +convert-to-slave slave 10.69.25.173:6379 10.69.25.173 6379 @ master-6379 10.69.30.170 6379
那麼用戶端如何知道主從進行切換了呢,如果是java那麼有jedis用戶端比較友善,如果是php,python語言呢,我們可以自己進行判斷。當然還有另外一種方法就是采用dns,修改dns解析。
我這裡用python簡單寫了一個daemon,不會php,哎。
#!/usr/bin/python
import redis
import os
sentinel_server=['10.36.30.203:26379','10.36.30.204:26379','10.37.124.202:26379','10.37.124.203:26379','10.37.124.204:26379']
def queue(host,port):
str=''.join(map(lambda xx:(hex(ord(xx))[2:]),os.urandom(16)))
pool = redis.ConnectionPool(host=host, port=port, db=0)
r = redis.Redis(connection_pool=pool)
r.lpush('low_task_queue',str)
def get_sentinel():
global master_host
global master_port
for info in sentinel_server:
host=info.split(':')[0]
port=info.split(':')[1]
try:
r = redis.Redis(host=host, port=port)
info=r.info('sentinel')['master0']['address'].split(':')
master_host=info[0]
master_port=info[1]
except Exception as error:
print 'concat to sentinel error: %s' % (error)
pass
else:
break
if __name__ == "__main__":
get_sentinel()
while True:
try:
queue(master_host,master_port)
except Exception as error:
print 'conct redis error %s' % (error)
get_sentinel()
continue
如果引入dns,那麼架構圖可以是下面這樣:
以上就是簡單的測試了,更多的測試交給大家了。
總結:
Redis Sentinel實作高可用還是比較靠譜的,後面線上也打算使用。需要注意的是Redis Sentinel節點推薦3個以上。相比keepalived+redis實作高可用更靠譜,且keepalived+redis還不能管理多個執行個體,這點是比較麻煩的。
參考資料:
http://redis.io/topics/sentinel-clients
https://pypi.python.org/pypi/redis/
作者:Atlas
出處:Atlas的部落格 http://www.cnblogs.com/gomysql
您的支援是對部落客最大的鼓勵,感謝您的認真閱讀。本文版權歸作者所有,歡迎轉載,但請保留該聲明。如果您需要技術支援,本人亦提供有償服務。