nginx+keepalived實作高可用負載均衡

上篇文章介紹了nginx作為反向代理/負載均衡伺服器，假如nginx出現當機的話，那麼将無法轉發請求到我們後端的網站伺服器，現在介紹nginx+keepalived實作前端反向代理/負載均衡高可用架構的搭建！

實驗環境如下需要四台伺服器，其實嚴格的講隻需要三台就可以了，後端的網站伺服器可以是單台也可以是多台，說一下我這裡四台機器的軟體包都是安裝作業系統時全部安裝的。這裡環境如下：

IP位址          用途                      系統版本             nginx版本        keepalived版本

192.168.2.73    nginx+keepalived（MASTER） RedHat 4.8(64位)     1.3.5            1.1.15    

192.168.5.55    nginx+keepalived（BACKUP） RedHat 4.8(64位)     1.3.5            1.1.15    

192.168.5.54    apache（系統自帶）         RedHat 4.8(64位)     N/A              N/A

192.168.5.57    apache（系統自帶）         RedHat 4.8(64位)     N/A              N/A    

192.168.2.100   VIP（用于切換）

1、MASTER上安裝nginx

groupadd www

useradd -g www www

tar zxvf nginx-1.3.5.tar.gz

cd nginx-1.3.5

./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_stub_status_module

make

make install

2、修改/usr/local/nginx/conf/nginx.conf配置檔案如下：

user  www www;

worker_processes 1;

pid        logs/nginx.pid;

worker_rlimit_nofile 1024; 

events

{

 use epoll;

 worker_connections 1024;

} 

http

 include       mime.types;

 default_type  application/octet-stream;

 keepalive_timeout 120;

 server_tokens off;

 send_timeout 60;

 tcp_nodelay on;

 upstream  https  {

 server 192.168.5.54:8080;

 server 192.168.5.57:8080; 

 }

 log_format access_log  '$remote_addr - $remote_user [$time_local] $request'

 '"$status" $body_bytes_sent "$http_referer"'

 '"$http_user_agent" "$http_x_forwarded_for"';

 access_log  /usr/local/nginx/logs/access.log  access_log;

 server

 {

 listen  80;

 server_name  192.168.2.73;

 location / {

 proxy_set_header   Host             $host;

 proxy_set_header   X-Real-IP        $remote_addr;

 proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

 } 

}

3、檢查配置檔案是否有錯誤，出現如下兩行則說明沒問題！

/usr/local/nginx/sbin/nginx -t

nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok

nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

4、安裝keepalived

tar zxvf keepalived-1.1.15.tar.gz

vi /usr/src/kernels/2.6.9-89.EL-smp-x86_64/include/linux/types.h

将如下兩行注釋掉，否則編譯會出錯，跟我這個版本的系統有關系，你的也許不要！

/*

typedef __u16 __bitwise __sum16;

typedef __u32 __bitwise __wsum;

*/

cd keepalived-1.1.15

./configure

将keepalived作為系統服務啟動

cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/

cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/

mkdir /etc/keepalived/

cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

cp /usr/local/sbin/keepalived /usr/sbin/

5、修改/etc/keepalived/keepalived.conf配置檔案如下：

! Configuration File for keepalived

global_defs {

 router_id LVS_DEVEL

vrrp_script Monitor_Nginx {

 script "/root/scripts/monitor_nginx.sh"   #根據自己的實際路徑放置monitor_nginx.sh    

 interval 2

 weight 2

vrrp_instance VI_1 {

 state MASTER

 interface eth0

 virtual_router_id 51

 priority 100

 advert_int 1

 authentication {

 auth_type PASS

 auth_pass 1111

 track_script {

 Monitor_Nginx

 virtual_ipaddress {

 192.168.2.100

6、從keepalived配置檔案裡面看到了有一處調用了一個腳本，腳本内容如下：

#!/bin/bash

if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]

then

 /usr/local/nginx/sbin/nginx

 sleep 5

 if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]

 then

 killall keepalived

 fi

fi

7、增加可執行權限

chmod +x /root/scripts/monitor_nginx.sh

注：備機的Nginx、keepalived和以上安裝步驟一樣，隻是個别的地方要修改！

例如nginx的配置檔案裡面的server_name  192.168.2.73的IP位址改為server_name  192.168.5.55

例如keepalived的配置檔案裡面修改兩處

state MASTER修改為state BACKUP

priority 100修改為priority 99

至此MASTER和BACKUP就配置完畢了！！！

7、配置兩台apache伺服器

登入192.168.5.54上操作：

[root@hadoop5 ~]# echo 'this is 192.168.5.54!' > /var/www/html/index.html

修改/etc/httpd/conf/httpd.conf檔案的監聽端口為8080

[root@hadoop5 ~]# sed -i 's/Listen 80/Listen 8080/g' /etc/httpd/conf/httpd.conf

[root@hadoop5 ~]# /etc/init.d/httpd start

登入192.168.5.57上操作：

[root@service ~]# echo 'Hello,This is 192.168.5.57!' > /var/www/html/index.html

[root@service ~]# sed -i 's/Listen 80/Listen 8080/g' /etc/httpd/conf/httpd.conf

[root@service ~]# /etc/init.d/httpd start

8、測試

啟動MASTER的keepalived服務

/etc/init.d/keepalived start

執行ip a指令看是否有192.168.2.100的VIP出現，再檢視nginx是否已經啟動？

ps -ef | grep nginx

this is 192.168.5.54!

Hello,This is 192.168.5.57!

啟動BACKUP的keepalived服務

檢視nginx服務也随之啟動了

停止MASTER的keepalived服務，檢視BACKUP是否已接替了VIP位址？

/etc/init.d/keepalived stop

[root@nagios-server scripts]# ip a

1: lo: mtu 16436 qdisc noqueue

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000

    link/ether 00:14:22:4a:ec:39 brd ff:ff:ff:ff:ff:ff

    inet 192.168.5.55/21 brd 192.168.7.255 scope global eth0

    inet 192.168.2.100/32 scope global eth0

    inet6 fe80::214:22ff:fe4a:ec39/64 scope link

3: sit0: mtu 1480 qdisc noop

    link/sit 0.0.0.0 brd 0.0.0.0

檢視BACKUP的/var/log/messages日志是否接管VIP？

Oct 11 12:27:18 nagios-server Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.

Oct 11 12:27:18 nagios-server Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.100

Oct 11 12:27:22 nagios-server Keepalived_vrrp: ip address associated with VRID not present in received packet : 1677895872

Oct 11 12:27:22 nagios-server Keepalived_vrrp: one or more VIP associated with VRID mismatch actual MASTER advert

然後再啟動MASTER的keepalived服務，看是否接管VIP？

執行ip a指令檢視是否有192.168.2.100位址？

檢視messages日志

Oct 11 13:06:27 hadoop3 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.100

Oct 11 13:06:28 hadoop3 Keepalived_vrrp: ip address associated with VRID not present in received packet : 1677895872

Oct 11 13:06:28 hadoop3 Keepalived_vrrp: one or more VIP associated with VRID mismatch actual MASTER advert

Oct 11 13:06:28 hadoop3 Keepalived_vrrp: bogus VRRP packet received on eth0 !!!

這樣說明就OK了！！！