天天看點
傳回

Nginx負載均衡&高可用配置

Nginx負載均衡&高可用配置

文章目錄

      • 部署RS
      • 部署LB
      • 部署HA
      • 配置監控腳本實作半自動主備切換

環境說明：

各主機均已關閉防火牆與SELinux。

主機名 IP位址 應用服務 系統
LB01 192.168.92.130

keepalived

nginx

 Centos8
LB02 192.168.92.129

keepalived

nginx

 Centos8
RS01 192.168.92.132 nginx Centos8
RS02 192.168.92.133 nginx Centos8

需求：

LB01做主負載均衡器，LB02做備負載均衡器，VIP設為192.168.92.200。RS01與RS02做實際處理業務請求的伺服器。

部署RS

RS01主機配置

#安裝nginx
[[email protected] ~]# yum -y install nginx

#先将原首頁檔案備份，再定義新的首頁檔案内容
[[email protected] ~]# cd /usr/share/nginx/html/
[[email protected] html]# ls
404.html  50x.html  index.html  nginx-logo.png  poweredby.png
[[email protected] html]# mv index.html{,.bak}
[[email protected] html]# echo 'This is RS01.' > index.html
[[email protected] html]# ls
404.html  50x.html  index.html  index.html.bak  nginx-logo.png  poweredby.png

#啟動nginx并設為開機自啟
[[email protected] html]# systemctl enable --now nginx.service

RS02主機配置

[[email protected] ~]# dnf -y install nginx
[[email protected] ~]# cd /usr/share/nginx/html/
[[email protected] html]# mv index.html{,.bak}
[[email protected] html]# echo "This is RS02." > index.html
[[email protected] html]# ls
404.html  50x.html  index.html  index.html.bak  nginx-logo.png  poweredby.png
[[email protected] html]# systemctl enable --now nginx.service

測試兩台RS能否通路

[[email protected] ~]# curl 192.168.92.132
This is RS01.
[[email protected] ~]# curl 192.168.92.133
This is RS02.

部署LB

LB01主機做負載均衡

#安裝nginx
[[email protected] ~]# dnf -y install nginx

#修改配置檔案前先對原檔案做備份，養成身為運維的良好習慣
[[email protected] ~]# cd /etc/nginx/
[[email protected] nginx]# cp nginx.conf nginx.conf.bak
[[email protected] nginx]# ls
conf.d                fastcgi_params          mime.types          nginx.conf.default   uwsgi_params.default
default.d             fastcgi_params.default  mime.types.default  scgi_params          win-utf
fastcgi.conf          koi-utf                 nginx.conf          scgi_params.default
fastcgi.conf.default  koi-win                 nginx.conf.bak      uwsgi_params

#配置負載均衡
[[email protected] nginx]# vim nginx.conf
    upstream webserver {		#定義後端實際處理業務請求的伺服器池
        server 192.168.92.132;	  #RS01的IP
        server 192.168.92.133;	  #RS02的IP
    }

    server {
        listen       80;
        server_name  _;
        root         /usr/share/nginx/html;

        include /etc/nginx/default.d/*.conf;

        location / {
            proxy_pass http://webserver;	
        }


[[email protected] nginx]# systemctl enable --now nginx.service

測試負載均衡：

#因沒有配置設定權重，預設是1:1輪詢
[[email protected] nginx]# curl 192.168.92.130
This is RS01.
[[email protected] nginx]# curl 192.168.92.130
This is RS02.
[[email protected] nginx]# curl 192.168.92.130
This is RS01.
[[email protected] nginx]# curl 192.168.92.130
This is RS02.

LB02主機做負載均衡

#安裝nginx
[[email protected] ~]# dnf -y install nginx

[[email protected] ~]# cd /etc/nginx/
[[email protected] nginx]# cp nginx.conf nginx.conf.bak
[[email protected] nginx]# vim nginx.conf

    upstream webserver {
        server 192.168.92.132;
        server 192.168.92.133;
    }

    server {
        listen       80;
        server_name  _;
        root         /usr/share/nginx/html;

        include /etc/nginx/default.d/*.conf;

        location / {
            proxy_pass http://webserver;
        }

[[email protected] nginx]# systemctl start nginx.service

測試負載均衡：

[[email protected] nginx]# curl 192.168.92.129
This is RS01.
[[email protected] nginx]# curl 192.168.92.129
This is RS02.
[[email protected] nginx]# curl 192.168.92.129
This is RS01.
[[email protected] nginx]# curl 192.168.92.129
This is RS02.
#測試完停止nginx服務
[[email protected] nginx]# systemctl stop nginx.service

部署HA

LB01做主LB

#下載下傳做高可用的軟體
[[email protected] ~]# dnf -y install keepalived

#生成8位數的密碼
[[email protected] keepalived]# strings /dev/urandom |tr -dc A-Za-z0-9 | head -c8; echo
pP5ek1YA

#配置keepalived
[[email protected] keepalived]# vim keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb01
}

vrrp_instance VI_1 {
    state MASTER
    interface ens32
    virtual_router_id 81
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass pP5ek1YA
    }
    virtual_ipaddress {
        192.168.92.200
    }
}

virtual_server 192.168.92.200 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.92.130 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.92.129 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
#開啟keepalived并設為開機自啟
[[email protected] ~]# systemctl enable --now keepalived.service

#可以看到VIP已經有了
[[email protected] ~]# ip a s ens32
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:9e:e3:c1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.92.130/24 brd 192.168.92.255 scope global dynamic noprefixroute ens32
       valid_lft 1707sec preferred_lft 1707sec
    inet 192.168.92.200/32 scope global ens32
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe9e:e3c1/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

#使用VIP進行通路。如果通路不了又确信配置無誤，則極有可能是備負載均衡器服務沒停止
[[email protected] ~]# curl 192.168.92.200
This is RS01.
[[email protected] ~]# curl 192.168.92.200
This is RS02.
[[email protected] ~]# curl 192.168.92.200
This is RS01.
[[email protected] ~]# curl 192.168.92.200
This is RS02.

驗證究竟是否是LB01(主)主機在做反向代理

這裡有必要簡述一下nginx反向代理的工作流程：反向代理伺服器接收通路使用者的請求後，會代理使用者重新發起請求代理下的節點伺服器，最後把資料傳回給用戶端用。是以被代理的節點伺服器并不知道用戶端的存在，因為它所處理的全部請求都是由代理伺服器請求的。

#在LB02主機上用VIP進行通路
[[email protected] nginx]# curl 192.168.92.200
This is RS01.
[[email protected] nginx]# curl 192.168.92.200
This is RS02.
[[email protected] nginx]# curl 192.168.92.200
This is RS01.
[[email protected] nginx]# curl 192.168.92.200
This is RS02.

#在RS01主機上檢視日志
[[email protected] html]# cd /var/log/nginx/
[[email protected] nginx]# ls
access.log  error.log
#可以看到通路主機的IP确實是LB01這台
[[email protected] nginx]# tail -f access.log
192.168.92.130 - - [17/Oct/2022:20:41:21 +0800] "GET / HTTP/1.0" 200 14 "-" "curl/7.61.1" "-"
192.168.92.130 - - [17/Oct/2022:20:41:23 +0800] "GET / HTTP/1.0" 200 14 "-" "curl/7.61.1" "-"

LB02做備LB

[[email protected] ~]# dnf -y install keepalived

[[email protected] ~]# cd /etc/keepalived/
[[email protected] keepalived]# mv keepalived.conf{,.bak}
#将LB01主機的keepalived配置檔案直接copy過來
[[email protected] keepalived]# scp [email protected]:/etc/keepalived/keepalived.conf ./
[[email protected] keepalived]# ls
keepalived.conf  keepalived.conf.bak

#修改配置檔案。僅有兩個地方需要注意，其一是state，設為backup。其二是priority，一定要比主低
[[email protected] keepalived]# vim keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb01
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens32
    virtual_router_id 81
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass pP5ek1YA
    }
    virtual_ipaddress {
        192.168.92.200
    }
}

virtual_server 192.168.92.200 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.92.130 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.92.129 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[[email protected] keepalived]# systemctl enable --now keepalived.service

測試主備切換

#模拟主負載均衡器出現故障
[[email protected] ~]# systemctl stop nginx keepalived.service

#去到備負載均衡器上檢視VIP
[[email protected] ~]# ip a s ens32
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:e2:b1:9f brd ff:ff:ff:ff:ff:ff
    inet 192.168.92.129/24 brd 192.168.92.255 scope global dynamic noprefixroute ens32
       valid_lft 1317sec preferred_lft 1317sec
    inet 192.168.92.200/32 scope global ens32
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fee2:b19f/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

#啟動nginx進行負載均衡
[[email protected] ~]# systemctl start nginx.service
[[email protected] ~]# curl 192.168.92.200
This is RS01.
[[email protected] ~]# curl 192.168.92.200
This is RS02.
[[email protected] ~]# curl 192.168.92.200
This is RS01.
[[email protected] ~]# curl 192.168.92.200
This is RS02.

#來到RS01主機上檢視通路日志，可以看到此時顯示源IP是LB02
[[email protected] nginx]# tail -f access.log
192.168.92.129 - - [17/Oct/2022:21:10:31 +0800] "GET / HTTP/1.0" 200 14 "-" "curl/7.61.1" "-"
192.168.92.129 - - [17/Oct/2022:21:10:33 +0800] "GET / HTTP/1.0" 200 14 "-" "curl/7.61.1" "-"

#如果你想繼續做監控腳本實作半自動主備切換，那麼請恢複到LB01為主負載均衡器
[[email protected] ~]# systemctl stop nginx.service
[[email protected] ~]# systemctl start nginx.service keepalived.service
[[email protected] ~]# ip a s ens32
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:9e:e3:c1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.92.130/24 brd 192.168.92.255 scope global dynamic noprefixroute ens32
       valid_lft 1205sec preferred_lft 1205sec
    inet 192.168.92.200/32 scope global ens32
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe9e:e3c1/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

配置監控腳本實作半自動主備切換

所謂半自動主備切換意思是，當主ka(keepalived)挂掉了，監控腳本檢測到後，備ka會自動成為新的主ka。當舊主ka恢複後想要重新成為主卡時需要系統管理者手動切換。

LB01主機配置

[[email protected] ~]# mkdir /scripts && cd /scripts
[[email protected] scripts]# vim check_nginx.sh
#!/bin/bash
nginx_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bnginx\b'|wc -l)
if [ $nginx_status -lt 1 ];then
        systemctl stop keepalived
fi

[[email protected] scripts]# vim notify.sh
#!/bin/bash
case "$1" in
    master)
        nginx_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bnginx\b'|wc -l)
        if [ $nginx_status -lt 1 ];then
            systemctl start nginx
        fi
    ;;
    backup)
        nginx_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bnginx\b'|wc -l)
        if [ $nginx_status -gt 0 ];then
            systemctl stop nginx
        fi
    ;;
    *)
        echo "Usage:$0 master|backup VIP"
    ;;
esac

[[email protected] scripts]# chmod +x check_nginx.sh notify.sh 
[[email protected] scripts]# ll
total 8
-rwxr-xr-x 1 root root 139 Oct 17 23:09 check_nginx.sh
-rwxr-xr-x 1 root root 392 Oct 17 23:20 notify.sh

#将監控腳本配置到keepalived
[[email protected] ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb01
}
#填空以下這五行
vrrp_script nginx_check {
    script "/scripts/check_nginx.sh"
    interval 1
    weight -20
}

vrrp_instance VI_1 {
    state MASTER
    interface ens32
    virtual_router_id 81
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass pP5ek1YA
    }
    virtual_ipaddress {
        192.168.92.200
    }
    track_ipaddress{	#添加以下四行
        nginx_check
    }
    notify_master "/scripts/notify.sh master"
}

virtual_server 192.168.92.200 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.92.130 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.92.129 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[[email protected] ~]# systemctl restart keepalived.service

LB02主機配置

backup無需檢測nginx是否正常，當更新為MASTER時啟動nginx，當降級為BACKUP時關閉

[[email protected] ~]# mkdir /scripts && cd /scripts
[[email protected] scripts]# scp [email protected]:/scripts/notify.sh ./
[[email protected] scripts]# cat notify.sh 
#!/bin/bash
case "$1" in
    master)
        nginx_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bnginx\b'|wc -l)
        if [ $nginx_status -lt 1 ];then
            systemctl start nginx
        fi
    ;;
    backup)
        nginx_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bnginx\b'|wc -l)
        if [ $nginx_status -gt 0 ];then
            systemctl stop nginx
        fi
    ;;
    *)
        echo "Usage:$0 master|backup VIP"
    ;;
esac
[[email protected] scripts]# ll
total 4
-rwxr-xr-x 1 root root 376 Oct 17 23:34 notify.sh

[[email protected] scripts]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb01
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens32
    virtual_router_id 81
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass pP5ek1YA
    }
    virtual_ipaddress {
        192.168.92.200
    }
    notify_master "/scripts/notify.sh master"	#添加這兩行
    notify_backup "/scripts/notify.sh backup"
}

virtual_server 192.168.92.200 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.92.130 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.92.129 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[[email protected] scripts]# systemctl restart keepalived.service

測試配置監控腳本是否能自動進行主備切換

#目前VIP在LB01主機上，說明此時還是主
[[email protected] ~]# ip a s ens32
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:9e:e3:c1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.92.130/24 brd 192.168.92.255 scope global dynamic noprefixroute ens32
       valid_lft 1534sec preferred_lft 1534sec
    inet 192.168.92.200/32 scope global ens32
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe9e:e3c1/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

#手動停止nginx均衡負載器，模拟故障
[[email protected] ~]# systemctl stop nginx.service
#可以看到由于負載均衡器挂掉了，運作腳本停掉了keepalived。VIP也不在了
[[email protected] scripts]# systemctl status keepalived.service 
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Mon 2022-10-17 23:42:38 CST; 10s ago
[[email protected] scripts]# ip a s ens32
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:9e:e3:c1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.92.130/24 brd 192.168.92.255 scope global dynamic noprefixroute ens32
       valid_lft 1326sec preferred_lft 1326sec
    inet6 fe80::20c:29ff:fe9e:e3c1/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

#此時去到LB02檢視VIP，可以看到VIP在這台負載均衡器上了
[[email protected] ~]# ip a s ens32
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:e2:b1:9f brd ff:ff:ff:ff:ff:ff
    inet 192.168.92.129/24 brd 192.168.92.255 scope global dynamic noprefixroute ens32
       valid_lft 1230sec preferred_lft 1230sec
    inet 192.168.92.200/32 scope global ens32
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fee2:b19f/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
#可以看到nginx的預設80也随之啟用
[[email protected] ~]# ss -anlt | grep 80
LISTEN 0      128          0.0.0.0:80        0.0.0.0:*    

#要想再次啟用LB01為主，則需自行手動啟動nginx與keepalived服務
服務管理 nginx 負載均衡 伺服器
上一篇: Linux 終端複用器TmuxTmux講解配置tmux
下一篇: Linux基礎指令1（常見的系統程序、狀态指令）Linux指令格式Linux快捷按鍵常用系統指令常用系統程序指令常用系統狀态指令

繼續閱讀