一、LVS-dr的基本原理
direct routing,它通过修改请求报文的目标MAC地址进行转发。
请求报文经由director发送至RS,那么就不能让除Director外的RS响应,因此,有三种解决方案:1.在上游路由器进行IP和MAC的绑定,2.在RS上做arptables,3.修改RS主机内核参数,也就是说vip别名在环回口,修改的内核参数能使vip对发来的请求不做处理。
响应报文不经由director,而是有vip直接响应cip,其中vip通过rip的MAC和gateway直接响应cip。
dr的基本规则:
(1)保证前端路由器将目标IP为VIP的请求报文发送给director
解决方案:
静态绑定
arptables
修改RS主机内核的参数
(2)RS的RIP可以使用私有地址;但也可以使用公网地址
(3)RS跟Director必须在同一物理网络中
(4)请求报文经由Director调度,但响应报文一定不能经由Director;
(5)不支持端口映射
(6)RS可以大多数OS
(7)RS的网关不能指向DIP
二、实验环境,局域网环境。
<code>Client:本机windows7</code>
<code>Director:CentOS 7.1</code>
<code>RealServer:node1,node2均为CentOS6.7</code>
<code>Director与RealServer的所有IP均为同一网段</code>
拓扑如下:
<a href="http://s4.51cto.com/wyfs02/M02/89/E5/wKiom1gggljS4SWhAACMEUQrhDM442.png-wh_500x0-wm_3-wmp_4-s_1110250351.png" target="_blank"></a>
三、配置
<code>director:</code>
<code>[root@localhost ~]</code><code># ifconfig ens33:0 192.168.1.15/32 broadcast 192.168.1.15 up</code>
<code>[root@localhost ~]</code><code># route add -host 192.168.1.15 dev ens33:0</code>
<code>RS:</code>
<code> </code><code>node1:</code>
<code> </code><code>[root@jymlinux ~]</code><code># echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore </code>
<code> </code><code>[root@jymlinux ~]</code><code># echo 1 > /proc/sys/net/ipv4/conf/eth2/arp_ignore </code>
<code> </code><code>[root@jymlinux ~]</code><code># echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce </code>
<code> </code><code>[root@jymlinux ~]</code><code># echo 2 > /proc/sys/net/ipv4/conf/eth2/arp_announce </code>
<code> </code><code>[root@jymlinux ~]</code><code># ifconfig lo:0 192.168.1.15/32 broadcast 192.168.1.15 up</code>
<code> </code><code>[root@jymlinux ~]</code><code># route add -host 192.168.1.15 dev lo:0</code>
<code> </code><code>node2:</code>
<code> </code><code>[root@jymlinux ~]</code><code># echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore</code>
<code> </code><code>[root@jymlinux ~]</code><code># echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore</code>
<code> </code><code>[root@jymlinux ~]</code><code># echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce </code>
<code> </code>
<code>或写脚本并给予执行权限设置内核参数</code>
<code>[root@jymlinux ~]</code><code># vim lvsdrka.sh</code>
<code>#!/bin/bash</code>
<code>#</code>
<code>case</code> <code>$1 </code><code>in</code>
<code>start)</code>
<code> </code><code>echo</code> <code>1 > </code><code>/proc/sys/net/ipv4/conf/all/arp_ignore</code>
<code> </code><code>echo</code> <code>1 > </code><code>/proc/sys/net/ipv4/conf/eth2/arp_ignore</code>
<code> </code><code>echo</code> <code>2 > </code><code>/proc/sys/net/ipv4/conf/all/arp_announce</code>
<code> </code><code>echo</code> <code>2 > </code><code>/proc/sys/net/ipv4/conf/eth2/arp_announce</code>
<code> </code><code>;;</code>
<code>stop)</code>
<code> </code><code>echo</code> <code>0 > </code><code>/proc/sys/net/ipv4/conf/all/arp_ignore</code>
<code> </code><code>echo</code> <code>0 > </code><code>/proc/sys/net/ipv4/conf/eth2/arp_ignore</code>
<code> </code><code>echo</code> <code>0 > </code><code>/proc/sys/net/ipv4/conf/all/arp_announce</code>
<code> </code><code>echo</code> <code>0 > </code><code>/proc/sys/net/ipv4/conf/eth2/arp_announce</code>
<code>esac</code>
<code>[root@jymlinux ~]</code><code># chmod +x lvsdrka.sh </code>
<code>[root@jymlinux ~]</code><code># ./lvsdrka.sh start</code>
<code>[root@jymlinux ~]</code><code># cat /proc/sys/net/ipv4/conf/all/arp_ignore </code>
<code>1</code>
<code>[root@jymlinux ~]</code><code># cat /proc/sys/net/ipv4/conf/all/arp_announce </code>
<code>2</code>
两个内核参数:
arp_announce:arp通告
0:通告全部IP 默认
1:尽量避免将非本网络的地址通告给网络中的其他地址
2:总是用最佳本地地址通告网络
arp_ignore:arp响应
0:无论从哪个接口请求的,只要主机有这个地址,就会响应 默认
1:从哪个接口请求的地址,就从哪个接口的地址响应
2,3,4,,5,6,7,8,有9个级别,其余不常用
在lvs-dr中,我们一般使用arp_ignore=1 arp_announce=2
测试web服务
<code>[root@localhost ~]</code><code># curl http://192.168.1.20</code>
<code><h1>this is node1 <\h1></code>
<code>[root@localhost ~]</code><code># curl http://192.168.1.21</code>
<code><h1>this is node2 <\h1></code>
配置集群规则
<code>[root@localhost ~]</code><code># ipvsadm -A -t 192.168.1.15:80 -s rr</code>
<code>[root@localhost ~]</code><code># ipvsadm -a -t 192.168.1.15:80 -r 192.168.1.20 -g</code>
<code>[root@localhost ~]</code><code># ipvsadm -a -t 192.168.1.15:80 -r 192.168.1.21 -g</code>
四、测试结果
<a href="http://s5.51cto.com/wyfs02/M02/89/E2/wKioL1ggh4jT7tizAAAy1T7XclI572.png-wh_500x0-wm_3-wmp_4-s_801917980.png" target="_blank"></a>
<a href="http://s5.51cto.com/wyfs02/M01/89/E5/wKiom1ggh4jggqR7AAA4D2ttTNo076.png-wh_500x0-wm_3-wmp_4-s_3797309380.png" target="_blank"></a>
五、配置http与https双集群服务
此处网段改为192.168.3.0网段。
<code>1、创建私有CA(以Director主机为例)</code>
<code>[root@localhost ~]</code><code># cd /etc/pki/CA</code>
<code>[root@localhost CA]</code><code># (umask 077; openssl genrsa -out private/cakey.pem 2048)</code>
<code>Generating RSA private key, 2048 bit long modulus</code>
<code>..................+++</code>
<code>.......+++</code>
<code>e is 65537 (0x10001)</code>
<code>[root@localhost CA]</code><code># touch index.txt</code>
<code>[root@localhost CA]</code><code># echo 01 > serial</code>
<code>[root@localhost CA]</code><code># openssl req -new -x509 -key private/cakey.pem -out cacert.pem -days 365</code>
<code>You are about to be asked to enter information that will be incorporated</code>
<code>into your certificate request.</code>
<code>What you are about to enter is what is called a Distinguished Name or a DN.</code>
<code>There are quite a few fields but you can leave some blank</code>
<code>For some fields there will be a default value,</code>
<code>If you enter </code><code>'.'</code><code>, the field will be left blank.</code>
<code>-----</code>
<code>Country Name (2 letter code) [XX]:CN</code>
<code>State or Province Name (full name) []:XJ</code>
<code>Locality Name (eg, city) [Default City]:XJ</code>
<code>Organization Name (eg, company) [Default Company Ltd]:JJ</code>
<code>Organizational Unit Name (eg, section) []:Ops</code>
<code>Common Name (eg, your name or your server's </code><code>hostname</code><code>) []:CA</code>
<code>Email Address []:ca.admin.com</code>
<code>2、RS主机申请证书</code>
<code>[root@jymlinux ~]</code><code># cd /etc/httpd/</code>
<code>[root@jymlinux httpd]</code><code># mkdir ssl</code>
<code>[root@jymlinux httpd]</code><code># cd ssl</code>
<code>[root@jymlinux ssl]</code><code># (umask 077; openssl genrsa -out httpd.key 2048)</code>
<code>..............+++</code>
<code>..................................+++</code>
<code>[root@jymlinux ssl]</code><code># openssl req -new -key httpd.key -out httpd.csr</code>
<code>Email Address []:rs1.admin.com</code>
<code>Please enter the following </code><code>'extra'</code> <code>attributes</code>
<code>to be sent with your certificate request</code>
<code>A challenge password []:</code>
<code>An optional company name []:</code>
<code>3、将申请的证书发往CA</code>
<code>[root@jymlinux ssl]</code><code># scp httpd.csr [email protected]:/root</code>
<code>The authenticity of host </code><code>'192.168.3.10 (192.168.3.10)'</code> <code>can't be established.</code>
<code>RSA key fingerprint is ef:85:f8:aa:1c:de:41:5a:fd:93:8d:9f:83:f7:a2:ff.</code>
<code>Are you sure you want to </code><code>continue</code> <code>connecting (</code><code>yes</code><code>/no</code><code>)? y</code>
<code>Please </code><code>type</code> <code>'yes'</code> <code>or </code><code>'no'</code><code>: </code><code>yes</code>
<code>Warning: Permanently added </code><code>'192.168.3.10'</code> <code>(RSA) to the list of known hosts.</code>
<code>[email protected]'s password: </code>
<code>httpd.csr 100% 1013 1.0KB</code><code>/s</code> <code>00:00 </code>
<code>4、CA签署并发证</code>
<code>[root@localhost CA]</code><code># openssl ca -in /root/httpd.csr -out /root/httpd.crt</code>
<code>Using configuration from </code><code>/etc/pki/tls/openssl</code><code>.cnf</code>
<code>Check that the request matches the signature</code>
<code>Signature ok</code>
<code>Certificate Details:</code>
<code> </code><code>Serial Number: 1 (0x1)</code>
<code> </code><code>Validity</code>
<code> </code><code>Not Before: Nov 9 13:48:21 2016 GMT</code>
<code> </code><code>Not After : Nov 9 13:48:21 2017 GMT</code>
<code> </code><code>Subject:</code>
<code> </code><code>countryName = CN</code>
<code> </code><code>stateOrProvinceName = XJ</code>
<code> </code><code>organizationName = JJ</code>
<code> </code><code>organizationalUnitName = Ops</code>
<code> </code><code>commonName = CA</code>
<code> </code><code>emailAddress = rs1.admin.com</code>
<code> </code><code>X509v3 extensions:</code>
<code> </code><code>X509v3 Basic Constraints: </code>
<code> </code><code>CA:FALSE</code>
<code> </code><code>Netscape Comment: </code>
<code> </code><code>OpenSSL Generated Certificate</code>
<code> </code><code>X509v3 Subject Key Identifier: </code>
<code> </code><code>7E:FA:3A:6F:89:28:EF:D1:CF:5C:42:75:50:7B:C6:99:1D:98:91:B6</code>
<code> </code><code>X509v3 Authority Key Identifier: </code>
<code> </code><code>keyid:91:9D:0E:8E:86:45:09:DE:C3:3F:63:61:C2:3D:CB:E1:E3:1C:F1:B6</code>
<code>Certificate is to be certified </code><code>until</code> <code>Nov 9 13:48:21 2017 GMT (365 days)</code>
<code>Sign the certificate? [y</code><code>/n</code><code>]:y</code>
<code>1 out of 1 certificate requests certified, commit? [y</code><code>/n</code><code>]y</code>
<code>Write out database with 1 new entries</code>
<code>Data Base Updated</code>
<code>[root@localhost CA]</code><code># scp /root/httpd.crt [email protected]:/etc/httpd/ssl/</code>
<code>The authenticity of host </code><code>'192.168.3.20 (192.168.3.20)'</code> <code>can't be established.</code>
<code>RSA key fingerprint is e5:84:6c:f7:c0:60:3d:0b:39:b6:1e:12:0d:48:8b:07.</code>
<code>Are you sure you want to </code><code>continue</code> <code>connecting (</code><code>yes</code><code>/no</code><code>)? </code><code>yes</code>
<code>Warning: Permanently added </code><code>'192.168.3.20'</code> <code>(RSA) to the list of known hosts.</code>
<code>[email protected]'s password: </code>
<code>httpd.crt 100% 4482 4.4KB</code><code>/s</code> <code>00:00 </code>
<code>4、安装mod-ssl</code>
<code>[root@jymlinux ~]</code><code># yum install mod_ssl</code>
<code>5、修改ssl的配置文件</code>
<code>[root@jymlinux ~]</code><code># cd /etc/httpd/conf.d/</code>
<code>[root@jymlinux conf.d]</code><code># vim ssl.conf </code>
<code>DocumentRoot </code><code>"/var/www/html"</code> <code>#启用</code>
<code>SSLCertificateFile </code><code>/etc/httpd/ssl/httpd</code><code>.crt </code><code>#修改证书以及密钥的所在路径</code>
<code>SSLCertificateKeyFile </code><code>/etc/httpd/ssl/httpd</code><code>.key</code>
<code>6、重启httpd服务</code>
<code>[root@jymlinux conf.d]</code><code># service httpd restart</code>
<code>7、Director配置ipvsadm规则</code>
<code>#使用iptables在PREROUTING链上的MARK标记将http与https标记为一组</code>
<code>[root@localhost ~]</code><code># iptables -t mangle -A PREROUTING -d 192.168.3.15 -p tcp --dport 80 -j MARK --set-mark 10</code>
<code>[root@localhost ~]</code><code># iptables -t mangle -A PREROUTING -d 192.168.3.15 -p tcp --dport 443 -j MARK --set-mark 10</code>
<code>[root@localhost ~]</code><code># ipvsadm -A -f 10 -s rr</code>
<code>[root@localhost ~]</code><code># ipvsadm -a -f 10 -r 192.168.3.20 -g</code>
<code>[root@localhost ~]</code><code># ipvsadm -a -f 10 -r 192.168.3.21 -g</code>
六、测试
<a href="http://s2.51cto.com/wyfs02/M01/89/FC/wKioL1gjMgCBvZvVAAAf_pegwN4555.png-wh_500x0-wm_3-wmp_4-s_3616437481.png" target="_blank"></a>
<a href="http://s3.51cto.com/wyfs02/M01/89/FF/wKiom1gjMgHRo46jAAAihsogkxQ439.png-wh_500x0-wm_3-wmp_4-s_2983972279.png" target="_blank"></a>
本文转自 元婴期 51CTO博客,原文链接:http://blog.51cto.com/jiayimeng/1870428