一、實驗拓撲圖：
二、實驗介紹
Corosync+pacemaker結合起來可以提供更強大的，功能更新的，維護狀态更活躍的高可用叢集，本文的主要
就是利用Corosync+pacemaker實作LVS的director的高可用。如上圖實驗拓撲圖所示：RS1、RS1和Dr1夠成一個負載均
衡叢集，為了實作此叢集的高可用，又将此叢集的Director即：Dr1與Dr2做成高可用叢集。
三、實驗前提：
（1）将Dr1和Dr2配置成雙機互信。并讓這四台伺服器的時間同步
（2）将RS1、RS2和Dr2配成LVS-DR模型的叢集，配好後讓ipvsadm開機不能自動啟動，并用#ipvsadm -C清空
定義的規則；并在Dr2中安裝ipvsadm。（此過程可參照： Heartbeat v2+heartbeat-ldirectord實作LVS(DR)中Director的高可用此博文）
四、實驗步驟：
完成以上步驟之後，就可以開始進行如下操作：
Dr1：
（1）下載下傳軟體包并安裝：
cluster-glue-1.0.6-1.6.el5.i386.rpm
cluster-glue-libs-1.0.6-1.6.el5.i386.rpm
（這兩個軟體包是Corosync和pacemaker的中間層，即使Corosync和pacemaker互相相容）
corosync-1.2.7-1.1.el5.i386.rpm
corosynclib-1.2.7-1.1.el5.i386.rpm
（這不做解釋，相信大家都明白O(∩_∩)O哈！）
heartbeat-3.0.3-2.3.el5.i386.rpm
heartbeat-libs-3.0.3-2.3.el5.i386.rpm
（以上兩個是pacemaker所依賴的軟體包，必須得安裝）
ldirectord-1.0.1-1.el5.i386.rpm
perl-MailTools-1.77-1.el5.noarch.rpm
perl-TimeDate-1.16-5.el5.noarch.rpm
（ldirectord可以實作Director的高可用，并可以檢查RS的健康狀況，其餘兩個都是其所依賴的軟體包）
openais-1.1.3-1.6.el5.i386.rpm
openaislib-1.1.3-1.6.el5.i386.rpm
（以上兩個可以安裝也可不安裝）
pacemaker-1.1.5-1.1.el5.i386.rpm
pacemaker-cts-1.1.5-1.1.el5.i386.rpm
pacemaker-libs-1.1.5-1.1.el5.i386.rpm
resource-agents-1.0.4-1.1.el5.i386.rpm（資源代理）
libesmtp-1.0.4-5.el5.i386.rpm
#yum -y --nogpgcheck localinstall *.rpm
注意同時需要在Dr2中也安裝相同的軟體包
（2）将corosync和pacemaker整合
- #cd /etc/corosync/
- #cp corosync.conf.example corosync.conf
- #vim corosync.conf
- totem {
- version: 2
- secauth: on
- threads: 0
- interfae {
- ringnumber: 0
- bindnetaddr: 172.16.0.0 //寫出需綁定位址的網段
- mcastaddr: 226.94.58.1 //多點傳播位址
- mcastport: 5405
- }
- }
- logging {
- fileline: off
- to_stderr: no
- to_logfile: yes
- #to_syslog: yes
- logfile: /var/log/cluster/corosync.log
- debug: off
- timestamp: on
- logger_subsys {
- subsys: AMF
- debug: off
- }
- }
- service {
- ver: 0
- name: pacemaker
- }
- aisexec {
- user: root
- group: root
- }
- amf {
- mode: disabled
- }
- #corosync-keygen //建立認證檔案
- #mkdir /var/log/cluster //建立日志檔案目錄
- #scp corosync.conf node2:/etc/corosync/
- #ssh node2 'mkdir /var/log/cluster'
（3）啟動服務并檢查其是否正常
#service corosync start
1、 驗證 corosync引擎是否啟動成功
- # grep -e 'Corosync Cluster Engine' -e 'configuration' /var/log/cluster/corosync.log
- Aug 10 11:05:59 corosync [MAIN ] Corosync Cluster Engine ('1.2.7'): started and ready to provide service.
- Aug 10 11:05:59 corosync [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'.
2、檢視初始化成員節點通知是否正常出發
- # grep TOTEM /var/log/cluster/corosync.log
- Aug 10 11:05:59 corosync [TOTEM ] Initializing transport (UDP/IP).
- Aug 10 11:05:59 corosync [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
- Aug 10 11:05:59 corosync [TOTEM ] The network interface [172.16.66.1] is now up.
- Aug 10 11:05:59 corosync [TOTEM ] Process pause detected for 525 ms, flushing membership messages.
- Aug 10 11:05:59 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
3、檢視啟動過程中是否有錯誤産生
- # grep ERROR /var/log/cluster/corosync.log
- Aug 10 11:07:02 node1 pengine: [22023]: ERROR: unpack_resources: Resource start-up disabled since no STONITH resources have been defined
- Aug 10 11:07:02 node1 pengine: [22023]: ERROR: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option
- Aug 10 11:07:02 node1 pengine: [22023]: ERROR: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity
- 若出現以上錯誤時由于未安裝STONITH裝置而造成的可以忽略
4、檢視pacemaker是否啟動
- # grep pcmk_startup /var/log/cluster/corosync.log
- Aug 10 11:05:59 corosync [pcmk ] info: pcmk_startup: CRM: Initialized
- Aug 10 11:05:59 corosync [pcmk ] Logging: Initialized pcmk_startup
- Aug 10 11:05:59 corosync [pcmk ] info: pcmk_startup: Maximum core file size is: 4294967295
- Aug 10 11:05:59 corosync [pcmk ] info: pcmk_startup: Service: 9
- Aug 10 11:05:59 corosync [pcmk ] info: pcmk_startup: Local hostname: node1
- # ssh node2 'service corosync start'(啟動node2的corosync，并檢查其是否成功，步驟同上)
（4）配置ldirectord
- # cp /usr/share/doc/heartbeat-ldirectord-2.1.4/ldirectord.cf /etc/ha.d/
- # vim /etc/ha.d/ldirectord.cf
- virtual=172.16.5.1:80
- real=172.16.44.1:80 gate
- real=172.16.55.1:80 gate
- fallback=127.0.0.1:80 gate
- service=http
- scheduler=rr
- #persistent=600
- #netmask=255.255.255.255
- protocol=tcp
- checktype=negotiate
- checkport=80
- request=".test.html"
- receive="OK"
- virtualhost=www.x.y.z
- # scp /etc/ha.d/ldirectord.cf node2:/etc/ha.d/
（5）配置資源
配置資源之前，先配置叢集的參數：
- （1）禁用stonith
- #crm(live)# configure
- crm(live)configure# property stonith-enabled=false
- crm(live)configure# property no-quorum-policy=ignore
- crm(live)configure# commit
- crm(live)configure# exit
- （2）crm(live)configure# property no-quorum-policy=ignore（忽略政策）
- crm(live)configure# commit
- crm(live)configure# exit
- 這樣的話可以當一個節點的服務停掉是，可以轉到另一個節點
資源的配置：
- # crm
- crm(live)# configure
- crm(live)configure# primitive VIP ocf:heartbeat:IPaddr params ip=172.16.5.1
- nic=eth0:0 cidr_netmask=255.255.255.255 broadcast=172.16.5.1 //配置VIP資源
- crm(live)configure# primitive LVS lsb:ldirectord //配置ldirectord資源
- crm(live)configure# colocation VIP_with_LVS inf: VIP LVS //定義排列限制，使VIP和LVS兩個資源必須同時在一個節點上
- crm(live)configure# location conn1 VIP 100: director2 //定義位置限制，使VIP資源更傾向于運作與director2節點
- crm(live)configure# commit //送出配置
- crm(live)configure# exit //退出
五、實驗結果測試：
（1）剛配完之後檢視資源是否生效
- （1）檢視狀态：
- root@node1 corosync]# crm_mon
- Connection to the CIB terminated
- Reconnecting...[root@node1 corosync]# crm_mon -1
- ============
- Last updated: Fri Aug 10 16:24:54 2012
- Stack: openais
- Current DC: node1 - partition with quorum
- Version: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
- 2 Nodes configured, 2 expected votes
- 2 Resources configured.
- ============
- Online: [ node1 node2 ]
- VIP (ocf::heartbeat:IPaddr): Started node2
- LVS (lsb:ldirectord): Started node2
- （2）檢視資源：
- [root@node1 corosync]#ssh node2 'ifconfig' （檢視vip是否生效）
- eth0 Link encap:Ethernet HWaddr 00:0C:29:10:55:33
- inet addr:172.16.77.1 Bcast:172.16.255.255 Mask:255.255.0.0
- UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
- RX packets:866240 errors:0 dropped:0 overruns:0 frame:0
- TX packets:168454 errors:0 dropped:0 overruns:0 carrier:0
- collisions:0 txqueuelen:1000
- RX bytes:186781842 (178.1 MiB) TX bytes:22115957 (21.0 MiB)
- Interrupt:67 Base address:0x2000
- eth0:0 Link encap:Ethernet HWaddr 00:0C:29:10:55:33
- inet addr:172.16.5.1 Bcast:172.16.5.1 Mask:255.255.255.255
- UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
- Interrupt:67 Base address:0x2000
- lo Link encap:Local Loopback
- inet addr:127.0.0.1 Mask:255.0.0.0
- UP LOOPBACK RUNNING MTU:16436 Metric:1
- RX packets:6880 errors:0 dropped:0 overruns:0 frame:0
- TX packets:6880 errors:0 dropped:0 overruns:0 carrier:0
- collisions:0 txqueuelen:0
- RX bytes:902177 (881.0 KiB) TX bytes:902177 (881.0 KiB)
- [root@node1 corosync]# ipvsadm -Ln （檢視ldirectord是否生效）
- IP Virtual Server version 1.2.1 (size=4096)
- Prot LocalAddress:Port Scheduler Flags
- -> RemoteAddress:Port Forward Weight ActiveConn InActConn
- TCP 172.16.5.1:80 rr
- -> 172.16.55.1:80 Route 1 0 0
- -> 172.16.44.1:80 Route 1 0 0
（2）測試效果，在浏覽其中通路10次，檢視結果：
- root@node1 corosync]# ipvsadm -Ln
- IP Virtual Server version 1.2.1 (size=4096)
- Prot LocalAddress:Port Scheduler Flags
- -> RemoteAddress:Port Forward Weight ActiveConn InActConn
- TCP 172.16.5.1:80 rr
- -> 172.16.55.1:80 Route 1 0 5
- -> 172.16.44.1:80 Route 1 0 5
- 這就是負載均衡的結果
- [root@node1 corosync]# ipvsadm -Ln
- IP Virtual Server version 1.2.1 (size=4096)
- Prot LocalAddress:Port Scheduler Flags
- -> RemoteAddress:Port Forward Weight ActiveConn InActConn
- TCP 172.16.5.1:80 rr
- -> 172.16.55.1:80 Route 1 0 11
- 隻有一個RS工作了，這就是ldirectord的效果
- 讓node2 standby看看是否轉移到node1上
- [root@node2 corosync]# ssh node1 'ifconfig'
- eth0 Link encap:Ethernet HWaddr 00:0C:29:C1:CB:91
- inet addr:172.16.66.1 Bcast:172.16.255.255 Mask:255.255.0.0
- UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
- RX packets:235799 errors:0 dropped:0 overruns:0 frame:0
- TX packets:218981 errors:0 dropped:0 overruns:0 carrier:0
- collisions:0 txqueuelen:1000
- RX bytes:83322431 (79.4 MiB) TX bytes:39479951 (37.6 MiB)
- Interrupt:67 Base address:0x2000
- eth0:0 Link encap:Ethernet HWaddr 00:0C:29:C1:CB:91
- inet addr:172.16.5.1 Bcast:172.16.5.1 Mask:255.255.255.255
- UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
- Interrupt:67 Base address:0x2000
- lo Link encap:Local Loopback
- inet addr:127.0.0.1 Mask:255.0.0.0
- UP LOOPBACK RUNNING MTU:16436 Metric:1
- RX packets:11597 errors:0 dropped:0 overruns:0 frame:0
- TX packets:11597 errors:0 dropped:0 overruns:0 carrier:0
- collisions:0 txqueuelen:0
- RX bytes:1565341 (1.4 MiB) TX bytes:1565341 (1.4 MiB)
- [root@node1 corosync]# ipvsadm -Ln
- IP Virtual Server version 1.2.1 (size=4096)
- Prot LocalAddress:Port Scheduler Flags
- -> RemoteAddress:Port Forward Weight ActiveConn InActConn
- TCP 172.16.5.1:80 rr
- -> 172.16.44.1:80 Route 1 0 0
- -> 172.16.55.1:80 Route 1 0 11
![RocketMQ高可用叢集搭建詳解[圖]](data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACwAAAAAAQABAAACAkQBADs=)