RAC添加節點
概述
添加節點主要分成4個步驟走:
1, 準備好主機環境
2, GI軟體的擴充。(在擴充之前可以做一個安裝條件是否滿足的檢查)
3, Database軟體的擴充
4, Instance的擴充
在第2,3,4中用到的指令也很簡單,單有一個前提是第一步準備的linux是完全符合要求的
檢查db3是否滿足rac安裝條件 su - grid cluvfy stage -pre nodeadd -n db3 -fixup -verbose cluvfy stage -post hwos -n db3 開始擴充叢集軟體 su - grid cd /u01/app/11.2.0/grid/oui/bin ./addNode.sh -silent "CLUSTER_NEW_NODES={db3}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={db3-vip}" 運作提示的root.sh腳本 /u01/grid/oraInventory/orainstRoot.sh #On nodes db3 /u01/grid/crs/root.sh #On nodes db3 驗證叢集軟體擴充成功 cluvfy stage -post nodeadd -n db3 -verbose 為新節點安裝資料庫軟體 su - oracle cd /u01/app/oracle/product/11.2.0/db/oui/bin ./addNode.sh -silent "CLUSTER_NEW_NODES={db3}" 運作提示的root.sh腳本 /u01/oracle/db/root.sh #On nodes db3 添加執行個體 dbca 或用指令行直接添加 dbca -silent -addInstance -nodeList db3 -gdbName db -instanceName db3 -sysDBAUserName sys -sysDBAPassword "oracle" grid使用者檢查狀态 crsctl status resource -t |
接下來示範具體的過程
準備Linux環境
首先克隆出一台幹淨的虛拟機
給虛拟機配置網絡
192.168.1.161 db1.up.com db1 192.168.1.162 db2.up.com db2 192.168.1.173 db3.up.com db3 10.0.1.161 db1-priv.up.com db1-priv 10.0.1.162 db2-priv.up.com db2-priv 10.0.1.173 db3-priv.up.com db3-priv 192.168.1.163 db1-vip.up.com db1-vip 192.168.1.164 db2-vip.up.com db2-vip 192.168.1.174 db32-vip.up.com db3-vip 192.168.1.165 db-cluster |
按照這種網絡規劃給db3節點的eth0, eth1分别配置網絡
192.168.1.173
10.0.1.173
設定主機名
[[email protected] ~]# cat /etc/sysconfig/network NETWORKING=yes NETWORKING_IPV6=yes HOSTNAME=db3.up.com [[email protected] ~]# hostname db3.up.com |
修改linux版本
[[email protected] ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 4.8 (Tikanga) |
修改hosts檔案配置
[[email protected] ~]# cat /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 odd.up.com odd localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 192.168.1.161 db1.up.com db1 192.168.1.162 db2.up.com db2 192.168.1.173 db3.up.com db3 10.0.1.161 db1-priv.up.com db1-priv 10.0.1.162 db2-priv.up.com db2-priv 10.0.1.173 db3-priv.up.com db3-priv 192.168.1.163 db1-vip.up.com db1-vip 192.168.1.164 db2-vip.up.com db2-vip 192.168.1.174 db3-vip.up.com db3-vip 192.168.1.165 db-cluster |
配置yum源
[[email protected] ~]# mkdir /iso [[email protected] ~]# mount -t iso9660 -o loop /mnt/share/LinuxSoftware/OracleLinux-R5-U8-Server-x86_64-dvd.iso /iso [[email protected] ~]# vim /etc/yum.repos.d/oel.repo [[email protected] ~]# cat /etc/yum.repos.d/oel.repo [source] name=Oracle Enterprise Linux $releasever - Source baseurl=file:///iso/Server enabled=1 gpgcheck=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release |
因為在裝rac的時候,系統提示一些包的缺失,需要安裝,是以,在這裡我們先安裝上這些包
[[email protected] ~]# yum install libaio-devel sysstat unixODBC unixODBC-devel
移走ntp配置
[[email protected] ~]# mv /etc/ntp.conf /etc/ntp.conf.bak
添加使用者
groupadd -g 1000 oinstall groupadd -g 1001 asmadmin groupadd -g 1002 dba groupadd -g 1003 oper groupadd -g 1004 asmdba groupadd -g 1005 asmoper useradd -u 1000 -g oinstall -G dba,oper,asmdba oracle useradd -u 1001 -g oinstall -G asmadmin,asmdba,asmoper grid passwd oracle passwd grid |
添加檔案夾
mkdir -p /u01/app/11.2.0/grid mkdir -p /u01/app/grid mkdir -p /u01/app/oracle chown -R grid:oinstall /u01/ chown -R oracle:oinstall /u01/app/oracle chmod -R 775 /u01/ |
配置udev裝置
這裡就直接将節點1的檔案拷貝過來就可以了 [[email protected] ~]# cat /etc/udev/rules.d/99-oracelasm.rules KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB0e123a72-41df0e11_", NAME="asm-diskb", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB840e50f3-375627c3_", NAME="asm-diskc", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBb8d6b677-65c5fa86_", NAME="asm-diskd", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBfc5cfb52-b0c7ce00_", NAME="asm-diske", OWNER="grid", GROUP="asmadmin", MODE="0660" [[email protected] ~]# ll /dev/asm* ls: /dev/asm*: No such file or directory [[email protected] ~]# start_udev Starting udev: [ OK ] [[email protected] ~]# ll /dev/asm* brw-rw---- 1 grid asmadmin 8, 16 Mar 6 15:15 /dev/asm-diskb brw-rw---- 1 grid asmadmin 8, 32 Mar 6 15:15 /dev/asm-diskc brw-rw---- 1 grid asmadmin 8, 48 Mar 6 15:15 /dev/asm-diskd brw-rw---- 1 grid asmadmin 8, 64 Mar 6 15:15 /dev/asm-dis |
将start_udev添加到rc.local裡
[[email protected] ~]# cat /etc/rc.local #!/bin/sh # # This script will be executed *after* all the other init scripts. # You can put your own initialization stuff in here if you don't # want to do the full Sys V style init stuff. touch /var/lock/subsys/local mount -t vboxsf Share /mnt/share start_ude |
編輯核心檔案/etc/sysctl.conf (兩個節點執行)
kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 fs.file-max = 6815744 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 1048576 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576 fs.aio-max-nr = 1048576 |
添加以上内容,然後執行sysctl -p指令加載這些核心參數
修改資源限制(兩個節點執行)
/etc/security/limits.conf 添加如下内容,添加如下内容
grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 |
編輯grid使用者的 .bash_profile檔案
export ORACLE_SID=+ASM3 export ORACLE_BASE=/u01/app/grid export ORACLE_HOME=/u01/app/11.2.0/grid export PATH=$ORACLE_HOME/bin:$PATH |
編輯oracle使用者的.bash_profile檔案
export ORACLE_SID=db3 export ORACLE_BASE=/u01/app/oracle export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db export PATH=$ORACLE_HOME/bin:$PAT |
還有一個很重要的工作,就是手工配置節點2,與1,2,的互通,這個很重要。
在節點3上操作 先切到grid使用者下。 mkdir .ssh chmod 755 .ssh ssh-keygen -t rsa 這樣才/home/grid/.ssh/目錄下就有一個id_rsa.pub這個檔案。将裡面的一行内容拷貝到節點1的/home/grid/.ssh/authorized_keys這個檔案裡,然後将這個檔案分别拷到節點2,3上。 [[email protected] .ssh]$ scp authorized_keys db3:/home/grid/.ssh [[email protected] .ssh]$ scp authorized_keys db2:/home/grid/.ssh 這樣在每個節點的grid使用者下執行 ssh db1 date ssh db2 date ssh db3 date ssh db1-priv date ssh db2-priv date ssh db3-priv date [[email protected] ~]$ ssh db1 date Thu Mar 6 16:29:51 CST 2014 [[email protected] ~]$ ssh db2 date Thu Mar 6 16:29:51 CST 2014 [[email protected] ~]$ ssh db3 date Thu Mar 6 16:29:51 CST 2014 [[email protected] ~]$ ssh db1-priv date Thu Mar 6 16:29:51 CST 2014 [[email protected] ~]$ ssh db2-priv date Thu Mar 6 16:29:52 CST 2014 [[email protected] ~]$ ssh db3-priv date Thu Mar 6 16:29:52 CST 2014 如果是第一次執行會讓你輸入密碼,直到執行這6個指令都不需要輸入密碼,則表示grid使用者的ssh互通配置完畢。 用同樣的方法配置oracle的互通 |
至此準備工作基本做完了。
GI擴充
在操作之前,確定所有節點都是正常狀态。
[[email protected] ~]$ crs_stat -t -v Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------- ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE db1 ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE db1 ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE db1 ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE db1 ora.db.db ora....se.type 0/2 0/1 OFFLINE OFFLINE ora....SM1.asm application 0/5 0/0 ONLINE ONLINE db1 ora....B1.lsnr application 0/5 0/0 ONLINE ONLINE db1 ora.db1.gsd application 0/5 0/0 OFFLINE OFFLINE ora.db1.ons application 0/3 0/0 ONLINE ONLINE db1 ora.db1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE db1 ora....SM2.asm application 0/5 0/0 ONLINE ONLINE db2 ora....B2.lsnr application 0/5 0/0 ONLINE ONLINE db2 ora.db2.gsd application 0/5 0/0 OFFLINE OFFLINE ora.db2.ons application 0/3 0/0 ONLINE ONLINE db2 ora.db2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE db2 ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE db1 ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE ora....network ora....rk.type 0/5 0/ ONLINE ONLINE db1 ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE db1 ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE db1 |
準備工作完成後,就可以根據新環境将新節點增加到RAC系統了,RAC增加節點總共分為三步進行,第一步就是将clusterware軟體系統進行擴充,将所有11g的clusterware軟體添加到新節點,第二步就是将ORACLE資料庫的軟體擴充到新節點,最後一步就是使用dbca建立新節點的資料庫執行個體instance。
先去節點3 ,安裝cvuqdisk包
[[email protected] ~]# rpm -ivh /mnt/share/oracle11g/64/linux.x64_11gR2_grid/grid/rpm/cvuqdisk-1.0.7-1.rpm |
檢查節點3是否滿足需求
[[email protected] ~]# su - grid [[email protected] ~]$ cluvfy stage -pre nodeadd -n db3 -fixup -verbose |
[[email protected] ~]$ cluvfy stage -pre nodeadd -n db3 -fixup -verbose Performing pre-checks for node addition Checking node reachability... Check: Node reachability from node "db1" Destination Node Reachable? ------------------------------------ ------------------------ db3 yes Result: Node reachability check passed from node "db1" Checking user equivalence... Check: User equivalence for user "grid" Node Name Comment ------------------------------------ ------------------------ db3 passed Result: User equivalence check passed for user "grid" Checking node connectivity... Checking hosts config file... Node Name Status Comment ------------ ------------------------ ------------------------ db3 passed db2 passed db1 passed Verification of the hosts config file successful Interface information for node "db1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.161 192.168.1.0 0.0.0.0 10.0.1.1 08:00:27:6C:37:49 1500 eth0 192.168.1.163 192.168.1.0 0.0.0.0 10.0.1.1 08:00:27:6C:37:49 1500 eth1 10.0.1.161 10.0.1.0 0.0.0.0 10.0.1.1 08:00:27:25:BF:57 1500 Interface information for node "db2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.162 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth0 192.168.1.165 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth0 192.168.1.164 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth1 10.0.1.162 10.0.1.0 0.0.0.0 192.168.1.1 08:00:27:9F:B3:8C 1500 Interface information for node "db3" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.173 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:CB:D9:A9 1500 eth1 10.0.1.173 10.0.1.0 0.0.0.0 192.168.1.1 08:00:27:FA:39:E3 1500 Check: Node connectivity for interface "eth0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- db1:eth0 db1:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db3:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db3:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db3:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db3:eth0 yes db2:eth0 db3:eth0 yes Result: Node connectivity passed for interface "eth0" Result: Node connectivity check passed Checking CRS integrity... The Oracle clusterware is healthy on node "db1" The Oracle clusterware is healthy on node "db2" CRS integrity check passed Checking shared resources... Checking CRS home location... The location "/u01/app/11.2.0/grid" is not shared but is present/creatable on all nodes Result: Shared resources check for node addition passed Checking node connectivity... Checking hosts config file... Node Name Status Comment ------------ ------------------------ ------------------------ db3 passed db2 passed db1 passed Verification of the hosts config file successful Interface information for node "db1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.161 192.168.1.0 0.0.0.0 10.0.1.1 08:00:27:6C:37:49 1500 eth0 192.168.1.163 192.168.1.0 0.0.0.0 10.0.1.1 08:00:27:6C:37:49 1500 eth1 10.0.1.161 10.0.1.0 0.0.0.0 10.0.1.1 08:00:27:25:BF:57 1500 Interface information for node "db2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.162 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth0 192.168.1.165 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth0 192.168.1.164 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth1 10.0.1.162 10.0.1.0 0.0.0.0 192.168.1.1 08:00:27:9F:B3:8C 1500 Interface information for node "db3" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.173 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:CB:D9:A9 1500 eth1 10.0.1.173 10.0.1.0 0.0.0.0 192.168.1.1 08:00:27:FA:39:E3 1500 Check: Node connectivity of subnet "192.168.1.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- db1:eth0 db1:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db3:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db2:eth0 yes db1:eth0 db3:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db3:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db3:eth0 yes db2:eth0 db3:eth0 yes Result: Node connectivity passed for subnet "192.168.1.0" with node(s) db1,db2,db3 Check: TCP connectivity of subnet "192.168.1.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- db1:192.168.1.161 db1:192.168.1.163 passed db1:192.168.1.161 db2:192.168.1.162 passed db1:192.168.1.161 db2:192.168.1.165 passed db1:192.168.1.161 db2:192.168.1.164 passed db1:192.168.1.161 db3:192.168.1.173 passed Result: TCP connectivity check passed for subnet "192.168.1.0" Check: Node connectivity of subnet "10.0.1.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- db1:eth1 db2:eth1 yes db1:eth1 db3:eth1 yes db2:eth1 db3:eth1 yes Result: Node connectivity passed for subnet "10.0.1.0" with node(s) db1,db2,db3 Check: TCP connectivity of subnet "10.0.1.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- db1:10.0.1.161 db2:10.0.1.162 passed db1:10.0.1.161 db3:10.0.1.173 passed Result: TCP connectivity check passed for subnet "10.0.1.0" WARNING: Could not find a suitable set of interfaces for VIPs WARNING: Could not find a suitable set of interfaces for the private interconnect Result: Node connectivity check passed Check: Total memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 1.96GB (2050664.0KB) 1.5GB (1572864.0KB) passed Result: Total memory check passed Check: Available memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 1.57GB (1651152.0KB) 50MB (51200.0KB) passed Result: Available memory check passed Check: Swap space Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 4.88GB (5116692.0KB) 2.93GB (3075996.0KB) passed Result: Swap space check passed Check: Free disk space for "db3:/tmp" Path Node Name Mount point Available Required Comment ---------------- ------------ ------------ ------------ ------------ ------------ /tmp db3 / 87.93GB 1GB passed Result: Free disk space check passed for "db3:/tmp" Check: User existence for "grid" Node Name Status Comment ------------ ------------------------ ------------------------ db3 exists passed Result: User existence check passed for "grid" Check: Run level Node Name run level Required Comment ------------ ------------------------ ------------------------ ---------- db3 5 3,5 passed Result: Run level check passed Check: Hard limits for "maximum open file descriptors" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- db3 hard 65536 65536 passed Result: Hard limits check passed for "maximum open file descriptors" Check: Soft limits for "maximum open file descriptors" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- db3 soft 1024 1024 passed Result: Soft limits check passed for "maximum open file descriptors" Check: Hard limits for "maximum user processes" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- db3 hard 16384 16384 passed Result: Hard limits check passed for "maximum user processes" Check: Soft limits for "maximum user processes" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- db3 soft 2047 2047 passed Result: Soft limits check passed for "maximum user processes" Check: System architecture Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 x86_64 x86_64 passed Result: System architecture check passed Check: Kernel version Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 2.6.32-300.10.1.el5uek 2.6.18 passed Result: Kernel version check passed Check: Kernel parameter for "semmsl" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 250 250 passed Result: Kernel parameter check passed for "semmsl" Check: Kernel parameter for "semmns" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 32000 32000 passed Result: Kernel parameter check passed for "semmns" Check: Kernel parameter for "semopm" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 32 100 failed Result: Kernel parameter check failed for "semopm" Check: Kernel parameter for "semmni" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 128 128 passed Result: Kernel parameter check passed for "semmni" Check: Kernel parameter for "shmmax" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 68719476736 536870912 passed Result: Kernel parameter check passed for "shmmax" Check: Kernel parameter for "shmmni" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 4096 4096 passed Result: Kernel parameter check passed for "shmmni" Check: Kernel parameter for "shmall" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 4294967296 2097152 passed Result: Kernel parameter check passed for "shmall" Check: Kernel parameter for "file-max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 199621 6815744 failed Result: Kernel parameter check failed for "file-max" Check: Kernel parameter for "ip_local_port_range" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 between 32768 & 61000 between 9000 & 65500 failed (ignorable) Result: Kernel parameter check passed for "ip_local_port_range" Check: Kernel parameter for "rmem_default" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 126976 262144 failed Result: Kernel parameter check failed for "rmem_default" Check: Kernel parameter for "rmem_max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 131071 4194304 failed Result: Kernel parameter check failed for "rmem_max" Check: Kernel parameter for "wmem_default" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 126976 262144 failed Result: Kernel parameter check failed for "wmem_default" Check: Kernel parameter for "wmem_max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 131071 1048576 failed Result: Kernel parameter check failed for "wmem_max" Check: Kernel parameter for "aio-max-nr" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- db3 65536 1048576 failed Result: Kernel parameter check failed for "aio-max-nr" Check: Package existence for "make-3.81" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 make-3.81-3.el5 make-3.81 passed Result: Package existence check passed for "make-3.81" Check: Package existence for "binutils-2.17.50.0.6" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 binutils-2.17.50.0.6-20.el5 binutils-2.17.50.0.6 passed Result: Package existence check passed for "binutils-2.17.50.0.6" Check: Package existence for "gcc-4.1.2" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 gcc-4.1.2-52.el5 gcc-4.1.2 passed Result: Package existence check passed for "gcc-4.1.2" Check: Package existence for "libaio-0.3.106 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 libaio-0.3.106-5 (i386) libaio-0.3.106 (i386) passed Result: Package existence check passed for "libaio-0.3.106 (i386)" Check: Package existence for "libaio-0.3.106 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 libaio-0.3.106-5 (x86_64) libaio-0.3.106 (x86_64) passed Result: Package existence check passed for "libaio-0.3.106 (x86_64)" Check: Package existence for "glibc-2.5-24 (i686)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 glibc-2.5-81 (i686) glibc-2.5-24 (i686) passed Result: Package existence check passed for "glibc-2.5-24 (i686)" Check: Package existence for "glibc-2.5-24 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 glibc-2.5-81 (x86_64) glibc-2.5-24 (x86_64) passed Result: Package existence check passed for "glibc-2.5-24 (x86_64)" Check: Package existence for "compat-libstdc++-33-3.2.3 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 compat-libstdc++-33-3.2.3-61 (i386) compat-libstdc++-33-3.2.3 (i386) passed Result: Package existence check passed for "compat-libstdc++-33-3.2.3 (i386)" Check: Package existence for "compat-libstdc++-33-3.2.3 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3 (x86_64) passed Result: Package existence check passed for "compat-libstdc++-33-3.2.3 (x86_64)" Check: Package existence for "elfutils-libelf-0.125 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 elfutils-libelf-0.137-3.el5 (x86_64) elfutils-libelf-0.125 (x86_64) passed Result: Package existence check passed for "elfutils-libelf-0.125 (x86_64)" Check: Package existence for "elfutils-libelf-devel-0.125" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125 passed Result: Package existence check passed for "elfutils-libelf-devel-0.125" Check: Package existence for "glibc-common-2.5" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 glibc-common-2.5-81 glibc-common-2.5 passed Result: Package existence check passed for "glibc-common-2.5" Check: Package existence for "glibc-devel-2.5 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 glibc-devel-2.5-81 (i386) glibc-devel-2.5 (i386) passed Result: Package existence check passed for "glibc-devel-2.5 (i386)" Check: Package existence for "glibc-devel-2.5 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 glibc-devel-2.5-81 (x86_64) glibc-devel-2.5 (x86_64) passed Result: Package existence check passed for "glibc-devel-2.5 (x86_64)" Check: Package existence for "glibc-headers-2.5" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 glibc-headers-2.5-81 glibc-headers-2.5 passed Result: Package existence check passed for "glibc-headers-2.5" Check: Package existence for "gcc-c++-4.1.2" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 gcc-c++-4.1.2-52.el5 gcc-c++-4.1.2 passed Result: Package existence check passed for "gcc-c++-4.1.2" Check: Package existence for "libaio-devel-0.3.106 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 libaio-devel-0.3.106-5 (i386) libaio-devel-0.3.106 (i386) passed Result: Package existence check passed for "libaio-devel-0.3.106 (i386)" Check: Package existence for "libaio-devel-0.3.106 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 libaio-devel-0.3.106-5 (x86_64) libaio-devel-0.3.106 (x86_64) passed Result: Package existence check passed for "libaio-devel-0.3.106 (x86_64)" Check: Package existence for "libgcc-4.1.2 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 libgcc-4.1.2-52.el5 (i386) libgcc-4.1.2 (i386) passed Result: Package existence check passed for "libgcc-4.1.2 (i386)" Check: Package existence for "libgcc-4.1.2 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 libgcc-4.1.2-52.el5 (x86_64) libgcc-4.1.2 (x86_64) passed Result: Package existence check passed for "libgcc-4.1.2 (x86_64)" Check: Package existence for "libstdc++-4.1.2 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 libstdc++-4.1.2-52.el5 (i386) libstdc++-4.1.2 (i386) passed Result: Package existence check passed for "libstdc++-4.1.2 (i386)" Check: Package existence for "libstdc++-4.1.2 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 libstdc++-4.1.2-52.el5 (x86_64) libstdc++-4.1.2 (x86_64) passed Result: Package existence check passed for "libstdc++-4.1.2 (x86_64)" Check: Package existence for "libstdc++-devel-4.1.2 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 libstdc++-devel-4.1.2-52.el5 (x86_64) libstdc++-devel-4.1.2 (x86_64) passed Result: Package existence check passed for "libstdc++-devel-4.1.2 (x86_64)" Check: Package existence for "sysstat-7.0.2" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 sysstat-7.0.2-11.el5 sysstat-7.0.2 passed Result: Package existence check passed for "sysstat-7.0.2" Check: Package existence for "unixODBC-2.2.11 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 unixODBC-2.2.11-10.el5 (i386) unixODBC-2.2.11 (i386) passed Result: Package existence check passed for "unixODBC-2.2.11 (i386)" Check: Package existence for "unixODBC-2.2.11 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 unixODBC-2.2.11-10.el5 (x86_64) unixODBC-2.2.11 (x86_64) passed Result: Package existence check passed for "unixODBC-2.2.11 (x86_64)" Check: Package existence for "unixODBC-devel-2.2.11 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 unixODBC-devel-2.2.11-10.el5 (i386) unixODBC-devel-2.2.11 (i386) passed Result: Package existence check passed for "unixODBC-devel-2.2.11 (i386)" Check: Package existence for "unixODBC-devel-2.2.11 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 unixODBC-devel-2.2.11-10.el5 (x86_64) unixODBC-devel-2.2.11 (x86_64) passed Result: Package existence check passed for "unixODBC-devel-2.2.11 (x86_64)" Check: Package existence for "ksh-20060214" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- db3 ksh-20100621-5.el5 ksh-20060214 passed Result: Package existence check passed for "ksh-20060214" Checking for multiple users with UID value 0 Result: Check for multiple users with UID value 0 passed Checking to make sure user "grid" is not in "root" group Node Name Status Comment ------------ ------------------------ ------------------------ db3 does not exist passed Result: User "grid" is not part of "root" group. Check passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes Result: Clock synchronization check using Network Time Protocol(NTP) passed Fixup information has been generated for following node(s): db3 Please run the following script on each node as "root" user to execute the fixups: '/tmp/CVU_11.2.0.1.0_grid/runfixup.sh' Pre-check for node addition was unsuccessful on all the nodes. |
在這個結果中,遇到軟體failed的,就用yum的方式給其裝上,其他的問題,網絡是通的,使用者是對等的,其他的failed就基本可以忽略掉。
[[email protected] ~]$ cluvfy stage -post hwos -n db3 Performing post-checks for hardware and operating system setup Checking node reachability... Node reachability check passed from node "db1" Checking user equivalence... User equivalence check passed for user "grid" Checking node connectivity... Checking hosts config file... Verification of the hosts config file successful Node connectivity passed for subnet "192.168.1.0" with node(s) db3 TCP connectivity check passed for subnet "192.168.1.0" Node connectivity passed for subnet "10.0.1.0" with node(s) db3 TCP connectivity check passed for subnet "10.0.1.0" Interfaces found on subnet "192.168.1.0" that are likely candidates for VIP are: db3 eth0:192.168.1.173 Interfaces found on subnet "10.0.1.0" that are likely candidates for a private interconnect are: db3 eth1:10.0.1.173 Node connectivity check passed Check for multiple users with UID value 0 passed Post-check for hardware and operating system setup was successful. |
這樣在節點1上做的,對節點3的檢查基本通過,可以開始添加操作了。
[[email protected] ~]$ cd /u01/app/11.2.0/grid/oui/bin/addNode.sh -silent "CLUSTER_NEW_NODES={db3}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={db3-vip}"-bash: cd: /u01/app/11.2.0/grid/oui/bin/addNode.sh: Not a directory [[email protected] ~]$ /u01/app/11.2.0/grid/oui/bin/addNode.sh -silent "CLUSTER_NEW_NODES={db3}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={db3-vip}" /u01/app/11.2.0/grid/oui/bin/addNode.sh: line 4: ./runInstaller: No such file or directory [[email protected] ~]$ cat /u01/app/11.2.0/grid/oui/bin/addNode.sh #!/bin/sh OHOME=/u01/app/11.2.0/grid INVPTRLOC=$OHOME/oraInst.loc ./runInstaller -addNode -invPtrLoc $INVPTRLOC ORACLE_HOME=$OHOME $* [[email protected] ~]$ cd /u01/app/11.2.0/grid/oui/bin/ [[email protected] bin]$./addNode.sh -silent "CLUSTER_NEW_NODES={db3}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={db3-vip}" Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 4996 MB Passed Oracle Universal Installer, Version 11.2.0.1.0 Production Copyright (C) 1999, 2009, Oracle. All rights reserved. Performing tests to see whether nodes db2,db3 are available ............................................................... 100% Done. ... ----------------------------------------------------------------------------- Cluster Node Addition Summary Global Settings Source: /u01/app/11.2.0/grid New Nodes Space Requirements New Nodes db3 /: Required 3.87GB : Available 81.88GB Installed Products Product Names Oracle Grid Infrastructure 11.2.0.1.0 Sun JDK 1.5.0.17.0 Installer SDK Component 11.2.0.1.0 Oracle One-Off Patch Installer 11.2.0.0.2 Oracle Universal Installer 11.2.0.1.0 Oracle Configuration Manager Deconfiguration 10.3.1.0.0 Enterprise Manager Common Core Files 10.2.0.4.2 Oracle DBCA Deconfiguration 11.2.0.1.0 Oracle RAC Deconfiguration 11.2.0.1.0 Oracle Quality of Service Management (Server) 11.2.0.1.0 Installation Plugin Files 11.2.0.1.0 Universal Storage Manager Files 11.2.0.1.0 Oracle Text Required Support Files 11.2.0.1.0 Automatic Storage Management Assistant 11.2.0.1.0 Oracle Database 11g Multimedia Files 11.2.0.1.0 Oracle Multimedia Java Advanced Imaging 11.2.0.1.0 Oracle Globalization Support 11.2.0.1.0 Oracle Multimedia Locator RDBMS Files 11.2.0.1.0 Oracle Core Required Support Files 11.2.0.1.0 Bali Share 1.1.18.0.0 Oracle Database Deconfiguration 11.2.0.1.0 Oracle Quality of Service Management (Client) 11.2.0.1.0 Expat libraries 2.0.1.0.1 Oracle Containers for Java 11.2.0.1.0 Perl Modules 5.10.0.0.1 Secure Socket Layer 11.2.0.1.0 Oracle JDBC/OCI Instant Client 11.2.0.1.0 Oracle Multimedia Client Option 11.2.0.1.0 LDAP Required Support Files 11.2.0.1.0 Character Set Migration Utility 11.2.0.1.0 Perl Interpreter 5.10.0.0.1 PL/SQL Embedded Gateway 11.2.0.1.0 OLAP SQL Scripts 11.2.0.1.0 Database SQL Scripts 11.2.0.1.0 Oracle Extended Windowing Toolkit 3.4.47.0.0 SSL Required Support Files for InstantClient 11.2.0.1.0 SQL*Plus Files for Instant Client 11.2.0.1.0 Oracle Net Required Support Files 11.2.0.1.0 Oracle Database User Interface 2.2.13.0.0 RDBMS Required Support Files for Instant Client 11.2.0.1.0 Enterprise Manager Minimal Integration 11.2.0.1.0 XML Parser for Java 11.2.0.1.0 Oracle Security Developer Tools 11.2.0.1.0 Oracle Wallet Manager 11.2.0.1.0 Enterprise Manager plugin Common Files 11.2.0.1.0 Platform Required Support Files 11.2.0.1.0 Oracle JFC Extended Windowing Toolkit 4.2.36.0.0 RDBMS Required Support Files 11.2.0.1.0 Oracle Ice Browser 5.2.3.6.0 Oracle Help For Java 4.2.9.0.0 Enterprise Manager Common Files 10.2.0.4.2 Deinstallation Tool 11.2.0.1.0 Oracle Java Client 11.2.0.1.0 Cluster Verification Utility Files 11.2.0.1.0 Oracle Notification Service (eONS) 11.2.0.1.0 Oracle LDAP administration 11.2.0.1.0 Cluster Verification Utility Common Files 11.2.0.1.0 Oracle Clusterware RDBMS Files 11.2.0.1.0 Oracle Locale Builder 11.2.0.1.0 Oracle Globalization Support 11.2.0.1.0 Buildtools Common Files 11.2.0.1.0 Oracle RAC Required Support Files-HAS 11.2.0.1.0 SQL*Plus Required Support Files 11.2.0.1.0 XDK Required Support Files 11.2.0.1.0 Agent Required Support Files 10.2.0.4.2 Parser Generator Required Support Files 11.2.0.1.0 Precompiler Required Support Files 11.2.0.1.0 Installation Common Files 11.2.0.1.0 Required Support Files 11.2.0.1.0 Oracle JDBC/THIN Interfaces 11.2.0.1.0 Oracle Multimedia Locator 11.2.0.1.0 Oracle Multimedia 11.2.0.1.0 HAS Common Files 11.2.0.1.0 Assistant Common Files 11.2.0.1.0 PL/SQL 11.2.0.1.0 HAS Files for DB 11.2.0.1.0 Oracle Recovery Manager 11.2.0.1.0 Oracle Database Utilities 11.2.0.1.0 Oracle Notification Service 11.2.0.0.0 SQL*Plus 11.2.0.1.0 Oracle Netca Client 11.2.0.1.0 Oracle Net 11.2.0.1.0 Oracle JVM 11.2.0.1.0 Oracle Internet Directory Client 11.2.0.1.0 Oracle Net Listener 11.2.0.1.0 Cluster Ready Services Files 11.2.0.1.0 Oracle Database 11g 11.2.0.1.0 Instantiating scripts for add node (Thursday, March 6, 2014 4:49:56 PM CST) . 1% Done. Instantiation of add node scripts complete Copying to remote nodes (Thursday, March 6, 2014 4:50:16 PM CST) ............................................................................................... 96% Done. Home copied to new nodes Saving inventory on nodes (Thursday, March 6, 2014 4:55:59 PM CST) . 100% Done. Save inventory complete WARNING:A new inventory has been created on one or more nodes in this session. However, it has not yet been registered as the central inventory of this system. To register the new inventory please run the script at '/u01/app/oraInventory/orainstRoot.sh' with root privileges on nodes 'db3'. If you do not register the inventory, you may not be able to update or patch the products you installed. The following configuration scripts need to be executed as the "root" user in each cluster node. /u01/app/oraInventory/orainstRoot.sh #On nodes db3 /u01/app/11.2.0/grid/root.sh #On nodes db3 To execute the configuration scripts: 1. Open a terminal window 2. Log in as "root" 3. Run the scripts in each cluster node The Cluster Node Addition of /u01/app/11.2.0/grid was successful. Please check '/tmp/silentInstall.log' for more details. |
接下來,就是使用root使用者,在需要添加的節點上執行,也就是節點3.
[[email protected] ~]# /u01/app/oraInventory/orainstRoot.sh Creating the Oracle inventory pointer file (/etc/oraInst.loc) Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete. [[email protected] ~]# /u01/app/11.2.0/grid/root.sh Running Oracle 11g root.sh script... The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed. 2014-03-06 17:05:21: Parsing the host name 2014-03-06 17:05:21: Checking for super user privileges 2014-03-06 17:05:21: User has super user privileges Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params Creating trace directory LOCAL ADD MODE Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Adding daemon to inittab CRS-4123: Oracle High Availability Services has been started. ohasd is starting acfsroot: ACFS-9301: ADVM/ACFS installation can not proceed: acfsroot: ACFS-9302: No installation files found at /u01/app/11.2.0/grid/install/usm/EL5/x86_64/2.6.18-8/2.6.18-8.el5uek-x86_64/bin. CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node db1, number 1, and is terminating An active cluster was found during exclusive startup, restarting to join the cluster CRS-2672: Attempting to start 'ora.mdnsd' on 'db3' CRS-2676: Start of 'ora.mdnsd' on 'db3' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'db3' CRS-2676: Start of 'ora.gipcd' on 'db3' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'db3' CRS-2676: Start of 'ora.gpnpd' on 'db3' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'db3' CRS-2676: Start of 'ora.cssdmonitor' on 'db3' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'db3' CRS-2672: Attempting to start 'ora.diskmon' on 'db3' CRS-2676: Start of 'ora.diskmon' on 'db3' succeeded CRS-2676: Start of 'ora.cssd' on 'db3' succeeded CRS-2672: Attempting to start 'ora.ctssd' on 'db3' CRS-2676: Start of 'ora.ctssd' on 'db3' succeeded CRS-2672: Attempting to start 'ora.asm' on 'db3' CRS-2676: Start of 'ora.asm' on 'db3' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'db3' CRS-2676: Start of 'ora.crsd' on 'db3' succeeded CRS-2672: Attempting to start 'ora.evmd' on 'db3' CRS-2676: Start of 'ora.evmd' on 'db3' succeeded clscfg: EXISTING configuration version 5 detected. clscfg: version 5 is 11g Release 2. Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. db3 2014/03/06 17:09:38 /u01/app/11.2.0/grid/cdata/db3/backup_20140306_170938.olr Configure Oracle Grid Infrastructure for a Cluster ... succeeded Updating inventory properties for clusterware Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 4996 MB Passed The inventory pointer is located at /etc/oraInst.loc The inventory is located at /u01/app/oraInventory 'UpdateNodeList' was successful. |
此時再節點1上可以看到節點3已經加到叢集裡了。
[[email protected] bin]$ crs_stat -t -v Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------- ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE db1 ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE db1 ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE db2 ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE db1 ora.db.db ora....se.type 0/2 0/1 OFFLINE OFFLINE ora....SM1.asm application 0/5 0/0 ONLINE ONLINE db1 ora....B1.lsnr application 0/5 0/0 ONLINE ONLINE db1 ora.db1.gsd application 0/5 0/0 OFFLINE OFFLINE ora.db1.ons application 0/3 0/0 ONLINE ONLINE db1 ora.db1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE db1 ora....SM2.asm application 0/5 0/0 ONLINE ONLINE db2 ora....B2.lsnr application 0/5 0/0 ONLINE ONLINE db2 ora.db2.gsd application 0/5 0/0 OFFLINE OFFLINE ora.db2.ons application 0/3 0/0 ONLINE ONLINE db2 ora.db2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE db2 ora....SM3.asm application 0/5 0/0 ONLINE ONLINE db3 ora....B3.lsnr application 0/5 0/0 ONLINE ONLINE db3 ora.db3.gsd application 0/5 0/0 OFFLINE OFFLINE ora.db3.ons application 0/3 0/0 ONLINE ONLINE db3 ora.db3.vip ora....t1.type 0/0 0/0 ONLINE ONLINE db3 ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE db1 ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE ora....network ora....rk.type 0/5 0/ ONLINE ONLINE db1 ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE db1 ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE db2 |
驗證叢集軟體擴充成功
[[email protected] bin]$cluvfy stage -post nodeadd -n db3 -verbose Performing post-checks for node addition Checking node reachability... Check: Node reachability from node "db1" Destination Node Reachable? ------------------------------------ ------------------------ db3 yes Result: Node reachability check passed from node "db1" Checking user equivalence... Check: User equivalence for user "grid" Node Name Comment ------------------------------------ ------------------------ db3 passed Result: User equivalence check passed for user "grid" Checking node connectivity... Checking hosts config file... Node Name Status Comment ------------ ------------------------ ------------------------ db3 passed db2 passed db1 passed Verification of the hosts config file successful Interface information for node "db3" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.173 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:CB:D9:A9 1500 eth0 192.168.1.174 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:CB:D9:A9 1500 eth1 10.0.1.173 10.0.1.0 0.0.0.0 192.168.1.1 08:00:27:FA:39:E3 1500 Interface information for node "db2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.162 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth0 192.168.1.165 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth0 192.168.1.164 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth1 10.0.1.162 10.0.1.0 0.0.0.0 192.168.1.1 08:00:27:9F:B3:8C 1500 Interface information for node "db1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.161 192.168.1.0 0.0.0.0 10.0.1.1 08:00:27:6C:37:49 1500 eth0 192.168.1.163 192.168.1.0 0.0.0.0 10.0.1.1 08:00:27:6C:37:49 1500 eth1 10.0.1.161 10.0.1.0 0.0.0.0 10.0.1.1 08:00:27:25:BF:57 1500 Check: Node connectivity for interface "eth0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- db3:eth0 db3:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db1:eth0 yes db3:eth0 db1:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db1:eth0 yes db3:eth0 db1:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db1:eth0 yes db2:eth0 db1:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db1:eth0 yes db2:eth0 db1:eth0 yes db2:eth0 db1:eth0 yes db2:eth0 db1:eth0 yes db1:eth0 db1:eth0 yes Result: Node connectivity passed for interface "eth0" Result: Node connectivity check passed Checking cluster integrity... Node Name ------------------------------------ db1 db2 db3 Cluster integrity check passed Checking CRS integrity... The Oracle clusterware is healthy on node "db3" The Oracle clusterware is healthy on node "db2" The Oracle clusterware is healthy on node "db1" CRS integrity check passed Checking shared resources... Checking CRS home location... The location "/u01/app/11.2.0/grid" is not shared but is present/creatable on all nodes Result: Shared resources check for node addition passed Checking node connectivity... Checking hosts config file... Node Name Status Comment ------------ ------------------------ ------------------------ db3 passed db2 passed db1 passed Verification of the hosts config file successful Interface information for node "db3" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.173 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:CB:D9:A9 1500 eth0 192.168.1.174 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:CB:D9:A9 1500 eth1 10.0.1.173 10.0.1.0 0.0.0.0 192.168.1.1 08:00:27:FA:39:E3 1500 Interface information for node "db2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.162 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth0 192.168.1.165 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth0 192.168.1.164 192.168.1.0 0.0.0.0 192.168.1.1 08:00:27:1D:22:8D 1500 eth1 10.0.1.162 10.0.1.0 0.0.0.0 192.168.1.1 08:00:27:9F:B3:8C 1500 Interface information for node "db1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.161 192.168.1.0 0.0.0.0 10.0.1.1 08:00:27:6C:37:49 1500 eth0 192.168.1.163 192.168.1.0 0.0.0.0 10.0.1.1 08:00:27:6C:37:49 1500 eth1 10.0.1.161 10.0.1.0 0.0.0.0 10.0.1.1 08:00:27:25:BF:57 1500 Check: Node connectivity of subnet "192.168.1.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- db3:eth0 db3:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db1:eth0 yes db3:eth0 db1:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db2:eth0 yes db3:eth0 db1:eth0 yes db3:eth0 db1:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db1:eth0 yes db2:eth0 db1:eth0 yes db2:eth0 db2:eth0 yes db2:eth0 db1:eth0 yes db2:eth0 db1:eth0 yes db2:eth0 db1:eth0 yes db2:eth0 db1:eth0 yes db1:eth0 db1:eth0 yes Result: Node connectivity passed for subnet "192.168.1.0" with node(s) db3,db2,db1 Check: TCP connectivity of subnet "192.168.1.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- db1:192.168.1.161 db3:192.168.1.173 passed db1:192.168.1.161 db3:192.168.1.174 passed db1:192.168.1.161 db2:192.168.1.162 passed db1:192.168.1.161 db2:192.168.1.165 passed db1:192.168.1.161 db2:192.168.1.164 passed db1:192.168.1.161 db1:192.168.1.163 passed Result: TCP connectivity check passed for subnet "192.168.1.0" Check: Node connectivity of subnet "10.0.1.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- db3:eth1 db2:eth1 yes db3:eth1 db1:eth1 yes db2:eth1 db1:eth1 yes Result: Node connectivity passed for subnet "10.0.1.0" with node(s) db3,db2,db1 Check: TCP connectivity of subnet "10.0.1.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- db1:10.0.1.161 db3:10.0.1.173 passed db1:10.0.1.161 db2:10.0.1.162 passed Result: TCP connectivity check passed for subnet "10.0.1.0" WARNING: Could not find a suitable set of interfaces for VIPs WARNING: Could not find a suitable set of interfaces for the private interconnect Result: Node connectivity check passed Checking node application existence... Checking existence of VIP node application Node Name Required Status Comment ------------ ------------------------ ------------------------ ---------- db3 yes online passed db2 yes online passed db1 yes online passed Result: Check passed. Checking existence of ONS node application Node Name Required Status Comment ------------ ------------------------ ------------------------ ---------- db3 no online passed db2 no online passed db1 no online passed Result: Check passed. Checking existence of GSD node application Node Name Required Status Comment ------------ ------------------------ ------------------------ ---------- db3 no does not exist ignored db2 no does not exist ignored db1 no does not exist ignored Result: Check ignored. Checking existence of EONS node application Node Name Required Status Comment ------------ ------------------------ ------------------------ ---------- db3 no online passed db2 no online passed db1 no online passed Result: Check passed. Checking existence of NETWORK node application Node Name Required Status Comment ------------ ------------------------ ------------------------ ---------- db3 no online passed db2 no online passed db1 no online passed Result: Check passed. Checking Single Client Access Name (SCAN)... SCAN VIP name Node Running? ListenerName Port Running? ---------------- ------------ ------------ ------------ ------------ ------------ db-cluster db2 true LISTENER 1521 true Checking name resolution setup for "db-cluster"... ERROR: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "db-cluster" SCAN Name IP Address Status Comment ------------ ------------------------ ------------------------ ---------- db-cluster 192.168.1.165 failed NIS Entry ERROR: PRVF-4657 : Name resolution setup check for "db-cluster" (IP address: 192.168.1.165) failed ERROR: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "db-cluster" Verification of SCAN VIP and Listener setup failed Checking to make sure user "grid" is not in "root" group Node Name Status Comment ------------ ------------------------ ------------------------ db3 does not exist passed Result: User "grid" is not part of "root" group. Check passed Checking if Clusterware is installed on all nodes... Check of Clusterware install passed Checking if CTSS Resource is running on all nodes... Check: CTSS Resource running on all nodes Node Name Status ------------------------------------ ------------------------ db3 passed Result: CTSS resource check passed Querying CTSS for time offset on all nodes... Result: Query of CTSS for time offset passed Check CTSS state started... Check: CTSS state Node Name State ------------------------------------ ------------------------ db3 Active CTSS is in Active state. Proceeding with check of clock time offsets on all nodes... Reference Time Offset Limit: 1000.0 msecs Check: Reference Time Offset Node Name Time Offset Status ------------ ------------------------ ------------------------ db3 0.0 passed Time offset is within the specified limits on the following set of nodes: "[db3]" Result: Check of clock time offsets passed Oracle Cluster Time Synchronization Services check passed Post-check for node addition was unsuccessful on all the nodes. |
驗證提示unsuccessful,從傳回結果來看
PRVF-4664 : Found inconsistent name resolution entries for SCAN name "db-cluster"
是這個導緻的
由于并沒有使用dns做輪詢去解析這個db-cluster域名,是以這個錯誤,可以忽略。
擴充database軟體了
[[email protected] ~]# su - oracle [[email protected] ~]$cd /u01/app/oracle/product/11.2.0/db/oui/bin [[email protected] bin]$ pwd /u01/app/oracle/product/11.2.0/db/oui/bin [[email protected] bin]$ ./addNode.sh -silent "CLUSTER_NEW_NODES={db3}" Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 4992 MB Passed Oracle Universal Installer, Version 11.2.0.1.0 Production Copyright (C) 1999, 2009, Oracle. All rights reserved. Performing tests to see whether nodes db2,db3 are available ............................................................... 100% Done. ...... ----------------------------------------------------------------------------- Cluster Node Addition Summary Global Settings Source: /u01/app/oracle/product/11.2.0/db New Nodes Space Requirements New Nodes db3 /: Required 4.00GB : Available 78.37GB Installed Products Product Names Oracle Database 11g 11.2.0.1.0 Sun JDK 1.5.0.17.0 Installer SDK Component 11.2.0.1.0 Oracle One-Off Patch Installer 11.2.0.0.2 Oracle Universal Installer 11.2.0.1.0 Oracle Configuration Manager Deconfiguration 10.3.1.0.0 Oracle DBCA Deconfiguration 11.2.0.1.0 Oracle RAC Deconfiguration 11.2.0.1.0 Oracle Database Deconfiguration 11.2.0.1.0 Oracle Configuration Manager 10.3.1.1.0 Oracle ODBC Driverfor Instant Client 11.2.0.1.0 LDAP Required Support Files 11.2.0.1.0 SSL Required Support Files for InstantClient 11.2.0.1.0 Bali Share 1.1.18.0.0 Oracle Extended Windowing Toolkit 3.4.47.0.0 Oracle JFC Extended Windowing Toolkit 4.2.36.0.0 Oracle Real Application Testing 11.2.0.1.0 Oracle Database Vault J2EE Application 11.2.0.1.0 Oracle Label Security 11.2.0.1.0 Oracle Data Mining RDBMS Files 11.2.0.1.0 Oracle OLAP RDBMS Files 11.2.0.1.0 Oracle OLAP API 11.2.0.1.0 Platform Required Support Files 11.2.0.1.0 Oracle Database Vault option 11.2.0.1.0 Oracle RAC Required Support Files-HAS 11.2.0.1.0 SQL*Plus Required Support Files 11.2.0.1.0 Oracle Display Fonts 9.0.2.0.0 Oracle Ice Browser 5.2.3.6.0 Oracle JDBC Server Support Package 11.2.0.1.0 Oracle SQL Developer 11.2.0.1.0 Oracle Application Express 11.2.0.1.0 XDK Required Support Files 11.2.0.1.0 RDBMS Required Support Files for Instant Client 11.2.0.1.0 SQLJ Runtime 11.2.0.1.0 Database Workspace Manager 11.2.0.1.0 RDBMS Required Support Files 11.2.0.1.0 Oracle Globalization Support 11.2.0.1.0 Exadata Storage Server 11.2.0.1.0 Provisioning Advisor Framework 10.2.0.4.2 Enterprise Manager Database Plugin -- Repository Support 11.2.0.1.0 Enterprise Manager Repository Core Files 10.2.0.4.2 Enterprise Manager Database Plugin -- Agent Support 11.2.0.1.0 Enterprise Manager Grid Control Core Files 10.2.0.4.2 Enterprise Manager Common Core Files 10.2.0.4.2 Enterprise Manager Agent Core Files 10.2.0.4.2 Agent Required Support Files 10.2.0.4.2 regexp 2.1.9.0.0 Parser Generator Required Support Files 11.2.0.1.0 Oracle 11g Warehouse Builder Required Files 11.2.0.1.0 Oracle Notification Service (eONS) 11.2.0.1.0 Oracle Text Required Support Files 11.2.0.1.0 Precompiler Required Support Files 11.2.0.1.0 Oracle Database 11g Multimedia Files 11.2.0.1.0 Oracle Multimedia Java Advanced Imaging 11.2.0.1.0 Oracle Multimedia Annotator 11.2.0.1.0 Oracle JDBC/OCI Instant Client 11.2.0.1.0 Oracle Multimedia Locator RDBMS Files 11.2.0.1.0 Oracle Core Required Support Files 11.2.0.1.0 Oracle Help For Java 4.2.9.0.0 Sample Schema Data 11.2.0.1.0 Oracle Starter Database 11.2.0.1.0 Oracle Message Gateway Common Files 11.2.0.1.0 Oracle XML Query 11.2.0.1.0 XML Parser for Oracle JVM 11.2.0.1.0 Expat libraries 2.0.1.0.1 Installation Plugin Files 11.2.0.1.0 Enterprise Manager Common Files 10.2.0.4.2 Perl Modules 5.10.0.0.1 Deinstallation Tool 11.2.0.1.0 Oracle Quality of Service Management (Client) 11.2.0.1.0 Perl Interpreter 5.10.0.0.1 JAccelerator (COMPANION) 11.2.0.1.0 Oracle Containers for Java 11.2.0.1.0 Oracle Code Editor 1.2.1.0.0I Oracle Net Required Support Files 11.2.0.1.0 Secure Socket Layer 11.2.0.1.0 Oracle Universal Connection Pool 11.2.0.1.0 Oracle JDBC/THIN Interfaces 11.2.0.1.0 Oracle Multimedia Client Option 11.2.0.1.0 Oracle Java Client 11.2.0.1.0 Character Set Migration Utility 11.2.0.1.0 Oracle Locale Builder 11.2.0.1.0 PL/SQL Embedded Gateway 11.2.0.1.0 OLAP SQL Scripts 11.2.0.1.0 Database SQL Scripts 11.2.0.1.0 Oracle Globalization Support 11.2.0.1.0 Required Support Files 11.2.0.1.0 SQL*Plus Files for Instant Client 11.2.0.1.0 Oracle ODBC Driver 11.2.0.1.0 Oracle Database User Interface 2.2.13.0.0 Oracle Notification Service 11.2.0.0.0 Enterprise Manager Minimal Integration 11.2.0.1.0 XML Parser for Java 11.2.0.1.0 Oracle Security Developer Tools 11.2.0.1.0 Oracle Wallet Manager 11.2.0.1.0 Cluster Verification Utility Common Files 11.2.0.1.0 Oracle Clusterware RDBMS Files 11.2.0.1.0 Oracle UIX 2.2.24.5.0 Enterprise Manager plugin Common Files 11.2.0.1.0 HAS Common Files 11.2.0.1.0 Precompiler Common Files 11.2.0.1.0 Installation Common Files 11.2.0.1.0 Oracle Help for the Web 2.0.14.0.0 Oracle LDAP administration 11.2.0.1.0 Buildtools Common Files 11.2.0.1.0 Assistant Common Files 11.2.0.1.0 Oracle Recovery Manager 11.2.0.1.0 PL/SQL 11.2.0.1.0 Generic Connectivity Common Files 11.2.0.1.0 Oracle Database Gateway for ODBC 11.2.0.1.0 Oracle Programmer 11.2.0.1.0 Oracle Database Utilities 11.2.0.1.0 Enterprise Manager Agent 10.2.0.4.2 Oracle Netca Client 11.2.0.1.0 SQL*Plus 11.2.0.1.0 Oracle Call Interface (OCI) 11.2.0.1.0 Oracle Multimedia Locator 11.2.0.1.0 Oracle Multimedia 11.2.0.1.0 Oracle Net 11.2.0.1.0 Database Configuration and Upgrade Assistants 11.2.0.1.0 Oracle XML Development Kit 11.2.0.1.0 Oracle JVM 11.2.0.1.0 Oracle Advanced Security 11.2.0.1.0 Oracle Internet Directory Client 11.2.0.1.0 HAS Files for DB 11.2.0.1.0 Oracle Enterprise Manager Console DB 11.2.0.1.0 Oracle Net Listener 11.2.0.1.0 Oracle Text 11.2.0.1.0 Oracle Net Services 11.2.0.1.0 Oracle Database 11g 11.2.0.1.0 Oracle OLAP 11.2.0.1.0 Oracle Spatial 11.2.0.1.0 Oracle Partitioning 11.2.0.1.0 Enterprise Edition Options 11.2.0.1.0 ----------------------------------------------------------------------------- Instantiating scripts for add node (Thursday, March 6, 2014 5:20:33 PM CST) . 1% Done. Instantiation of add node scripts complete Copying to remote nodes (Thursday, March 6, 2014 5:20:57 PM CST) ............................................................................................... 96% Done. Home copied to new nodes Saving inventory on nodes (Thursday, March 6, 2014 5:35:03 PM CST) . 100% Done. Save inventory complete WARNING: The following configuration scripts need to be executed as the "root" user in each cluster node. /u01/app/oracle/product/11.2.0/db/root.sh #On nodes db3 To execute the configuration scripts: 1. Open a terminal window 2. Log in as "root" 3. Run the scripts in each cluster node The Cluster Node Addition of /u01/app/oracle/product/11.2.0/db was successful. Please check '/tmp/silentInstall.log' for more details. |
現在就需要在節點3上執行這個腳本
[[email protected] ~]# /u01/app/oracle/product/11.2.0/db/root.sh Running Oracle 11g root.sh script... The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/11.2.0/db Enter the full pathname of the local bin directory: [/usr/local/bin]: The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed. Finished product-specific root actions. [[email protected] ~]# |
至此,grid,database軟體都已經裝完了,接下來添加,在節點1上執行
添加執行個體
[[email protected] ~]$ dbca
或用指令行直接添加
[[email protected] ~]$ dbca -silent -addInstance -nodeList db3 -gdbName db -instanceName db3 -sysDBAUserName sys -sysDBAPassword "oracle"
![](https://img.laitimes.com/img/9ZDMuAjOiMmIsIjOiQnIsIyMwcjMzkTN5AzNwMDM0EDMy8CX0Vmbu4GZzNmLn9Gbi1yZtl2Lc9CX6MHc0RHaiojIsJye.jpg)
這個地方一定要注意原來的執行個體是running的
[email protected] ~]$ crsctl status res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.DATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ONLINE ONLINE db3 ora.LISTENER.lsnr ONLINE ONLINE db1 ONLINE ONLINE db2 ONLINE ONLINE db3 ora.asm ONLINE ONLINE db1 Started ONLINE ONLINE db2 Started ONLINE ONLINE db3 Started ora.eons ONLINE ONLINE db1 ONLINE ONLINE db2 ONLINE ONLINE db3 ora.gsd OFFLINE OFFLINE db1 OFFLINE OFFLINE db2 OFFLINE OFFLINE db3 ora.net1.network ONLINE ONLINE db1 ONLINE ONLINE db2 ONLINE ONLINE db3 ora.ons ONLINE ONLINE db1 ONLINE ONLINE db2 ONLINE ONLINE db3 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE db2 ora.db.db 1 ONLINE ONLINE db1 Open 2 ONLINE ONLINE db2 Open 3 ONLINE ONLINE db3 Open ora.db1.vip 1 ONLINE ONLINE db1 ora.db2.vip 1 ONLINE ONLINE db2 ora.db3.vip 1 ONLINE ONLINE db3 ora.oc4j 1 OFFLINE OFFLINE ora.scan1.vip 1 ONLINE ONLINE db2 |