天天看点

Weblogic Cluster BEA-000116故障处理

部署环境:weblogic 10 mp2 for unix

实验:

weblogic cluster: (mserver1,mserver2) 7222

multicast ip: 239.192.0.0

adminserver: 192.168.1.33:7001

mserver1: 192.168.1.33:7201

mserver2: 192.168.1.33:7202

machine1: (mserver1) nodemanager(192.168.1.33:5556)

machine2: (mserver2) nodemanager(192.168.1.33:5557)

注意:本实验环境各个server ssl设置---->hostname verification: none 

描述:

(nodemanager 成功启动)

当启动某managedserver时分别报以下错误:

/home/weblogic/bea/user_projects/domains/clustersys/servers/mserver_1/logs/mserver_1.out

dec 16, 2010 9:12:56 pm cst> <info> <weblogicserver> <bea-000215> 

<loaded license : /home/weblogic/bea/license.bea> 

<dec 16, 2010 9:12:57 pm cst> <notice> <weblogicserver> <bea-000365> <server state changed to starting> 

<dec 16, 2010 9:12:57 pm cst> <info> <workmanager> <bea-002900> <initializing self-tuning thread pool> 

<dec 16, 2010 9:12:58 pm cst> <notice> <log management> <bea-170019> <the server log file

<dec 16, 2010 9:13:15 pm cst> <notice> <security> <bea-090082> <security initializing using security realm myrealm.> 

<dec 16, 2010 9:13:31 pm cst> <notice> <weblogicserver> <bea-000365> <server state changed to standby> 

<dec 16, 2010 9:13:31 pm cst> <notice> <weblogicserver> <bea-000365> <server state changed to starting> 

<dec 16, 2010 9:13:34 pm cst> <error> <cluster> <bea-000116> 

no such device.java.net.socketexception: 

no such device 

at java.net.plaindatagramsocketimpl.join(native method)

at java.net.plaindatagramsocketimpl.join(plaindatagramsocketimpl.java:172)

at java.net.multicastsocket.joingroup(multicastsocket.java:276)

at weblogic.cluster.multicastfragmentsocket.initializemulticastsocket(multicastfragmentsocket.java:112)

at weblogic.cluster.multicastfragmentsocket.start(multicastfragmentsocket.java:134)

truncated. see log file for complete stacktrace

> 

<dec 16, 2010 9:13:34 pm cst> <critical> <weblogicserver> <bea-000362> 

<server failed. reason: there are 1 nested errors:

java.net.socketexception: no such device

at weblogic.cluster.fragmentsocketwrapper.start(fragmentsocketwrapper.java:87)

at weblogic.cluster.multicastmanager$1.run(multicastmanager.java:186)

at weblogic.cluster.multicastmanager.startlistening(multicastmanager.java:192)

at weblogic.cluster.inboundservice.startlistening(inboundservice.java:40)

at weblogic.cluster.inboundservice.start(inboundservice.java:29)

at weblogic.t3.srvr.subsystemrequest.run(subsystemrequest.java:64)

at weblogic.work.selftuningworkmanagerimpl$workadapterimpl.run(selftuningworkmanagerimpl.java:464)

at weblogic.work.executethread.execute(executethread.java:200)

at weblogic.work.executethread.run(executethread.java:172)

>

<dec 16, 2010 9:13:34 pm cst> <notice> <weblogicserver> <bea-000365> <server state changed to failed> 

<dec 16, 2010 9:13:34 pm cst> <error> <weblogicserver> <bea-000383> 

<a critical service failed. the server will shut itself down> 

<dec 16, 2010 9:13:34 pm cst> <notice> <weblogicserver> <bea-000365> 

<server state changed to force_shutting_down> 

<dec 16, 2010 9:13:37 pm> <debug> <nodemanager> <waiting for the process to die: 26869>

<dec 16, 2010 9:13:38 pm> <info> <nodemanager> <server failed during startup so will not be restarted>

<dec 16, 2010 9:13:38 pm> <debug> <nodemanager> 

<runmonitor returned, setting finished=true and notifying waiters> 

adminserver 前台报以下信息: 

<dec 16, 2010 9:13:40 pm cst> <error> <nodemanager> <bea-300048> <unable to start the server mserver_1 :

exception while starting server 'mserver_1': java.io.ioexception: server failed to start up.

see server output log for more details.> 

分析:依据以下提示,可判断组播地址问题。 

dec 16, 2010 9:13:34 pm cst> <error> <cluster> <bea-000116> <failed to join cluster cluster_1 

at address 239.192.0.0 due to: java.net.socketexception: no such device. 

查看系统路由表: 

[root@localhost ~]# route -n

kernel ip routing table

destination gateway genmask flags metric ref use iface

192.168.1.0 0.0.0.0 255.255.255.0 u 0 0 0 eth0

169.254.0.0 0.0.0.0 255.255.0.0 u 0 0 0 eth0 

缺一条组播239.192.0.0路由

手动添加路由: 

[root@localhost ~]# route add -host 239.192.0.0 dev eth0

239.192.0.0 0.0.0.0 255.255.255.255 uh 0 0 0 eth0

再启动受控服务器,ok

Weblogic Cluster BEA-000116故障处理

同样的原因也会导致以下异常抛出: 

注意:1. 本实验环境各个server ssl设置---->hostname verification:: bea hostname verifier

2. weblogic 11gr3(10.3) 不适合此环境

基于ssl环境下 ,成功启动节点管理器后,受控服务器无法正常启动

现象:

当启动mserver_1时,服务器状态:

mserver_1   machine_1   failed_not_restartable    failed 

nodemanager 报错:

dec 16, 2010 11:23:54 pm> <info> <clustersys> <states = {mserver_2=unknown, adminserver=unknown, mserver_1=unknown, domain_bak=unknown}>

<dec 16, 2010 11:23:54 pm> <info> <clustersys> <states = {mserver_2=unknown, adminserver=unknown, mserver_1=unknown, domain_bak=unknown}>

<dec 16, 2010 11:23:57 pm> <info> <clustersys> <states = {mserver_2=unknown, adminserver=unknown,

<dec 16, 2010 11:23:57 pm> <info> <clustersys> <mserver_1>

<dec 16, 2010 11:23:57 pm> <info> <clustersys> <mserver_1> <startup configuration properties saved to "/home/weblogic/bea/user_projects/domains/clustersys/servers/mserver_1/data/nodemanager/startup.properties">

<dec 16, 2010 11:23:57 pm> <info> <clustersys> <mserver_1> <rotated server output log to "/home/weblogic/bea/user_projects/domains/clustersys/servers/mserver_1/logs/mserver_1.out00005">

<dec 16, 2010 11:23:57 pm> <info> <clustersys> <mserver_1> <server error log also redirected to server log>

<dec 16, 2010 11:26:44 pm> <info> <clustersys> <mserver_1> <server failed during startup so will not be restarted>

<dec 16, 2010 11:26:44 pm> <warning> <exception while starting server 'mserver_1': java.io.ioexception: server failed to start up. see server output log for more details.>

java.io.ioexception: server failed to start up. see server output log for more details.

        at weblogic.nodemanager.server.servermanager.start(servermanager.java:303)

        at weblogic.nodemanager.server.handler.handlestart(handler.java:542)

        at weblogic.nodemanager.server.handler.handlecommand(handler.java:119)

        at weblogic.nodemanager.server.handler.run(handler.java:66)

        at java.lang.thread.run(thread.java:595)

<dec 16, 2010 11:28:18 pm> <info> <clustersys> <states = {mserver_2=unknown, adminserver=unknown, mserver_1=failed_not_restartable, domain_bak=unknown}> 

控制台:

<dec 16, 2010 11:26:44 pm cst>  <error> <nodemanager> <bea-300048> <unable to start the server mserver_1 : exception while starting server 'mserver_1': java.io.ioexception: server failed to start up. see server output log for more details.> 

查看日志,分析原因:

<dec 16, 2010 11:26:36 pm cst> <notice> <security> <bea-090082> <security initializing using security realm myrealm.> 

<dec 16, 2010 11:26:42 pm cst> <notice> <weblogicserver> <bea-000365> <server state changed to standby> 

<dec 16, 2010 11:26:42 pm cst> <notice> <weblogicserver> <bea-000365> <server state changed to starting>

<dec 16, 2010 11:26:43 pm cst>  <error> <cluster> <bea-000116> <failed to join cluster cluster_1 at address 239.192.0.0 due to : 

        at java.net.plaindatagramsocketimpl.join(native method)

        at java.net.plaindatagramsocketimpl.join(plaindatagramsocketimpl.java:172)

        at java.net.multicastsocket.joingroup(multicastsocket.java:276)

        at weblogic.cluster.multicastfragmentsocket.initializemulticastsocket(multicastfragmentsocket.java:112)

        at weblogic.cluster.multicastfragmentsocket.start(multicastfragmentsocket.java:134)

        truncated. see log file for complete stacktrace

<dec 16, 2010 11:26:43 pm cst> <critical> <weblogicserver> <bea-000362> <server failed. reason: there are 1 nested errors:

        at weblogic.cluster.fragmentsocketwrapper.start(fragmentsocketwrapper.java:87)

        at weblogic.cluster.multicastmanager$1.run(multicastmanager.java:186)

        at weblogic.cluster.multicastmanager.startlistening(multicastmanager.java:192)

        at weblogic.cluster.inboundservice.startlistening(inboundservice.java:40)

        at weblogic.cluster.inboundservice.start(inboundservice.java:29)

        at weblogic.t3.srvr.subsystemrequest.run(subsystemrequest.java:64)

        at weblogic.work.selftuningworkmanagerimpl$workadapterimpl.run(selftuningworkmanagerimpl.java:464)

        at weblogic.work.executethread.execute(executethread.java:200)

        at weblogic.work.executethread.run(executethread.java:172)

<dec 16, 2010 11:26:43 pm cst> <notice> <weblogicserver> <bea-000365> <server state changed to failed> 

<dec 16, 2010 11:26:43 pm cst> <error> <weblogicserver> <bea-000383> <a critical service failed. the server will shut itself down> 

<dec 16, 2010 11:26:43 pm cst> <notice> <weblogicserver> <bea-000365> <server state changed to force_shutting_down> 

<dec 16, 2010 11:26:44 pm> <debug> <nodemanager> <waiting for the process to die: 4428>

<dec 16, 2010 11:26:44 pm> <info> <nodemanager> <server failed during startup so will not be restarted>

<dec 16, 2010 11:26:44 pm> <debug> <nodemanager> <runmonitor returned, setting finished=true and notifying waiters> 

解决方法同上,手动添加组播路由

如有错误,欢迎指正

邮箱:[email protected]

作者:czmmiao  文章出处:http://czmmiao.iteye.com/blog/1617243