天天看点

ORA-07445 [SIGBUS] [Object specific hardware error]错误一例

一套Solaris上的9.2.0.7系统,实例意外终止,告警日志中出现以下记录:

<a href="http://blog.51cto.com/maclean/1277651#">?</a>

<code>Thu Sep 2 02:15:41 2010</code>

<code>Errors </code><code>in</code> <code>file /u01/app/oracle/admin/preg063/bdump/preg063_smon_11391.trc:</code>

<code>ORA-07445: exception encountered: core dump [0000000101E05500] [SIGBUS] [Object specific hardware error] [0xFFFFFFFF7CB3BF90] [] []</code>

<code>Thu Sep 2 02:15:48 2010</code>

<code>Errors </code><code>in</code> <code>file /u01/app/oracle/admin/preg063/bdump/preg063_pmon_11379.trc:</code>

<code>ORA-00474: SMON process terminated </code><code>with</code> <code>error</code>

<code>PMON: terminating instance due </code><code>to</code> <code>error 474</code>

<code>Wed Sep 1 15:04:20 2010</code>

<code>Errors </code><code>in</code> <code>file /u01/app/oracle/admin/preg063/udump/preg063_ora_1772.trc:</code>

<code>ORA-07445: exception encountered: core dump [FFFFFFFF7F400980] [SIGBUS] [Object specific hardware error] [0xFFFFFFFF7CA78000] [] []</code>

<code>Wed Sep 1 15:06:24 2010</code>

<code>Errors </code><code>in</code> <code>file /u01/app/oracle/admin/preg063/udump/preg063_ora_5316.trc:</code>

<code>ORA-07445: exception encountered: core dump [0000000101E10F94] [SIGBUS] [Object specific hardware error] [0xFFFFFFFF7C843DC8] [] []</code>

<code>Errors </code><code>in</code> <code>file /u01/app/oracle/admin/preg063/udump/preg063_ora_5207.trc:</code>

<code>ORA-07445: exception encountered: core dump [0000000101E10F94] [SIGBUS] [Object specific hardware error] [0xFFFFFFFF7C887A70] [] []</code>

<code>Errors </code><code>in</code> <code>file /u01/app/oracle/admin/preg063/udump/preg063_ora_28532.trc:</code>

<code>ORA-07445: exception encountered: core dump [0000000101E05500] [SIGBUS] [Object specific hardware error] [0xFFFFFFFF7C9B7FE8] [] []</code>

<code>Wed Sep 1 15:06:31 2010</code>

<code>Errors </code><code>in</code> <code>file /u01/app/oracle/admin/preg063/udump/preg063_ora_4392.trc:</code>

<code>ORA-07445: exception encountered: core dump [0000000100624600] [SIGBUS] [Object specific hardware error] [0xFFFFFFFF7CA80000] [] []</code>

<code>Wed Sep 1 15:06:33 2010</code>

<code>Errors </code><code>in</code> <code>file /u01/app/oracle/admin/preg063/udump/preg063_ora_3748.trc:</code>

<code>ORA-07445: exception encountered: core dump [FFFFFFFF7F400980] [SIGBUS] [Object specific hardware error]</code>

<code>trace日志:</code>

<code>Dump file /u01/app/oracle/admin/preg063/bdump/preg063_smon_11391.trc</code>

<code>Oracle9i Enterprise Edition Release 9.2.0.7.0 - 64bit Production</code>

<code>With</code> <code>the Partitioning, OLAP </code><code>and</code> <code>Oracle Data Mining options</code>

<code>JServer Release 9.2.0.7.0 - Production</code>

<code>ORACLE_HOME = /u01/app/oracle/product/920preg063</code>

<code>System </code><code>name</code><code>: SunOS</code>

<code>2010-09-02 02:15:41.917</code>

<code>*** SESSION ID:(7.1) 2010-09-02 02:15:33.090</code>

<code>Exception signal: 10 (SIGBUS), code: 3 (Object specific hardware error), addr: 0xffffffff7cb3bf90, PC: [0x101e05500, 0000000101E05500]</code>

<code>*** 2010-09-02 02:15:41.936</code>

<code>ksedmp: internal </code><code>or</code> <code>fatal error</code>

<code>Current</code> <code>SQL statement </code><code>for</code> <code>this session:</code>

<code>update</code> <code>sys.col_usage$ </code><code>set</code> <code>equality_preds = equality_preds + decode(bitand(:flag,1),0,0,1), equijoin_preds = equijoin_preds + decode(bitand(:flag,2),0,0,1), nonequijoin_preds = nonequijoin_preds + decode(bitand(:flag,4),0,0,1), range_preds = range_preds + decode(bitand(:flag,8),0,0,1), like_preds = like_preds + decode(bitand(:flag,16),0,0,1), null_preds = null_preds + decode(bitand(:flag,32),0,0,1), </code><code>timestamp</code> <code>= :</code><code>time</code> <code>where</code> <code>obj# = :objn </code><code>and</code> <code>intcol# = :coln</code>

<code>----- Call Stack Trace -----</code>

<code>calling call entry argument </code><code>values</code> <code>in</code> <code>hex</code>

<code>location type point (? means dubious value)</code>

<code>-------------------- -------- -------------------- ----------------------------</code>

<code>ksedmp()+328 CALL ksedst() 00000000B ? 000000000 ?</code>

<code>000000000 ? 103357D68 ?</code>

<code>00000003E ?</code>

<code>FFFFFFFF7FFE3EE8 ?</code>

<code>ssexhd()+676 CALL ksedmp() 000103788 ? 103788000 ?</code>

<code>103788C68 ? 10378D000 ?</code>

<code>000102C00 ? 000000000 ?</code>

<code>sigacthandler()+44 PTR_CALL 0000000000000000 000103790 ?</code>

<code>FFFFFFFF7FFEAF80 ?</code>

<code>103790000 ? 10378DE30 ?</code>

<code>000000000 ? 103790DB0 ?</code>

<code>kghbshrt()+96 PTR_CALL 0000000000000000 00000000A ?</code>

<code>FFFFFFFF7FFEACA0 ?</code>

<code>000000000 ? 00000BF78 ?</code>

<code>0B38F0000 ?</code>

<code>kghalo()+612 CALL kghbshrt() 103788EC8 ? 000000000 ?</code>

<code>10378C2E0 ?</code>

<code>FFFFFFFF7CB30018 ?</code>

<code>000000458 ? 080000000 ?</code>

<code>kghgex()+336 CALL kghalo() 000000000 ?</code>

<code>103788EC8 ? 000001000 ?</code>

<code>FFFFFFFF7CB3E120 ?</code>

<code>C0B38F000000C3D1 ?</code>

<code>kghalf()+316 CALL kghgex() 000007C00 ? 000000440 ?</code>

<code>000000228 ? 103788EC8 ?</code>

<code>000007FFF ? 000000000 ?</code>

<code>kdb4chk()+216 PTR_CALL 0000000000000000 103788EC8 ? 000000178 ?</code>

<code>FFFFFFFF7CB3E180 ?</code>

<code>FFFFFFFF7CB3E150 ?</code>

<code>000000000 ? 102DA35D0 ?</code>

<code>kd4chk()+88 CALL kdb4chk() 3F5484014 ? 1007E7780 ?</code>

<code>000000000 ? 000000000 ?</code>

<code>1007E76C0 ? 000000000 ?</code>

<code>kcbchk()+244 PTR_CALL 0000000000000000 3F5484000 ? 000000000 ?</code>

<code>000002000 ? 000000000 ?</code>

<code>1007E76C0 ? 1007E7720 ?</code>

<code>kcoapl()+1288 CALL kcbchk() 3F5484000 ? 0001007E7 ?</code>

<code>1007E7000 ? 101BA4440 ?</code>

<code>000100400 ? 0001007E7 ?</code>

<code>kcbapl()+96 CALL kcoapl() 000000002 ? 000000000 ?</code>

<code>000000001 ? 3F5484000 ?</code>

<code>10388DA08 ? 3F5486000 ?</code>

<code>kcrfwr()+1716 CALL kcbapl() 10388D990 ? 3F5FA6200 ?</code>

<code>FFFFFFFF7FFEC3B0 ?</code>

<code>00010378E ? 000000004 ?</code>

<code>000000004 ?</code>

<code>kcbchg1()+2056 CALL kcrfwr() FFFFFFFF7FFEC638 ?</code>

<code>40E36C218 ? 10378F5E8 ?</code>

<code>10378F690 ? 000000002 ?</code>

<code>ktuchg()+1044 CALL kcbchg1() 000000000 ?</code>

<code>FFFFFFFF7FFECE08 ?</code>

<code>000000000 ? 10378B530 ?</code>

<code>000000000 ?</code>

<code>FFFFFFFF7FFEC730 ?</code>

<code>ktbchg2()+132</code>

<code>Dump file /u01/app/oracle/admin/preg063/udump/preg063_ora_10110.trc</code>

<code>2010-09-01 00:29:31.655</code>

<code>ORA-07445: exception encountered: core dump [0000000101E10F94] [SIGBUS] [Object specific hardware error] [0xFFFFFFFF7CAAC000] [] []</code>

<code>select</code> <code>lxOid,lxVal </code><code>from</code> <code>lxString_c83b5454 </code><code>where</code> <code>lxOid </code><code>in</code> <code>(:va,:vb,:vc,:vd,:ve,:vf,:vg,:vh,:vi,:vj,:vk,:vl,:vm,:vn,:vo,:vp,:vq,:vr,:vs,:vt,:vu,:vv,:vw,:vx,:vy,:vz,:v26,:v27,:v28,:v29,:v30,:v31,:v32,:v33,:v34,:v35,:v36,:v37,:v38,:v39,:v40,:v41,:v42,:v43,:v44,:v45,:v46,:v47,:v48,:v49,:v50,:v51,:v52,:v53,:v54,:v55,:v56,:v57,:v58,:v59) </code><code>and</code> <code>lxType=:v60</code>

<code>FFFFFFFF7FFF3B98 ?</code>

<code>FFFFFFFF7FFFAC30 ?</code>

<code>kghalf()+436 PTR_CALL 0000000000000000 00000000A ?</code>

<code>FFFFFFFF7FFFA950 ?</code>

<code>103788EC8 ? 0B38F0000 ?</code>

<code>B38F0000000000 ?</code>

<code>qerhjManageListElem CALL kghalf() 103788EC8 ?</code>

<code>ent()+116 10B38F0000000001 ?</code>

<code>FFFFFFFF7CC73418 ?</code>

<code>FFFFFFFF7CAABFF8 ?</code>

<code>1034CC568 ?</code>

<code>qerhjadf()+248 CALL qerhjManageListElem FFFFFFFF7CBA2568 ?</code>

<code>ent() FFFFFFFF7CC733A8 ?</code>

<code>000000002 ? 000000049 ?</code>

<code>FFFFFFFF7CA41E9A ?</code>

<code>0000000AF ?</code>

<code>qerhjBuildHashTable CALL qerhjadf() FFFFFFFF7FFFB0F8 ?</code>

<code>()+4236 FFFFFFFF7C9D2DB0 ?</code>

<code>000000013 ?</code>

<code>FFFFFFFF7C9D2D90 ?</code>

<code>000000001 ?</code>

<code>qerhjFetch()+772 CALL qerhjBuildHashTable 000000000 ? 1038698D8 ?</code>

<code>() 000000040 ?</code>

<code>FFFFFFFF7C9B16A8 ?</code>

<code>FFFFFFFF7CBA2630 ?</code>

<code>qervwFetch()+148 PTR_CALL 0000000000000000 43C80CBB0 ? 101624E80 ?</code>

<code>FFFFFFFF7FFFB338 ?</code>

<code>00000001E ? 000000008 ?</code>

<code>00000005F ?</code>

<code>kpofrws()+124 PTR_CALL 0000000000000000 43C80CB58 ? 100FEF620 ?</code>

<code>FFFFFFFF7FFFB940 ?</code>

<code>00000001E ? 000101400 ?</code>

<code>000101624 ?</code>

<code>opifch2()+1704 CALL kpofrws() 000000004 ? 000000004 ?</code>

<code>100FEF620 ?</code>

<code>102EC9160 ? 000000010 ?</code>

<code>opiall0()+3900 CALL opifch2() 10101B000 ? 000000006 ?</code>

提交SR后,MOS认为可能是swap空间不足导致,建议检查系统日志:"Please check your OS logs for any errors during this time.Also please check swap space. This error is sometimes recorded when swap space becomes full"。 系统日志显示在Sep 2出现过/tmp文件系统空间耗尽,swap空间不足的问题:

<code>Aug 15 02:41:25 v08k419 tmpfs: [ID 518458 kern.warning] WARNING: /tmp: File system </code><code>full</code><code>, swap </code><code>space</code> <code>limit exceeded</code>

<code>Aug 15 02:48:06 v08k419 tmpfs: [ID 518458 kern.warning] WARNING: /tmp: File system </code><code>full</code><code>, swap </code><code>space</code> <code>limit exceeded</code>

<code>Sep 2 02:15:42 v08k419 tmpfs: [ID 518458 kern.warning] WARNING: /tmp: File system </code><code>full</code><code>, swap </code><code>space</code> <code>limit exceeded</code>

该内部错误通过增大swap最后解决了,that's great!

本文转自maclean_007 51CTO博客,原文链接:http://blog.51cto.com/maclean/1277651