2018年1月份第一次巡檢中,某地市oracle資料庫告警ORA-20011&KUP-11024&ORA-29913,
相關處理如下:
作業系統版本:rhel 2.6.39-400.17.1.el6uek.x86_64
作業系統核心:rhel 2.6.39-400.17.1.el6uek.x86_64
資料庫版本:oracle 11.2.0.4
問題描述:資料庫對資料泵作業殘餘外部表收集統計資訊報錯:ORA-20011&KUP-11024&ORA-29913
告警日志報錯資訊如下:
09/01/2018 22:00:10 DBMS_STATS: GATHER_STATS_JOB encountered errors. Check the trace file.
ORA-20011: Approximate NDV failed: ORA-29913: error in executing ODCIEXTTABLEOPEN callout
Tue Jan 09 22:00:10 2018
KUP-11024: This external table can only be accessed from within a Data Pump job.
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_j003_333.trc
檢視告警日志提示的trc跟蹤檔案orcl_j003_333.trc
Trace file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_j003_333.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1
System name: Linux
Node name: cis.db
Release: 2.6.39-400.17.1.el6uek.x86_64
Version: #1 SMP Fri Feb 22 18:16:18 PST 2013
Machine: x86_64
Instance name: orcl
Redo thread mounted by this instance: 1
Oracle process number: 57
Unix process pid: 333, image: [email protected] (J003)
*** 2018-01-09 22:00:10.612
*** SESSION ID:(634.22947) 2018-01-09 22:00:10.612
*** CLIENT ID:() 2018-01-09 22:00:10.612
*** SERVICE NAME:(SYS$USERS) 2018-01-09 22:00:10.612
*** MODULE NAME:(DBMS_SCHEDULER) 2018-01-09 22:00:10.612
*** ACTION NAME:(ORA$AT_OS_OPT_SY_152) 2018-01-09 22:00:10.612
ORA-20011: Approximate NDV failed: ORA-29913: error in executing ODCIEXTTABLEOPEN callout
KUP-11024: This external table can only be accessed from within a Data Pump job.
*** 2018-01-09 22:00:10.612
DBMS_STATS: GATHER_STATS_JOB: GATHER_TABLE_STATS('"EXPDP"','"ET$01B7017F0001"','""', ...)
DBMS_STATS: ORA-20011: Approximate NDV failed: ORA-29913: error in executing ODCIEXTTABLEOPEN callout
KUP-11024: This external table can only be accessed from within a Data Pump job.
*** 2018-01-09 22:00:10.656
DBMS_STATS: GATHER_STATS_JOB: GATHER_TABLE_STATS('"EXPDP"','"ET$01C303530001"','""', ...)
DBMS_STATS: ORA-20011: Approximate NDV failed: ORA-29913: error in executing ODCIEXTTABLEOPEN callout
KUP-11024: This external table can only be accessed from within a Data Pump job.
*** 2018-01-09 22:00:10.686
DBMS_STATS: GATHER_STATS_JOB: GATHER_TABLE_STATS('"EXPDP"','"ET$01982C090001"','""', ...)
DBMS_STATS: ORA-20011: Approximate NDV failed: ORA-29913: error in executing ODCIEXTTABLEOPEN callout
KUP-11024: This external table can only be accessed from within a Data Pump job.
*** 2018-01-09 22:00:10.711
DBMS_STATS: GATHER_STATS_JOB: GATHER_TABLE_STATS('"EXPDP"','"ET$01D400370001"','""', ...)
DBMS_STATS: ORA-20011: Approximate NDV failed: ORA-29913: error in executing ODCIEXTTABLEOPEN callout
KUP-11024: This external table can only be accessed from within a Data Pump job.
*** 2018-01-09 22:00:10.735
DBMS_STATS: GATHER_STATS_JOB: GATHER_TABLE_STATS('"EXPDP"','"ET$01D3186F0001"','""', ...)
DBMS_STATS: ORA-20011: Approximate NDV failed: ORA-29913: error in executing ODCIEXTTABLEOPEN callout
KUP-11024: This external table can only be accessed from within a Data Pump job.
*** 2018-01-09 22:00:10.759
DBMS_STATS: GATHER_STATS_JOB: GATHER_TABLE_STATS('"EXPDP"','"ET$00251AEF0001"','""', ...)
DBMS_STATS: ORA-20011: Approximate NDV failed: ORA-29913: error in executing ODCIEXTTABLEOPEN callout
KUP-11024: This external table can only be accessed from within a Data Pump job.
處理方法:
1、DBMS_STATS job 運作時,沒有DataPump job運作
SQL> !ps -ef|grep expdp
oracle 28440 28067 0 09:49 pts/0 00:00:00 /bin/bash -c ps -ef|grep expdp
oracle 28442 28440 0 09:49 pts/0 00:00:00 grep expdp
2、确定外部表屬于datapump job
set linesize 200 trimspool on
set pagesize 2000
col owner form a30
col created form a25
col last_ddl_time form a25
col object_name form a30
col object_type form a25
SQL> select OWNER,OBJECT_NAME,OBJECT_TYPE, status,
2 to_char(CREATED,'dd-mon-yyyy hh24:mi:ss') created,
3 to_char(LAST_DDL_TIME, 'dd-mon-yyyy hh24:mi:ss') last_ddl_time
4 from dba_objects
5 where object_name like 'ET$%';
OWNER OBJECT_NAME OBJECT_TYPE STATUS CREATED LAST_DDL_TIME
------------------------------ ------------------------------ ------------------------- ------- ------------------------- -------------------------
EXPDP ET$01D400370001 TABLE VALID 09-oct-2015 23:11:41 09-oct-2015 23:11:41
EXPDP ET$01D3186F0001 TABLE VALID 18-dec-2015 02:20:04 18-dec-2015 02:20:04
EXPDP ET$01C303530001 TABLE VALID 15-mar-2015 00:45:46 15-mar-2015 00:45:46
EXPDP ET$01B7017F0001 TABLE VALID 15-mar-2015 00:42:03 15-mar-2015 00:42:03
EXPDP ET$01982C090001 TABLE VALID 08-oct-2015 23:51:06 08-oct-2015 23:51:06
EXPDP ET$00251AEF0001 TABLE VALID 29-dec-2015 00:50:20 29-dec-2015 00:50:20
6 rows selected.
SQL> select owner, TABLE_NAME, DEFAULT_DIRECTORY_NAME, ACCESS_TYPE
from dba_external_tables
order by 1,2;
OWNER TABLE_NAME DEFAULT_DIRECTORY_NAME ACCESS_
------------------------------ ------------------------------ ------------------------------ -------
EXPDP ET$00251AEF0001 EXPDATA CLOB
EXPDP ET$01982C090001 EXPDATA CLOB
EXPDP ET$01B7017F0001 EXPDATA CLOB
EXPDP ET$01C303530001 EXPDATA CLOB
EXPDP ET$01D3186F0001 EXPDATA CLOB
EXPDP ET$01D400370001 EXPDATA CLOB
SYS ALERT_LOG_VIEW BDUMP CLOB
--查詢外部表,報的錯誤和alert日志一樣
SQL> select count(*) from EXPDP.ET$00251AEF0001;
select count(*) from EXPDP.ET$00251AEF0001
*
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
KUP-11024: This external table can only be accessed from within a Data Pump job.
SQL>
--檢視外部表定義
SQL> desc ET$00251AEF0001
ERROR:
ORA-04043: object ET$00251AEF0001 does not exist
3、清理DataPump jobs。
SQL> select 'drop table '||owner||'.'||object_name||' purge ;' from dba_objects where object_name like 'ET$%';
'DROPTABLE'||OWNER||'.'||OBJECT_NAME||'PURGE;'
----------------------------------------------------
drop table EXPDP.ET$01D400370001 purge ;
drop table EXPDP.ET$01D3186F0001 purge ;
drop table EXPDP.ET$01C303530001 purge ;
drop table EXPDP.ET$01B7017F0001 purge ;
drop table EXPDP.ET$01982C090001 purge ;
drop table EXPDP.ET$00251AEF0001 purge ;
6 rows selected.
SQL> drop table EXPDP.ET$01D400370001 purge ;
drop table EXPDP.ET$01D3186F0001 purge ;
drop table EXPDP.ET$01C303530001 purge ;
drop table EXPDP.ET$01B7017F0001 purge ;
drop table EXPDP.ET$01982C090001 purge ;
drop table EXPDP.ET$00251AEF0001 purge ;
Table dropped.
SQL>
Table dropped.
SQL>
Table dropped.
SQL>
Table dropped.
SQL>
Table dropped.
SQL>
Table dropped.
由以上資訊可知,資料泵的外部表産生的時間分别是2015年3月份和10月份,經溝通得知該地市資料庫服務
器發生過重裝和機房搬遷:
猜測:可能當時資料庫重裝和機房搬遷,資料庫使用過資料泵備份而資料泵作業沒有成功退出,導緻資料泵
作業外部表殘留,影響到資料庫統計資訊收集,處理方法就是删除資料泵作業殘留外部表。個人覺得比較奇
怪的是資料泵作業外部表實體已經不存在,DDL定義已經不存在,但是外部表明卻存在,應該是資料庫字典
資訊錯誤。