[20180705]关于hash join 2.txt

--//昨天优化sql语句,执行计划hash join right sna,加入一个约束设置XX字段not null,逻辑读从上万下降到50.

--//关于hash join派生的执行计划,而且hash join还在外连接时支持右关联,特别是11g,加入NULL-AWARW/Single Null-Aware的判断,许

--//多我自己很混乱,做一点总结.

--//NA => Null-Aware.

--//SNA => Single Null-Aware

1.环境:

SCOTT@book> @ ver1

PORT_STRING VERSION BANNER

------------------------------ -------------- --------------------------------------------------------------------------------

x86_64/Linux 2.4.xx 11.2.0.4.0 Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

create table t1 as select level id ,'t1'||to_char(level) name from dual connect by level<=4;

create table t2 as select level+1 id ,'t2'||to_char(level) name from dual connect by level<=4;

insert into t1 values (null,'t1null');

insert into t2 values (null,'t2null');

commit ;

--//分析略.

2.执行计划包含hash join:

SCOTT@test01p> select * from t1,t2 where t1.id=t2.id;

ID NAME ID NAME

---------- -------------------- ---------- --------------------

2 t12 2 t21

3 t13 3 t22

4 t14 4 t23

Plan hash value: 1838229974

------------------------------------------------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | | | 8 (100)| | 3 |00:00:00.01 | 15 | | | |

|* 1 | HASH JOIN | | 1 | 4 | 64 | 8 (0)| 00:00:01 | 3 |00:00:00.01 | 15 | 1888K| 1888K| 1063K (0)|

|* 2 | TABLE ACCESS FULL| T1 | 1 | 4 | 32 | 4 (0)| 00:00:01 | 4 |00:00:00.01 | 7 | | | |

|* 3 | TABLE ACCESS FULL| T2 | 1 | 4 | 32 | 4 (0)| 00:00:01 | 4 |00:00:00.01 | 8 | | | |

--//如果改变连接顺序,加入提示:

SCOTT@book> select /*+ leading(t2 t1) */ * from t1,t2 where t1.id=t2.id;

Plan hash value: 2959412835

| 0 | SELECT STATEMENT | | 1 | | | 7 (100)| | 3 |00:00:00.01 | 9 | | | |

|* 1 | HASH JOIN | | 1 | 4 | 56 | 7 (0)| 00:00:01 | 3 |00:00:00.01 | 9 | 1888K| 1888K| 830K (0)|

| 2 | TABLE ACCESS FULL| T2 | 1 | 4 | 28 | 3 (0)| 00:00:01 | 4 |00:00:00.01 | 2 | | | |

|* 3 | TABLE ACCESS FULL| T1 | 1 | 4 | 28 | 4 (0)| 00:00:01 | 4 |00:00:00.01 | 7 | | | |

3.执行计划包含HASH JOIN OUTER/HASH JOIN RIGHT OUTER:

SCOTT@book> select * from t1,t2 where t1.id=t2.id(+);

t1null

1 t11

Plan hash value: 1823443478

| 0 | SELECT STATEMENT | | 1 | | | 7 (100)| | 5 |00:00:00.01 | 9 | | | |

|* 1 | HASH JOIN OUTER | | 1 | 5 | 84 | 7 (0)| 00:00:01 | 5 |00:00:00.01 | 9 | 1888K| 1888K| 935K (0)|

| 2 | TABLE ACCESS FULL| T1 | 1 | 5 | 42 | 4 (0)| 00:00:01 | 5 |00:00:00.01 | 6 | | | |

| 3 | TABLE ACCESS FULL| T2 | 1 | 4 | 28 | 3 (0)| 00:00:01 | 4 |00:00:00.01 | 3 | | | |

--// 通过改变连接顺序,HASH JOIN OUTER:

SCOTT@book> select /*+ SWAP_JOIN_INPUTS(t2) */ * from t1,t2 where t1.id=t2.id(+);

Plan hash value: 312430291

---------------------------------------------------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | | | 8 (100)| | 5 |00:00:00.01 | 15 | | | |

|* 1 | HASH JOIN RIGHT OUTER| | 1 | 5 | 80 | 8 (0)| 00:00:01 | 5 |00:00:00.01 | 15 | 1888K| 1888K| 926K (0)|

|* 2 | TABLE ACCESS FULL | T2 | 1 | 4 | 32 | 4 (0)| 00:00:01 | 4 |00:00:00.01 | 7 | | | |

| 3 | TABLE ACCESS FULL | T1 | 1 | 5 | 40 | 4 (0)| 00:00:01 | 5 |00:00:00.01 | 8 | | | |

--//在nested loop,merge join,hash join连接中,仅仅hash join在外连接中支持right连接模式,其它一般不提都是left连接.

--//nested loop 算法根本不可能支持right连接模式.

--//而merge join理论可以支持,但是因为不管如何metge join 两边都要排序,完全没有必要在支持right连接模式.(注这个是我个人的理解,也许不对)

--//而hash join如果右关联表生成hash表小,完全可能右关联表作为驱动表.

4.执行计划包含hash join anti/hash join right anti:

SCOTT@book> select * from t1 where id not in (select id from t2 where id is not null) and id is not null;

ID NAME

---------- --------------------

--//我现在查询2个表的id is not null.

Plan hash value: 2706079091

| 0 | SELECT STATEMENT | | 1 | | | 8 (100)| | 1 |00:00:00.01 | 12 | | | |

|* 1 | HASH JOIN ANTI | | 1 | 1 | 11 | 8 (0)| 00:00:01 | 1 |00:00:00.01 | 12 | 1888K| 1888K| 978K (0)|

|* 2 | TABLE ACCESS FULL| T1 | 1 | 4 | 32 | 4 (0)| 00:00:01 | 4 |00:00:00.01 | 6 | | | |

|* 3 | TABLE ACCESS FULL| T2 | 1 | 4 | 12 | 4 (0)| 00:00:01 | 4 |00:00:00.01 | 6 | | | |

--//如何实现交换顺序呢?

SCOTT@book> select * from t1 where id not in (select /*+ SWAP_JOIN_INPUTS(t2) */ id from t2 where id is not null) and id is not null;

Plan hash value: 629543484

--------------------------------------------------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | | | 8 (100)| | 1 |00:00:00.01 | 13 | | | |

|* 1 | HASH JOIN RIGHT ANTI| | 1 | 1 | 11 | 8 (0)| 00:00:01 | 1 |00:00:00.01 | 13 | 2440K| 2440K| 855K (0)|

|* 2 | TABLE ACCESS FULL | T2 | 1 | 4 | 12 | 4 (0)| 00:00:01 | 4 |00:00:00.01 | 6 | | | |

|* 3 | TABLE ACCESS FULL | T1 | 1 | 4 | 32 | 4 (0)| 00:00:01 | 4 |00:00:00.01 | 7 | | | |

5.执行计划包含hash join semi/hash join right semi:

SCOTT@book> select * from t1 where id in (select id from t2 where id is not null) and id is not null;

2 t12

3 t13

4 t14

Plan hash value: 1713220790

| 0 | SELECT STATEMENT | | 1 | | | 8 (100)| | 3 |00:00:00.01 | 13 | | | |

|* 1 | HASH JOIN SEMI | | 1 | 3 | 33 | 8 (0)| 00:00:01 | 3 |00:00:00.01 | 13 | 1888K| 1888K| 935K (0)|

|* 3 | TABLE ACCESS FULL| T2 | 1 | 4 | 12 | 4 (0)| 00:00:01 | 4 |00:00:00.01 | 7 | | | |

--//改变交换顺序呢?

SCOTT@book> select * from t1 where id in (select /*+ SWAP_JOIN_INPUTS(t2) */ id from t2 where id is not null) and id is not null;

Plan hash value: 1275841967

| 0 | SELECT STATEMENT | | 1 | | | 8 (100)| | 3 |00:00:00.01 | 13 | | | |

|* 1 | HASH JOIN RIGHT SEMI| | 1 | 3 | 33 | 8 (0)| 00:00:01 | 3 |00:00:00.01 | 13 | 2440K| 2440K| 814K (0)|

5.执行计划包含HASH JOIN FULL OUTER:

SCOTT@book> select * from t1 full outer join t2 on t1.id=t2.id;

5 t24

t2null

7 rows selected.

Plan hash value: 53297166

-------------------------------------------------------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | | | 8 (100)| | 7 |00:00:00.01 | 13 | | | |

| 1 | VIEW | VW_FOJ_0 | 1 | 5 | 360 | 8 (0)| 00:00:01 | 7 |00:00:00.01 | 13 | | | |

|* 2 | HASH JOIN FULL OUTER| | 1 | 5 | 80 | 8 (0)| 00:00:01 | 7 |00:00:00.01 | 13 | 1888K| 1888K| 936K (0)|

| 3 | TABLE ACCESS FULL | T1 | 1 | 5 | 40 | 4 (0)| 00:00:01 | 5 |00:00:00.01 | 6 | | | |

| 4 | TABLE ACCESS FULL | T2 | 1 | 5 | 40 | 4 (0)| 00:00:01 | 5 |00:00:00.01 | 7 | | | |

--//交换连接顺序略.

6.执行计划包含hash join anti NA/hash join right anti NA:

SCOTT@book> select * from t1 where id not in (select id from t2 ) ;

no rows selected

Plan hash value: 1275484728

| 0 | SELECT STATEMENT | | 1 | | | 8 (100)| | 0 |00:00:00.01 | 10 | | | |

|* 1 | HASH JOIN ANTI NA | | 1 | 2 | 22 | 8 (0)| 00:00:01 | 0 |00:00:00.01 | 10 | 1888K| 1888K| 936K (0)|

| 2 | TABLE ACCESS FULL| T1 | 1 | 5 | 40 | 4 (0)| 00:00:01 | 5 |00:00:00.01 | 6 | | | |

| 3 | TABLE ACCESS FULL| T2 | 1 | 5 | 15 | 4 (0)| 00:00:01 | 5 |00:00:00.01 | 4 | | | |

--//NA表示Null-Aware,我开始以为2边都要判断id是否存在NULL值.实际上如果执行如下:

SCOTT@book> select * from t1 where id not in (select id from t2 ) and id is not null;

|* 1 | HASH JOIN ANTI NA | | 1 | 1 | 11 | 8 (0)| 00:00:01 | 0 |00:00:00.01 | 10 | 1888K| 1888K| 936K (0)|

--//说明实际上NA仅仅判断右关联表查询连接条件是否为NULL.

--//另外这种查询非常容易引起歧义性,许多开发包括我,以前一直以为至少返回id=1的记录.

--//实际上正是T2表里面id存在空值导致没有结果.即使写成如下,结果也一样:

select * from t1 where id not in (select id from t2 ) and id is not null;

--//另外注意仅仅not in,not exists在执行计划中才存在NA,SNA.

--//如何交换:

select * from t1 where id not in (select /*+ SWAP_JOIN_INPUTS(t2) */ id from t2 ) ;

--//以上这样写不行.要写成如下(注我使用通过获得outlined的执行计划),猜测这样加提示的:

SCOTT@book> select /*+ SWAP_JOIN_INPUTS(@"SEL$5DA710D3" "T2"@"SEL$2") */ * from t1 where id not in (select id from t2 ) ;

Plan hash value: 2739594415

-----------------------------------------------------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | | | 8 (100)| | 0 |00:00:00.01 | 4 | | | |

|* 1 | HASH JOIN RIGHT ANTI NA| | 1 | 2 | 22 | 8 (0)| 00:00:01 | 0 |00:00:00.01 | 4 | 1817K| 1817K| 841K (0)|

| 2 | TABLE ACCESS FULL | T2 | 1 | 5 | 15 | 4 (0)| 00:00:01 | 5 |00:00:00.01 | 4 | | | |

| 3 | TABLE ACCESS FULL | T1 | 0 | 5 | 40 | 4 (0)| 00:00:01 | 0 |00:00:00.01 | 0 | | | |

--//这里注意一个细节,T1实际上根本没有做全表扫描,逻辑读是0,starts=0.

--//插曲:补充测试,给T2增加一些测试数据:

insert into t2 select rownum+4 ,'t2'||to_char(rownum+4) from dual connect by level<=10000;

commit;

SCOTT@book> set autot traceonly

SCOTT@book> select count(*) from t2;

Execution Plan

----------------------------------------------------------

Plan hash value: 3321871023

-------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | 10 (0)| 00:00:01 |

| 1 | SORT AGGREGATE | | 1 | | |

| 2 | TABLE ACCESS FULL| T2 | 10005 | 10 (0)| 00:00:01 |

Statistics

0 recursive calls

0 db block gets

30 consistent gets

0 physical reads

0 redo size

528 bytes sent via SQL*Net to client

520 bytes received via SQL*Net from client

2 SQL*Net roundtrips to/from client

0 sorts (memory)

0 sorts (disk)

1 rows processed

SCOTT@book> set autot off

--//可以发现全表扫描T1,逻辑读是30.

| 0 | SELECT STATEMENT | | 1 | | | 14 (100)| | 0 |00:00:00.01 | 11 | | | |

|* 1 | HASH JOIN ANTI NA | | 1 | 1 | 12 | 14 (0)| 00:00:01 | 0 |00:00:00.01 | 11 | 1888K| 1888K| 1008K (0)|

| 3 | TABLE ACCESS FULL| T2 | 1 | 10005 | 40020 | 10 (0)| 00:00:01 | 459 |00:00:00.01 | 5 | | | |

--//你可以发现一个奇特现象,T2实际读取459条记录,buffers=5.并不是30,实际上因为前面已经存在id is null的记录,这样返回的结果集一定是没有结果,这样

--//停止了T2表的扫描.

SCOTT@book> column PARTITION_NAME noprint

SCOTT@book> select * from dba_extents where segment_name='T2';

OWNER SEGMENT_NAME SEGMENT_TYPE TABLESPACE_NAME EXTENT_ID FILE_ID BLOCK_ID BYTES BLOCKS RELATIVE_FNO

------ -------------------- ------------------ --------------- --------- ---------- ---------- ---------- ---------- ------------

SCOTT T2 TABLE USERS 0 4 552 65536 8 4

SCOTT T2 TABLE USERS 1 4 560 65536 8 4

SCOTT T2 TABLE USERS 2 4 672 65536 8 4

SCOTT T2 TABLE USERS 3 4 680 65536 8 4

SCOTT@book> select count(*) from t2 where DBMS_ROWID.ROWID_BLOCK_NUMBER(rowid) between 555 and 556;

COUNT(*)

----------

458

--//458,差1条与执行计划统计.可以推测仅仅扫描读取了数据块555,556,557块.

7.执行计划包含hash join anti SNA/hash join right anti SNA:

SCOTT@book> select * from t1 where id not in (select id from t2 where id is not null) ;

Plan hash value: 1270581391

| 0 | SELECT STATEMENT | | 1 | | | 14 (100)| | 1 |00:00:00.01 | 36 | | | |

|* 1 | HASH JOIN ANTI SNA| | 1 | 2 | 24 | 14 (0)| 00:00:01 | 1 |00:00:00.01 | 36 | 1888K| 1888K| 1008K (0)|

|* 3 | TABLE ACCESS FULL| T2 | 1 | 10004 | 40016 | 10 (0)| 00:00:01 | 10004 |00:00:00.01 | 30 | | | |

--//注:当限制T2查询条件id is not null,这样查询仅仅检查T1表的id是否存在NULL.也就是SNA.

--//SNA 表示 Single Null-Aware ,也就是检查左关联表连接条件是否存在null.

--//EXECUTE SYS.DBMS_STATS.SET_TABLE_STATS (OWNNAME=>'SCOTT', TABNAME=>'T1', NUMBLKS=> 800000);

--//EXECUTE SYS.DBMS_STATS.SET_TABLE_STATS (OWNNAME=>'SCOTT', TABNAME=>'T1', NUMROWS=> 8000);

--//交换顺序:

SCOTT@book> select /*+ SWAP_JOIN_INPUTS(@"SEL$5DA710D3" "T2"@"SEL$2") */ * from t1 where id not in (select id from t2 where id is not null) ;

Plan hash value: 1521920066

------------------------------------------------------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | | | 14 (100)| | 1 |00:00:00.01 | 37 | | | |

|* 1 | HASH JOIN RIGHT ANTI SNA| | 1 | 2 | 24 | 14 (0)| 00:00:01 | 1 |00:00:00.01 | 37 | 2440K| 2440K| 1900K (0)|

|* 2 | TABLE ACCESS FULL | T2 | 1 | 10004 | 40016 | 10 (0)| 00:00:01 | 10004 |00:00:00.01 | 30 | | | |

| 3 | TABLE ACCESS FULL | T1 | 1 | 5 | 40 | 4 (0)| 00:00:01 | 5 |00:00:00.01 | 7 | | | |

总结:

1.NA仅仅判断右关联表查询连接条件是否为NULL.

2.SNA仅仅判断左关联表查询连接条件是否为NULL.

3.注意NULL判断,NULL的逻辑仅仅存在 NULL is NULL 是true,NULL is not NULL是false,其它NULL=NULL之类的都表示noknown.

4.仅仅hash join支持右连接模式.

5.注意仅仅not in,not exists在执行计划中才存在NA,SNA.

6.个人建议,使用exists/not exists代替in/not in,大部分满足业务与实际查询的需要.

7.适当给字段加入not null,规避一些全表扫描,因为全NULL值,在oracle中的常规索引不记录,导致无法使用索引.

8.从以上测试,可以发现hash join派生许多操作方式:

HASH JOIN,HASH JOIN OUTER/HASH JOIN RIGHT OUTER,

HASH JOIN ANTI/HASH JOIN RIGHT ANTI,

HASH JOIN SEMI/HASH JOIN RIGHT SEMI,

HASH JOIN FULL OUTER,

HASH JOIN ANTI NA/HASH JOIN RIGHT ANTI NA,

HASH JOIN ANTI SNA/HASH JOIN RIGHT ANTI SNA.

9.以上完全是基于测试的猜测,不知道是否正确.

10.补充使用exists/not exists的测试:(12c)

SCOTT@test01p> @ ver1

PORT_STRING VERSION BANNER CON_ID

------------------------------ -------------- -------------------------------------------------------------------------------- ----------

IBMPC/WIN_NT64-9.1.0 12.1.0.1.0 Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production 0

SCOTT@test01p> select * from t1 where not exists (select id from t2 where t2.id=t1.id) ;

| 0 | SELECT STATEMENT | | 1 | | | 14 (100)| | 2 |00:00:00.02 | 38 | | | |

|* 1 | HASH JOIN ANTI | | 1 | 2 | 24 | 14 (0)| 00:00:01 | 2 |00:00:00.02 | 38 | 1888K| 1888K| 1040K (0)|

| 2 | TABLE ACCESS FULL| T1 | 1 | 5 | 40 | 4 (0)| 00:00:01 | 5 |00:00:00.01 | 7 | | | |

| 3 | TABLE ACCESS FULL| T2 | 1 | 10005 | 40020 | 10 (0)| 00:00:01 | 10005 |00:00:00.01 | 31 | | | |

--//注意条件是t2.id=t1.id,这样T2.id是null的记录被排除了.注意与使用not in结果不同:

SCOTT@test01p> select * from t1 where id not in (select id from t2 where id is not null) ;

--//如果使用exists这样的效果.

SCOTT@test01p> select * from t1 where not exists (select id from t2 where t2.id=t1.id) and id is not null;

--//exists

SCOTT@test01p> select * from t1 where exists (select id from t2 where t2.id=t1.id) ;

| 0 | SELECT STATEMENT | | 1 | | | 14 (100)| | 3 |00:00:00.01 | 39 | | | |

|* 1 | HASH JOIN SEMI | | 1 | 4 | 48 | 14 (0)| 00:00:01 | 3 |00:00:00.01 | 39 | 1888K| 1888K| 1040K (0)|

| 3 | TABLE ACCESS FULL| T2 | 1 | 10005 | 40020 | 10 (0)| 00:00:01 | 10005 |00:00:00.01 | 32 | | | |