Innodb undo之 undo實體結構的初始化

水準有限，如果有誤請指出。

一直以來未對Innodb 的undo進行好好的學習，最近剛好有點時間準備學習一下，通過阿裡核心月報和自己看代碼的綜合總結一下。本文環境：

代碼版本 percona 5.7.22
參數 innodb_undo_tablespaces = 4 及使用了4個undo tablespace
參數 innodb_rollback_segments = 128

本文描述使用如上參數的設定。

一、undo 表空間實體檔案的建立

本過程調用函數srv_undo_tablespaces_init進行，棧幀如下：

#0  srv_undo_tablespaces_init (create_new_db=true, n_conf_tablespaces=4, n_opened=0x2ef55b0)
    at /root/mysqlc/percona-server-locks-detail-5.7.22/storage/innobase/srv/srv0start.cc:824
#1  0x0000000001bbd7e0 in innobase_start_or_create_for_mysql () at /root/mysqlc/percona-server-locks-detail-5.7.22/storage/innobase/srv/srv0start.cc:2188
#2  0x00000000019ca74e in innobase_init (p=0x2f2a420) at /root/mysqlc/percona-server-locks-detail-5.7.22/storage/innobase/handler/ha_innodb.cc:4409
#3  0x0000000000f7ec2a in ha_initialize_handlerton (plugin=0x2fca110) at /root/mysqlc/percona-server-locks-detail-5.7.22/sql/handler.cc:871
#4  0x00000000015f9edf in plugin_initialize (plugin=0x2fca110) at /root/mysqlc/percona-server-locks-detail-5.7.22/sql/sql_plugin.cc:1252

本過程主要有如下幾個步驟：

根據參數innodb_undo_tablespaces 的配置通過調用srv_undo_tablespace_create分别進行檔案建立，預設建立的大小為10M：

for (i = 0; create_new_db && i < n_conf_tablespaces; ++i) //n_conf_tablespaces 為innodb_undo_tablespaces的配置的個數

/** Default undo tablespace size in UNIV_PAGEs count (10MB). */
const ulint SRV_UNDO_TABLESPACE_SIZE_IN_PAGES =
    ((1024 * 1024) * 10) / UNIV_PAGE_SIZE_DEF;
...
        err = srv_undo_tablespace_create(
            name, SRV_UNDO_TABLESPACE_SIZE_IN_PAGES); //建立undo檔案
...

本步驟會有一個注釋如下：

/* Create the undo spaces only if we are creating a new
    instance. We don't allow creating of new undo tablespaces
    in an existing instance (yet).  This restriction exists because
    we check in several places for SYSTEM tablespaces to be less than
    the min of user defined tablespace ids. Once we implement saving
    the location of the undo tablespaces and their space ids this
    restriction will/should be lifted. */

簡單的講就是建立undo tablespace隻能在初始化執行個體的時候，因為space id已經固定了。

分别對4個undo tablespace調用srv_undo_tablespace_open 其主要調用fil_space_create 和 fil_node_create将建立立的undo tablespace加入Innodb的檔案體系。

for (i = 0; i < n_undo_tablespaces; ++i) {
....
err = srv_undo_tablespace_open(name, undo_tablespace_ids[i]); //打開UNDO檔案 建立 file node
...
}

分别對4個undo tablespace 進行fsp header初始化

for (i = 0; i < n_undo_tablespaces; ++i) {

            fsp_header_init( //初始化fsp header 明顯 space id 已經寫入
                undo_tablespace_ids[i],
                SRV_UNDO_TABLESPACE_SIZE_IN_PAGES, &mtr); //SRV_UNDO_TABLESPACE_SIZE_IN_PAGES 預設的undo大小 10MB
        }

其中fsp_header_init部分代碼如下：

mlog_write_ulint(header + FSP_SPACE_ID, space_id, MLOG_4BYTES, mtr);
    mlog_write_ulint(header + FSP_NOT_USED, 0, MLOG_4BYTES, mtr);

    mlog_write_ulint(header + FSP_SIZE, size, MLOG_4BYTES, mtr);
    mlog_write_ulint(header + FSP_FREE_LIMIT, 0, MLOG_4BYTES, mtr);
    mlog_write_ulint(header + FSP_SPACE_FLAGS, space->flags,
             MLOG_4BYTES, mtr);
    mlog_write_ulint(header + FSP_FRAG_N_USED, 0, MLOG_4BYTES, mtr);

    flst_init(header + FSP_FREE, mtr);
    flst_init(header + FSP_FREE_FRAG, mtr);
    flst_init(header + FSP_FULL_FRAG, mtr);
    flst_init(header + FSP_SEG_INODES_FULL, mtr);
    flst_init(header + FSP_SEG_INODES_FREE, mtr);

這些都是fsp的内容。

做完這個步驟隻是生成了4個大小為10MB的 undo tablespace檔案，并且已經加入到Innodb檔案體系，但是裡面沒有任何類容。

二、ibdata中system segment header的初始化

本步驟調用 trx_sys_create_sys_pages->trx_sysf_create進行，本步驟除了初始化transaction system segment以外還會初始化其header( ibdata page no 5))資訊如下：

/* Create the trx sys file block in a new allocated file segment */
    block = fseg_create(TRX_SYS_SPACE, 0, TRX_SYS + TRX_SYS_FSEG_HEADER,
                mtr); //建立segment
    buf_block_dbg_add_level(block, SYNC_TRX_SYS_HEADER);

    ut_a(block->page.id.page_no() == TRX_SYS_PAGE_NO);

    page = buf_block_get_frame(block); //擷取記憶體位置

    mlog_write_ulint(page + FIL_PAGE_TYPE, FIL_PAGE_TYPE_TRX_SYS, //寫入block 的類型
             MLOG_2BYTES, mtr);

    ...
    /* Start counting transaction ids from number 1 up */
    mach_write_to_8(sys_header + TRX_SYS_TRX_ID_STORE, 1); // 初始化TRX_SYS_TRX_ID_STORE

    /* Reset the rollback segment slots.  Old versions of InnoDB
    define TRX_SYS_N_RSEGS as 256 (TRX_SYS_OLD_N_RSEGS) and expect
    that the whole array is initialized. */
    ptr = TRX_SYS_RSEGS + sys_header;
    len = ut_max(TRX_SYS_OLD_N_RSEGS, TRX_SYS_N_RSEGS)
        * TRX_SYS_RSEG_SLOT_SIZE;//TRX_SYS_OLD_N_RSEGS 為256個
    memset(ptr, 0xff, len); //将slot的資訊的全部初始化為ff
    ptr += len;
    ut_a(ptr <= page + (UNIV_PAGE_SIZE - FIL_PAGE_DATA_END));

    /* Initialize all of the page.  This part used to be uninitialized. */
    memset(ptr, 0, UNIV_PAGE_SIZE - FIL_PAGE_DATA_END + page - ptr); //将剩下的空間設定為0x00

    mlog_log_string(sys_header, UNIV_PAGE_SIZE - FIL_PAGE_DATA_END
            + page - sys_header, mtr);

    /* Create the first rollback segment in the SYSTEM tablespace */
    slot_no = trx_sysf_rseg_find_free(mtr, false, 0);
    page_no = trx_rseg_header_create(TRX_SYS_SPACE, univ_page_size,
                     ULINT_MAX, slot_no, mtr); //将第一個slot固定在ibdata中

完成了這一步過後ibdata的 block 5 就初始化完了，而且我們看到所有的rollback segment slots 都初始化完成(源碼所示有256個，實際上最多隻會有128個，其中0号solt固定在ibdata中)，注意這裡的槽大小是TRX_SYS_RSEG_SLOT_SIZE設定的大小為8位元組，4位元組space id ,4位元組 page no，它們會指向 rollback segment header所在的位置。

下面是system segment header的定位：

/** Transaction system header */
/*------------------------------------------------------------- @{ */
#define    TRX_SYS_TRX_ID_STORE    0    /*!< the maximum trx id or trx
                    number modulo
                    TRX_SYS_TRX_ID_UPDATE_MARGIN
                    written to a file page by any
                    transaction; the assignment of
                    transaction ids continues from
                    this number rounded up by
                    TRX_SYS_TRX_ID_UPDATE_MARGIN
                    plus
                    TRX_SYS_TRX_ID_UPDATE_MARGIN
                    when the database is
                    started */  //最大的事物ID，下次執行個體啟動會加上TRX_SYS_TRX_ID_UPDATE_MARGIN啟動
#define TRX_SYS_FSEG_HEADER    8    /*!< segment header for the
                    tablespace segment the trx
                    system is created into */
#define    TRX_SYS_RSEGS        (8 + FSEG_HEADER_SIZE)
                    /*!< the start of the array of
                    rollback segment specification
                    slots *///指向rollback segment header的槽
/*------------------------------------------------------------- @} */

三、進行rollback segment header的初始化

調用 trx_sys_create_rsegs進行：

說明一下關于innodb_undo_logs參數和innodb_rollback_segments參數，他們作用就是設定rollback segment 的個數，本文以128為例。

根據注釋和代碼innodb_undo_logs已經是個淘汰的參數，應該用innodb_rollback_segments代替。

這兩個參數預設是就是TRX_SYS_N_RSEGS及 128 其實不用設定的。本文也用128進行讨論。

參數 innodb_rollback_segments

static MYSQL_SYSVAR_ULONG(rollback_segments, srv_rollback_segments,
  PLUGIN_VAR_OPCMDARG,
  "Number of rollback segments to use for storing undo logs.",
  NULL, NULL,
  TRX_SYS_N_RSEGS,    /* Default setting */
  1,            /* Minimum value */
  TRX_SYS_N_RSEGS, 0);    /* Maximum value */

參數 innodb_undo_logs

static MYSQL_SYSVAR_ULONG(undo_logs, srv_undo_logs,
  PLUGIN_VAR_OPCMDARG,
  "Number of rollback segments to use for storing undo logs. (deprecated)",
  NULL, innodb_undo_logs_update,
  TRX_SYS_N_RSEGS,    /* Default setting */
  1,            /* Minimum value */
  TRX_SYS_N_RSEGS, 0);    /* Maximum value */

TRX_SYS_N_RSEGS 就是128

下面是注釋和代碼

/* Deprecate innodb_undo_logs.  But still use it if it is set to
    non-default and innodb_rollback_segments is default. */

    if (srv_undo_logs < TRX_SYS_N_RSEGS) {
        ib::warn() << deprecated_undo_logs;
        if (srv_rollback_segments == TRX_SYS_N_RSEGS) {
            srv_rollback_segments = srv_undo_logs;
        }
    }

初始化rollback segments 段

n_noredo_created = trx_sys_create_noredo_rsegs(n_tmp_rsegs); //建立 32個 臨時rollback segments

我們這裡不準備考慮臨時rollback segments

建立 95個(33-128) 普通rollback segments

ulint    new_rsegs = n_rsegs - n_used; //eg:128 -33 = 95

        for (i = 0; i < new_rsegs; ++i) { //對每個rollback segment進行初始化
            ulint    space_id;
            space_id = (n_spaces == 0) ? 0
                : (srv_undo_space_id_start + i % n_spaces); //擷取 undo space_id 采用 取模的方式循環初始化 1 2 3 4

            ut_ad(n_spaces == 0
                  || srv_is_undo_tablespace(space_id));

            if (trx_rseg_create(space_id, 0) != NULL)

我們能夠注意到這裡是i % n_spaces的取模方式n_spaces為我們innodb_undo_tablespaces參數設定的值，是以每個rollback segment 是輪序的方式分布到4個不同的undo tablespace中的。

具體的rollback segment header初始化過程

如上是trx_rseg_create調用trx_rseg_header_create完成的。步驟大概如下：

1、建立rollback segment

block = fseg_create(space, 0, TRX_RSEG + TRX_RSEG_FSEG_HEADER, mtr); //建立一個復原段，傳回段頭所在的塊

2、初始化TRX_RSEG_MAX_SIZE和TRX_RSEG_HISTORY_SIZE資訊

/* Initialize max size field */
    mlog_write_ulint(rsegf + TRX_RSEG_MAX_SIZE, max_size,
             MLOG_4BYTES, mtr);

    /* Initialize the history list */

    mlog_write_ulint(rsegf + TRX_RSEG_HISTORY_SIZE, 0, MLOG_4BYTES, mtr);
    flst_init(rsegf + TRX_RSEG_HISTORY, mtr);

3、初始化每個undo segment header所在的page no

for (i = 0; i < TRX_RSEG_N_SLOTS; i++) { //TRX_RSEG_N_SLOTS 為1024 初始化每個槽 值為 4位元組指向 undo segment header的page no

        trx_rsegf_set_nth_undo(rsegf, i, FIL_NULL, mtr);
    }

初始化的情況下我們看到指向的page no都是 FIL_NULL，說明沒有配置設定任何實際的undo segment。

4、整個rollback segment 初始化完成後将space id和page no 寫回到 transaction system segment header中。

sys_header = trx_sysf_get(mtr); //擷取 5号 block指針 跳過  FIL_PAGE_DATA 38U 

trx_sysf_rseg_set_space(sys_header, rseg_slot_no, space, mtr); //設定space

trx_sysf_rseg_set_page_no(sys_header, rseg_slot_no, page_no, mtr); //設定 no

下面是 rollback segment header的結構

/* Transaction rollback segment header */
/*-------------------------------------------------------------*/
#define    TRX_RSEG_MAX_SIZE    0    /* Maximum allowed size for rollback
                    segment in pages */
#define    TRX_RSEG_HISTORY_SIZE    4    /* Number of file pages occupied
                    by the logs in the history list */ //history 連結清單大小
#define    TRX_RSEG_HISTORY    8    /* The update undo logs for committed
                    transactions */ //連結清單頭base node  他們通常調用include/fut0lst.ic中的函數進行更改
#define    TRX_RSEG_FSEG_HEADER    (8 + FLST_BASE_NODE_SIZE)
                    /* Header for the file segment where
                    this page is placed */
#define TRX_RSEG_UNDO_SLOTS    (8 + FLST_BASE_NODE_SIZE + FSEG_HEADER_SIZE)
                    /* Undo log segment slots */ //
/*-------------------------------------------------------------*/

作為 base node的 TRX_RSEG_HISTORY我們可以看到定義如下

/* We define the field offsets of a base node for the list */
#define FLST_LEN    0    /* 32-bit list length field */
#define    FLST_FIRST    4    /* 6-byte address of the first element
                of the list; undefined if empty list */
#define    FLST_LAST    (4 + FIL_ADDR_SIZE) /* 6-byte address of the
                last element of the list; undefined
                if empty list */

#define FIL_ADDR_PAGE    0    /* first in address is the page offset */
#define    FIL_ADDR_BYTE    4    /* then comes 2-byte byte offset within page*/
#endif /* !UNIV_INNOCHECKSUM */
#define    FIL_ADDR_SIZE    6    /* address size is 6 bytes */

多了一個長度

到這裡128 rollback segment已經初始化完成，并且每個都包含1024個 undo segment slots。

四、整個過程初始化完成後的分布圖

為了讓圖更加美觀和好了解，我這裡使用的是innodb_undo_tablespaces=2的情況下作圖，也就是隻有2個 undo tablespace的情況。其實4個也是同樣的道理，因為rollback segment slot是輪詢在表空間配置設定的。

最終我們看到初始化完成後undo segment slot指向的都是FIL_NULL，及沒有指向，當實際配置設定的時候這些slot就會指向我們的undo segment header。

同時我們可以看看undotablespace到底包含哪些類型塊，使用自制的小工具讀取如下：

./myblock undo001 -d|more

current read blocks is : 0 --This Block is file space header blocks!
current read blocks is : 1 --This Block is insert buffer bitmap  blocks!
current read blocks is : 2 --This Block is inode blocks!
current read blocks is : 3 --This Block is system   blocks!
current read blocks is : 4 --This Block is system   blocks!
current read blocks is : 5 --This Block is system   blocks!
current read blocks is : 6 --This Block is system   blocks!
current read blocks is : 7 --This Block is system   blocks!
current read blocks is : 8 --This Block is system   blocks!
current read blocks is : 9 --This Block is system   blocks!
current read blocks is : 10 --This Block is system   blocks!
current read blocks is : 11 --This Block is system   blocks!
current read blocks is : 12 --This Block is system   blocks!
current read blocks is : 13 --This Block is system   blocks!
current read blocks is : 14 --This Block is system   blocks!
current read blocks is : 15 --This Block is system   blocks!
current read blocks is : 16 --This Block is system   blocks!
current read blocks is : 17 --This Block is system   blocks!
current read blocks is : 18 --This Block is system   blocks!
current read blocks is : 19 --This Block is system   blocks!
current read blocks is : 20 --This Block is system   blocks!
current read blocks is : 21 --This Block is system   blocks!
current read blocks is : 22 --This Block is system   blocks!
current read blocks is : 23 --This Block is system   blocks!
current read blocks is : 24 --This Block is system   blocks!
current read blocks is : 25 --This Block is system   blocks!
current read blocks is : 26 --This Block is system   blocks!
current read blocks is : 27 --This Block is undo blocks!
current read blocks is : 28 --This Block is undo blocks!
current read blocks is : 29 --This Block is undo blocks!
current read blocks is : 30 --This Block is undo blocks!
current read blocks is : 31 --This Block is undo blocks!
current read blocks is : 32 --This Block is undo blocks!
current read blocks is : 33 --This Block is undo blocks!
current read blocks is : 34 --This Block is undo blocks!
current read blocks is : 35 --This Block is undo blocks!
current read blocks is : 36 --This Block is undo blocks!
current read blocks is : 37 --This Block is undo blocks!
current read blocks is : 38 --This Block is new allocate blocks!
current read blocks is : 39 --This Block is new allocate blocks!
current read blocks is : 40 --This Block is new allocate blocks!
current read blocks is : 41 --This Block is new allocate blocks!
current read blocks is : 42 --This Block is new allocate blocks!

這裡 block3-block26 就是我們的rollback segment header block。我這裡當然是 4個undo tablespace的情況，看的是undo tablespace 1。看來沒有問題。分析正确。

五、總結

普通的undo segment的關聯方式是：ibdata的block 5 system segment header通過33-128這些 rollback segment slot 輪詢指向不同的undo tablespace 的rollback segment header，然後每個rollback segment header中有1024個slot來指向實際的undo segment header，來實作的。實際的undo block會挂載到undo segment header下的連結清單中。
undo tablespaces數量的變化隻能通過重新初始化執行個體來改變，space id是固定了，是以要考慮清楚
innodb_undo_tablespaces是undo tablespace的數量而innodb_rollback_segments是 rollback segment的數量，參數innodb_undo_logs已經過時了，它和innodb_rollback_segments是同樣的功能，預設他們都是128
rollback segment slot 0 固定在 ibdata中，而 rollback segment slot 1-32 為臨時rollback segment，33-128才是普通事物的rollback segment。

參考文獻：

http://mysql.taobao.org/monthly/2015/04/01/

阿裡核心月報

Innodb undo之 undo實體結構的初始化

水準有限，如果有誤請指出。

一、undo 表空間實體檔案的建立

二、ibdata中system segment header的初始化

三、進行rollback segment header的初始化

四、整個過程初始化完成後的分布圖

五、總結

繼續閱讀

oracle中的start with connect by用法

JDBC連接配接資料庫（statement）

MyBatis-Plus 之AR模式

ASP程式設計經典例子

ASP程式設計中20個非常有用的例子

龍珠訓練營task04

阿裡雲天池龍珠計劃SQL訓練營打卡

阿裡雲天池龍珠計劃SQL訓練營day1

實驗樓sql進階之成績管理系統的資料操作(window)

Oracle的基本操作

SQL優化SQL語句優化的目的

JAVA高效程式設計指南

關于SQL語言

SQL語言基礎：常用的資料查詢語句

neo4j之cypher使用文檔

sqlServer根據經緯查距離