閱讀xtrabackup代碼的一點筆記

xtrabackup

binary最重要的兩個過程是backup和prepare，對應的函數分别是xtrabackup_backup_func()和xtrabackup_prepare_func()，這裡做一些閱讀代碼時的筆記。

xtrabackup backup的線程模型：

1. 一個log拷貝線程；

2. n個ibd檔案拷貝線程；

3. 一個io監控線程；

4. 通過suspend_start/suspend_end檔案來标注是否啟動終止線程；

typedef struct {

    datafiles_iter_t *it;

    uint num;

    uint *count;

    os_ib_mutex_t  count_mutex;

    os_thread_id_t  id;

}data_thread_ctxt_t;

資料線程上下文

切換工作目錄；

 116 /** set if innodb must operate in read-only mode. we don‘t do

any

 117 recovery and open all tables in ro mode instead of rw mode. we

don‘t

 118 sync the max trx id to disk either. */

xb_set_innodb_read_only()  将innodb設成隻讀模式

srv_backup_mode=true;  将innodb設成backup模式；

設定innodb的一系列參數

innodb_init_param()

xb_normalize_init_values(void)

修改srv_unix_file_flush_method

根據bp大小，調整srv_max_n_threads參數

1017

/*********************************************************************//**

1018 initializes the synchronization primitives, memory system, and the

thread

1019 local storage. */

srv_general_init()

ut_crc32_init()

xb_filters_init()

2567

/************************************************************************

2568 initializes the i/o and tablespace cache subsystems. */

xb_fil_io_init(void)

 838 /******************************************************//**

 839 initializes the log. */

log_init(void)

 585

 586 creates the lock system at database start. */

lock_sys_create()

open_or_create_log_file

建立xtrabackup_extra_lsndir/extrabackup_traget_dir

表空間memory cache

fil_system_t* f_system = fil_system;

recv_find_max_checkpoint(&max_cp_group, &max_cp_field)

log_group_read_checkpoint_info(max_cp_group, max_cp_field)

checkpoint_lsn_start/checkpoint_no_start

确認一緻的checkpoint狀态；

建立xb_log_filename檔案，寫入檔案頭資訊；

建立io_watching_thread;

從checkpoint位置開始copy log檔案；

xtrabackup_copy_logfile(checkpoint_lsn_start, false)

log_copying/log_copying_stop

建立日志copy線程

os_thread_create(log_copying_thread, null,

&log_copying_thread_id);

2591

/****************************************************************************

2592 populates the tablespace memory cache by scanning for and opening data

files.

2593 @returns db_success or error code.*/

xb_load_tablespaces()

挂起，等待xb_fn_suspended_at_start檔案被删除

xtrabackup_suspend

xb_page_bitmap_init()

根據xtrabackup_parallel設定，建立data_copy_thread_func線程

等待所有data_copy_thread_func線程退出

挂起，等待xb_fn_suspended_at_end檔案被删除

讀取最新的checkpoint， 記錄在metadata的to_lsn字段；

通過設定log_copying=false && set

log_copying_stop，停止log_copying_thread；

建立一個檔案xb_fn_log_copied，通知外部腳本，log_copying_thread已經結束；

寫metadata；

prepare_func

1. 切換到xtrabackup_real_target_dir

2. 讀取xtrabackup_metadata_filename，擷取原資訊；

    metadata_type

3. xtrabackup_init_temp_log()

4. innodb_init_param()

2670

2671 initialize the tablespace memory cache and populate it by scanning for

and

2672 opening data files.

2673 @returns db_success or error code.*/

xb_data_files_init()

應用增量到全量

xtrabackup_apply_deltas()

重設innodb初始化參數

innodb_init()

周遊檔案

mtr_start -> mtr_commit

trx_sys_print_mysql_binlog_offset()

将binlog位置資訊輸出到 xtrabackup_binlog_pos_innodb檔案中

xtrabackup_close_temp_log(true)

輸出記錄metadata_log

backup

$./xtrabackup_56 --defaults-file=/u01/my3928/my.cnf --backup

--target_dir=/u01/xianlin.lh/backup_dir/

./xtrabackup_56 version 2.1.8 for mysql server 5.6.15 linux (x86_64)

(revision id: undefined)

xtrabackup: uses posix_fadvise().

xtrabackup: cd to /u01/my3928/data

xtrabackup: using the following innodb configuration:

xtrabackup: innodb_data_home_dir = /u01/my3928/data

xtrabackup: innodb_data_file_path = ibdata1:4g;ibdata2:16m:autoextend

xtrabackup: innodb_log_group_home_dir = /u01/my3928/data

xtrabackup: innodb_log_files_in_group = 4

xtrabackup: innodb_log_file_size = 1073741824

2014-05-05 17:29:35 2ac06bc4a2c0 innodb: using linux native aio

xtrabackup: using o_direct

>> log scanned up to (1451746590)

[01] copying /u01/my3928/data/ibdata1 to

/u01/xianlin.lh/backup_dir/ibdata1

[01] ...done

[01] copying /u01/my3928/data/ibdata2 to

/u01/xianlin.lh/backup_dir/ibdata2

[01] copying ./test/t2.ibd to /u01/xianlin.lh/backup_dir/test/t2.ibd

[01] copying ./test/t4.ibd to /u01/xianlin.lh/backup_dir/test/t4.ibd

[01] copying ./test/t3.ibd to /u01/xianlin.lh/backup_dir/test/t3.ibd

[01] copying ./test/sbtest1.ibd to

/u01/xianlin.lh/backup_dir/test/sbtest1.ibd

[01] copying ./test/t1.ibd to /u01/xianlin.lh/backup_dir/test/t1.ibd

[01] copying ./mysql/innodb_index_stats.ibd to

/u01/xianlin.lh/backup_dir/mysql/innodb_index_stats.ibd

[01] copying ./mysql/slave_worker_info.ibd to

/u01/xianlin.lh/backup_dir/mysql/slave_worker_info.ibd

[01] copying ./mysql/innodb_table_stats.ibd to

/u01/xianlin.lh/backup_dir/mysql/innodb_table_stats.ibd

[01] copying ./mysql/slave_relay_log_info.ibd to

/u01/xianlin.lh/backup_dir/mysql/slave_relay_log_info.ibd

[01] copying ./mysql/slave_master_info.ibd to

/u01/xianlin.lh/backup_dir/mysql/slave_master_info.ibd

xtrabackup: the latest check point (for incremental): ‘1451746590‘

xtrabackup: stopping log copying thread.

.>> log scanned up to (1451746590)

xtrabackup: transaction log of lsn (1451746590) to (1451746590) was

copied.

建立備份的流程

1 backup_type = full-backuped

2 from_lsn = 0

3 to_lsn = 1451746590

4 last_lsn = 1451746590

5 compact = 0

prepare

$./xtrabackup_56 --defaults-file=/u01/my3928/my.cnf --prepare

--target-dir=/u01/xianlin.lh/backup_dir/

xtrabackup: cd to /u01/xianlin.lh/backup_dir/

xtrabackup: this target seems to be not prepared yet.

xtrabackup: xtrabackup_logfile detected: size=2097152,

start_lsn=(1451746590)

xtrabackup: using the following innodb configuration for recovery:

xtrabackup: innodb_data_home_dir = ./

xtrabackup: innodb_log_group_home_dir = ./

xtrabackup: innodb_log_files_in_group = 1

xtrabackup: innodb_log_file_size = 2097152

2014-05-05 18:38:32 2b7f2f5202c0 innodb: using linux native aio

xtrabackup: starting innodb instance for recovery.

xtrabackup: using 104857600 bytes for buffer pool (set by --use-memory

parameter)

innodb: the innodb memory heap is disabled

innodb: mutexes and rw_locks use gcc atomic builtins

innodb: compressed tables use zlib 1.2.3

innodb: using linux native aio

innodb: using cpu crc32 instructions

innodb: initializing buffer pool, size = 100.0m

innodb: completed initialization of buffer pool

innodb: highest supported file format is barracuda.

innodb: the log sequence numbers 552524932 and 552524932 in ibdata files do

not match the log sequence number 1451746590 in the ib_logfiles!

innodb: database was not shutdown normally!

innodb: starting crash recovery.

innodb: reading tablespace information from the .ibd files...

innodb: restoring possible half-written data pages

innodb: from the doublewrite buffer...

innodb: last mysql binlog file position 0 26767310, file name

mysql-bin.000023

innodb: 128 rollback segment(s) are active.

innodb: waiting for purge to start

2014-05-05 18:38:33 2b7f49056700 innodb: warning: table

‘test/sbtest1‘

innodb: in innodb data dictionary has unknown flags 50.

innodb: 5.6.15 started; log sequence number 1451746590

[notice (again)]

  if you use binary log and don‘t use any hack of group commit,

  the binary log position seems to be:

xtrabackup: starting shutdown with innodb_fast_shutdown = 1

innodb: fts optimize thread exiting.

innodb: starting shutdown...

innodb: shutdown completed; log sequence number 1451747281

prepare的流程：

  1 backup_type = full-prepared

  2 from_lsn = 0

  3 to_lsn = 1451746590

  4 last_lsn = 1451746590

  5 compact = 0