天天看點

mysql pt online ddl_MySQL Online DDL 工具之pt-online-schema-change

MySQL DDL:

DDL是一個令所有MySQL dDBA 诟病的一個功能,因為在MySQL中在對表進行dDDL時,會鎖表,當表比較小比如小于1W行時,對前端影響較小,當時遇到千萬級别的表,就會影響前端應用對表的寫操作!

InnoDB引擎是通過以下步驟來進行DDL的:

1、按照原始表(original_table)的表結構和DDL語句,建立一個不可見的臨時表(tmp_table)

2、在原表上加write lock,阻塞所有更新操作(insert、delete、update等)

3、執行insert into tmp_table select * from original_table

4、rename original_table和tmp_table,最後drop original_table

5、釋放 write lock。

可以看見在InnoDB執行DDL的時候,原表是隻能讀不能寫的。為此 perconal 推出一個工具 pt-online-schema-change ,其特點是修改過程中不會造成讀寫阻塞。

工作原理:

如果表有外鍵,除非使用 --alter-foreign-keys-method 指定特定的值,否則工具不予執行。

1 建立一個和你要執行 alter 操作的表一樣的空表結構。

2 執行表結構修改,然後從原表中的資料到copy到 表結構修改後的表,

3 在原表上建立觸發器将 copy 資料的過程中,在原表的更新操作 更新到新表.

注意:如果表中已經定義了觸發器這個工具就不能工作了。

4 copy 完成以後,用rename table 新表代替原表,預設删除原表。

用法介紹:

pt-online-schema-change [OPTIONS] DSN

options 可以自行檢視 help,DNS 為你要操作的資料庫和表。這裡有兩個參數需要介紹一下:

--dry-run

這個參數不建立觸發器,不拷貝資料,也不會替換原表。隻是建立和更改新表。

--execute

這個參數的作用和前面工作原理的介紹的一樣,會建立觸發器,來保證最新變更的資料會影響至新表。注意:如果不加這個參數,這個工具會在執行一些檢查後退出。

依賴條件:

操作的表必須有主鍵否則 報如下錯誤:

[[email protected] ~]# pt-online-schema-change -u root -plinuxidc  -h127.0.0.1 --alter='add column vname varchar(20)' --execute D=linuxidc,t=linuxidc

No slaves found.  See --recursion-method if host linuxidc.com has slaves.

Not checking slave lag because no slaves were found and --check-slave-lag was not specified.

Operation, tries, wait:

copy_rows, 10, 0.25

create_triggers, 10, 1

drop_triggers, 10, 1

swap_tables, 10, 1

update_foreign_keys, 10, 1

Altering `linuxidc`.`linuxidc`...

Creating new table...

Created new table linuxidc._linuxidc_new OK.

Altering new table...

Altered `linuxidc`.`_linuxidc_new` OK.

2016-01-08T17:51:43 Dropping new table...

2016-01-08T17:51:43 Dropped new table OK.

`linuxidc`.`linuxidc` was not altered.

The new table `linuxidc`.`_linuxidc_new` does not have a PRIMARY KEY or a unique index which is required for the DELETE trigger.

字段vname沒有添加成功!

mysql> show create table linuxidc\G

*************************** 1. row ***************************

Table: linuxidc

Create Table: CREATE TABLE `linuxidc` (

`actor_id` smallint(8) unsigned NOT NULL DEFAULT '0',

`first_name` varchar(45) NOT NULL,

`last_name` varchar(45) NOT NULL,

`last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP

) ENGINE=InnoDB DEFAULT CHARSET=utf8

添加主鍵:

mysql> alter table linuxidc modify actor_id smallint(8) unsigned primary key;

Query OK, 0 rows affected (0.06 sec)

Records: 0  Duplicates: 0  Warnings: 0

[[email protected] ~]# pt-online-schema-change -u root -plinuxidc  -h127.0.0.1 --alter='add column vname varchar(20)' --execute D=linuxidc,t=linuxidc

No slaves found.  See --recursion-method if host linuxidc.com has slaves.

Not checking slave lag because no slaves were found and --check-slave-lag was not specified.

Operation, tries, wait:

copy_rows, 10, 0.25

create_triggers, 10, 1

drop_triggers, 10, 1

swap_tables, 10, 1

update_foreign_keys, 10, 1

Altering `linuxidc`.`linuxidc`...

Creating new table...

Created new table linuxidc._linuxidc_new OK.

Altering new table...

Altered `linuxidc`.`_linuxidc_new` OK.

2016-01-08T17:57:09 Creating triggers...

2016-01-08T17:57:09 Created triggers OK.

2016-01-08T17:57:09 Copying approximately 200 rows...

2016-01-08T17:57:09 Copied rows OK.

2016-01-08T17:57:09 Swapping tables...

2016-01-08T17:57:09 Swapped original and new tables OK.

2016-01-08T17:57:09 Dropping old table...

2016-01-08T17:57:09 Dropped old table `linuxidc`.`_linuxidc_old` OK.

2016-01-08T17:57:09 Dropping triggers...

2016-01-08T17:57:09 Dropped triggers OK.

Successfully altered `linuxidc`.`linuxidc`.

mysql> show create table linuxidc\G

*************************** 1. row ***************************

Table: linuxidc

Create Table: CREATE TABLE `linuxidc` (

`actor_id` smallint(8) unsigned NOT NULL,

`first_name` varchar(45) NOT NULL,

`last_name` varchar(45) NOT NULL,

`last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,

`vname` varchar(20) DEFAULT NULL,

PRIMARY KEY (`actor_id`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8

添加多個字段:

[[email protected] ~]# pt-online-schema-change -u root -plinuxidc  -h127.0.0.1 --alter='add column aname varchar(20),add column bname varchar(30)' --execute D=linuxidc,t=linuxidc

No slaves found.  See --recursion-method if host linuxidc.com has slaves.

Not checking slave lag because no slaves were found and --check-slave-lag was not specified.

Operation, tries, wait:

copy_rows, 10, 0.25

create_triggers, 10, 1

drop_triggers, 10, 1

swap_tables, 10, 1

update_foreign_keys, 10, 1

Altering `linuxidc`.`linuxidc`...

Creating new table...

Created new table linuxidc._linuxidc_new OK.

Altering new table...

Altered `linuxidc`.`_linuxidc_new` OK.

2016-01-08T18:04:25 Creating triggers...

2016-01-08T18:04:25 Created triggers OK.

2016-01-08T18:04:25 Copying approximately 200 rows...

2016-01-08T18:04:25 Copied rows OK.

2016-01-08T18:04:25 Swapping tables...

2016-01-08T18:04:26 Swapped original and new tables OK.

2016-01-08T18:04:26 Dropping old table...

2016-01-08T18:04:26 Dropped old table `linuxidc`.`_linuxidc_old` OK.

2016-01-08T18:04:26 Dropping triggers...

2016-01-08T18:04:26 Dropped triggers OK.

Successfully altered `linuxidc`.`linuxidc`.

mysql> show create table linuxidc\G

*************************** 1. row ***************************

Table: linuxidc

Create Table: CREATE TABLE `linuxidc` (

`actor_id` smallint(8) unsigned NOT NULL,

`first_name` varchar(45) NOT NULL,

`last_name` varchar(45) NOT NULL,

`last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,

`vname` varchar(20) DEFAULT NULL,

`aname` varchar(20) DEFAULT NULL,

`bname` varchar(30) DEFAULT NULL,

PRIMARY KEY (`actor_id`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8

删除字段:

[[email protected] ~]# pt-online-schema-change -u root -plinuxidc  -h127.0.0.1 --alter='drop column aname,drop column bname' --execute D=linuxidc,t=linuxidc

No slaves found.  See --recursion-method if host linuxidc.com has slaves.

Not checking slave lag because no slaves were found and --check-slave-lag was not specified.

Operation, tries, wait:

copy_rows, 10, 0.25

create_triggers, 10, 1

drop_triggers, 10, 1

swap_tables, 10, 1

update_foreign_keys, 10, 1

Altering `linuxidc`.`linuxidc`...

Creating new table...

Created new table linuxidc._linuxidc_new OK.

Altering new table...

Altered `linuxidc`.`_linuxidc_new` OK.

2016-01-08T18:05:45 Creating triggers...

2016-01-08T18:05:45 Created triggers OK.

2016-01-08T18:05:45 Copying approximately 200 rows...

2016-01-08T18:05:45 Copied rows OK.

2016-01-08T18:05:45 Swapping tables...

2016-01-08T18:05:45 Swapped original and new tables OK.

2016-01-08T18:05:45 Dropping old table...

2016-01-08T18:05:45 Dropped old table `linuxidc`.`_linuxidc_old` OK.

2016-01-08T18:05:45 Dropping triggers...

2016-01-08T18:05:46 Dropped triggers OK.

Successfully altered `linuxidc`.`linuxidc`.

添加索引:

[[email protected] ~]# pt-online-schema-change -u root -plinuxidc  -h127.0.0.1 --alter='add key index_first(first_name)' --execute D=linuxidc,t=linuxidc  No slaves found.  See --recursion-method if host linuxidc.com has slaves.

Not checking slave lag because no slaves were found and --check-slave-lag was not specified.

Operation, tries, wait:

copy_rows, 10, 0.25

create_triggers, 10, 1

drop_triggers, 10, 1

swap_tables, 10, 1

update_foreign_keys, 10, 1

Altering `linuxidc`.`linuxidc`...

Creating new table...

Created new table linuxidc._linuxidc_new OK.

Altering new table...

Altered `linuxidc`.`_linuxidc_new` OK.

2016-01-08T18:06:38 Creating triggers...

2016-01-08T18:06:38 Created triggers OK.

2016-01-08T18:06:38 Copying approximately 200 rows...

2016-01-08T18:06:38 Copied rows OK.

2016-01-08T18:06:38 Swapping tables...

2016-01-08T18:06:38 Swapped original and new tables OK.

2016-01-08T18:06:38 Dropping old table...

2016-01-08T18:06:38 Dropped old table `linuxidc`.`_linuxidc_old` OK.

2016-01-08T18:06:38 Dropping triggers...

2016-01-08T18:06:38 Dropped triggers OK.

Successfully altered `linuxidc`.`linuxidc`.

mysql pt online ddl_MySQL Online DDL 工具之pt-online-schema-change