![](https://img.laitimes.com/img/9ZDMuAjOiMmIsIjOiQnIsIyZuBnLzkTN2cDNwYDNtADO5gjM3UzNxMTM4ADMyAjMtIzM1ITO48CX4ADMyAjMvwlMzUjM5gzLcd2bsJ2Lc12bj5ycn9Gbi52YuAjMwIzZtl2Lc9CX6MHc0RHaiojIsJye.png)
向导地址
https://debezium.io/documentation/reference/1.2/tutorial.html
1.启动kafka connector
$ docker run -it --rm --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my_connect_configs -e OFFSET_STORAGE_TOPIC=my_connect_offsets -e STATUS_STORAGE_TOPIC=my_connect_statuses --link zookeeper:zookeeper --link kafka:kafka --link mysql:mysql debezium/connect:1.2
按照文档说法,运行下面命令可见结果
$ curl -H "Accept:application/json" localhost:8083/
{"version":"2.5.0","commit":"cb8625948210849f"}
但发现绑定8083不成功,定位半天发现connector绑定在pod ip的8083端口去了。而且没有映射到主机上来,进入容器运行成功
[kafka@f3211e56f0ba ~]$ curl -H "Accept:application/json" http://172.17.0.5:8083/
{"version":"2.5.0","commit":"66563e712b0b9f84","kafka_cluster_id":"2r5ESXyUS_W1IvOmZVF3zg"}
2.部署mysql Connector
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" 172.17.0.5:8083/connectors/ -d '{ "name": "inventory-connector", "config": { "connector.class": "io.debezium.connector.mysql.MySqlConnector", "tasks.max": "1", "database.hostname": "mysql", "database.port": "3306", "database.user": "debezium", "database.password": "dbz", "database.server.id": "184054", "database.server.name": "dbserver1", "database.whitelist": "inventory", "database.history.kafka.bootstrap.servers": "kafka:9092", "database.history.kafka.topic": "dbhistory.inventory" } }'
这里dbserver1是指mysql server的逻辑识别id,这个值会作为kafka topic的前缀。(prefix)
3.查看变化
基于watch-topic工具查看各个队列中的情况
$ docker run -it --rm --name watcher --link zookeeper:zookeeper --link kafka:kafka debezium/kafka:1.2 watch-topic -a -k dbserver1.inventory.customers
不同的表的变化放到不同的队列中。
dbserver1.inventory.customers对应着mysql的customers表格
dbserver1.inventory.addresses对应着mysql的addresses表格
。。。。
因为变化是一串String可阅读性不强,比如更新的如下
{"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.customers.Key"},"payload":{"id":1004}} {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"first_name"},{"type":"string","optional":false,"field":"last_name"},{"type":"string","optional":false,"field":"email"}],"optional":true,"name":"dbserver1.inventory.customers.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"first_name"},{"type":"string","optional":false,"field":"last_name"},{"type":"string","optional":false,"field":"email"}],"optional":true,"name":"dbserver1.inventory.customers.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"int64","optional":false,"field":"server_id"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"dbserver1.inventory.customers.Envelope"},"payload":{"before":{"id":1004,"first_name":"Anne Marie","last_name":"Kretchmar","email":"[email protected]"},"after":{"id":1004,"first_name":"Anne","last_name":"Kretchmar","email":"[email protected]"},"source":{"version":"1.2.1.Final","connector":"mysql","name":"dbserver1","ts_ms":1597310172000,"snapshot":"false","db":"inventory","table":"customers","server_id":223344,"gtid":null,"file":"mysql-bin.000003","pos":1066,"row":0,"thread":9,"query":null},"op":"u","ts_ms":1597310172015,"transaction":null}}
可以通过在线的工具codebeautify进行查看,最主要关注payload部分。
https://codebeautify.org/jsonviewer
"payload": {
"before": {
"id": 1004,
"first_name": "Anne Marie",
"last_name": "Kretchmar",
"email": "[email protected]"
},
"after": {
"id": 1004,
"first_name": "Anne",
"last_name": "Kretchmar",
"email": "[email protected]"
},
"source": {
"version": "1.2.1.Final",
"connector": "mysql",
"name": "dbserver1",
"ts_ms": 1597310172000,
"snapshot": "false",
"db": "inventory",
"table": "customers",
"server_id": 223344,
"gtid": null,
"file": "mysql-bin.000003",
"pos": 1066,
"row": 0,
"thread": 9,
"query": null
},
"op": "u",
"ts_ms": 1597310172015,
"transaction": null
}