檢視host/container veth pair 關系

1 問題背景

容器中的eth0實際上和外面host上的某個veth是pair關系，然後通過bridge，如docker0 實作在一個host上容器間通信。但是有沒有一個辦法可以知道host上的vethxxx,到底是和哪個container eth0 是pair關系呢 ？

2 實作思路

2.1 思路1 

容器裡面，檢視

# cat /sys/class/net/eth0/iflink 5

host上 周遊/sys/claas/net下面的全部目錄檢視子目錄ifindex的值和容器裡面查出來的iflink值相當的veth名字，也就是：veth63a89a3。這樣就找到了container 和 veth pair的關系。

# cat /sys/class/net/veth63a89a3/ifindex 5

2.1 思路2 

容器裡執行，ip link show eth0 指令，然後可以看到 116: [email protected]，其中116是eth0接口的index, 117是和他pair的veth的index

~ $ ip link show eth0 116: [email protected]: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP link/ether 02:42:0a:00:04:08 brd ff:ff:ff:ff:ff:ff

在host執行下面指令可以看到對應117的vethinterface是哪一個，這樣就得到了container和veth pair關系

# ip link show | grep 117 117: [email protected]: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue master docker0 state UP

3. 代碼實作

不管那種思路，總有一個問題，要能進入到容器中，進入容器有一個比較友善的方法是用nscenter 。因為nscenter安裝比較煩，需要下載下傳源代碼然後編譯等，這裡直接将編譯好的nscenter 編譯出docker image，運作container的時候，會把相關的檔案copy 到/user/bin/local 下面。nscenter 參考了： https://github.com/jpetazzo/nsenter

dockerveth 具體代碼如下，代碼思路蠻簡單的，主要是要差別不同的網絡模式。

# /bin/bash get_network_mode() { docker inspect --format='{{.HostConfig.NetworkMode}}' "$1" } created_by_kubelet() { [[ $(docker inspect --format='{{.Name}}' "$1") =~ ^/k8s_ ]] } for container_name in $(docker ps --format '{{.Names}}'); do container_id=$(docker ps -aqf "name=${container_name}") network_mode=$(get_network_mode "${container_id}") # skip the containers whose network_mode is 'host' or 'none', # but do NOT skip the container created by kubelet. if [[ "${network_mode}" == "host" || \ $(! created_by_kubelet "${container_id}") && "${network_mode}" == "none" ]]; then echo "${container_id} => ${network_mode}" continue fi # if one container's network_mode is 'other container', # then get its root parent container's network_mode. while grep container <<< "${network_mode}" -q; do network_mode=$(get_network_mode "${network_mode/container:/}") # skip the containers whose network_mode is 'host' or 'none', # but do NOT skip the container created by kubelet. if [[ "${network_mode}" == "host" || \ $(! created_by_kubelet "${container_id}") && "${network_mode}" == "none" ]]; then echo "${container_id} => ${network_mode}" continue 2 fi done # get current container's 'container_id'. pid=$(docker inspect --format='{{.State.Pid}}' "${container_id}") # get the 'id' of veth device in the container. veth_id=$(nsenter -t "${pid}" -n ip link show eth0 |grep -oP '(?<[email protected])\d+(?=:)') # get the 'name' of veth device in the 'docker0' bridge (or other name), # which is the peer of veth device in the container. veth_name=$(ip link show |sed -nr "s/^${veth_id}: *([^ ]*)@if.*/\1/p") echo "${container_id} => ${veth_name} => ${container_name}" done

編譯Dockefile

FROM jpetazzo/nsenter ADD installer /installer ADD dockerveth /dockerveth

installer 是：

#!/bin/sh if mountpoint -q /target; then echo "Installing nsenter to /target" cp /nsenter /target echo "Installing docker-enter to /target" cp /docker-enter /target echo "Installing importenv to /target" cp /importenv /target echo "Installing dockerveth to /target" cp /dockerveth /target else echo "/target is not a mountpoint." echo "You can either:" echo "- re-run this container with -v /usr/local/bin:/target" echo "- extract the nsenter binary (located at /nsenter)" fi

4. 如何用

only run once below command by root: 

# docker run --rm -v /usr/local/bin:/target shufanhao/dockerveth:v1.0

然後需要的時候運作下dockerveth：

[email protected]:~/veth# dockerveth CONTAINER ID VETH NAMES 3b16162ea818 veth145042b dib_agent_jpaas f501a753a9da veth0a4537c dib_service_jpaas de653c67370c vethd28f2ef dib_jpaas_redis cde9e15f0f80 veth13e141e dib_agent_test 0eba4d6fc7c2 vethdd2c3d7 dib_service_test

5 Issue

1. permission問題如下：

   nsenter: cannot open /proc/2609/ns/net: Permission denied

   解決： sudo dockerveth  // 用root賬戶運作
2.  腳本執行錯誤如下：

   Syntax error: redirection unexpected

   解決：bash dockerveth