lxd容器内docker無法啟動，報錯open /proc/sys/net/ipv4/ip_forward: read-only file system

問題出現場景

在lxd容器中建立了docker容器，有一天所有lxd容器内的docker容器全都起不來；啟動時報錯

[email protected]:~# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2021-01-27 20:20:21 CST; 52s ago
     Docs: https://docs.docker.com
  Process: 435 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
 Main PID: 435 (code=exited, status=1/FAILURE)

Jan 27 20:20:21 bb systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jan 27 20:20:21 bb systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
Jan 27 20:20:21 bb systemd[1]: Stopped Docker Application Container Engine.
Jan 27 20:20:21 bb systemd[1]: docker.service: Start request repeated too quickly.
Jan 27 20:20:21 bb systemd[1]: docker.service: Failed with result 'exit-code'.
Jan 27 20:20:21 bb systemd[1]: Failed to start Docker Application Container En
           

檢視日志詳情：

[email protected]:~# sudo journalctl -xe

-- Subject: Unit docker.service has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit docker.service has begun starting up.
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.784691543+08:00" level=info msg="systemd-resolved is running, so using resolvconf: /run/systemd/resolve/resolv.conf"
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.787316279+08:00" level=info msg="parsed scheme: \"unix\"" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.787383289+08:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.787547821+08:00" level=info msg="parsed scheme: \"unix\"" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.787583664+08:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.787577781+08:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0  <nil>}]" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.787636606+08:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.787714113+08:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42007f440, CONNECTING" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.787750049+08:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0  <nil>}]" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.787786012+08:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.787876910+08:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4207f54e0, CONNECTING" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.788110267+08:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42007f440, READY" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.788142830+08:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4207f54e0, READY" module=grpc
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.792378731+08:00" level=error msg="'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support l
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.795697602+08:00" level=error msg="AUFS was not found in /proc/filesystems" storage-driver=aufs
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.798731718+08:00" level=error msg="'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support l
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.835798600+08:00" level=info msg="Graph migration to content-addressability took 0.00 seconds"
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.836420988+08:00" level=warning msg="Your kernel does not support swap memory limit"
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.836574929+08:00" level=warning msg="Your kernel does not support cgroup rt period"
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.836612848+08:00" level=warning msg="Your kernel does not support cgroup rt runtime"
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.836650679+08:00" level=warning msg="Your kernel does not support cgroup blkio weight"
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.836687231+08:00" level=warning msg="Your kernel does not support cgroup blkio weight_device"
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.838965475+08:00" level=info msg="Loading containers: start."
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.842786432+08:00" level=warning msg="Running modprobe nf_nat failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.846767605+08:00" level=warning msg="Running modprobe xt_conntrack failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() cou
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.931682527+08:00" level=warning msg="Enabling IP forwarding failed: open /proc/sys/net/ipv4/ip_forward: read-only file system"
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.933652825+08:00" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
Jan 27 20:21:40 bb dockerd[626]: Error starting daemon: Error initializing network controller: error obtaining controller instance: Enabling IP forwarding failed: open /proc/sys/net/ipv4/ip_forward: read-only file
Jan 27 20:21:40 bb dockerd[626]: time="2021-01-27T20:21:40.934417502+08:00" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby
Jan 27 20:21:40 bb systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Jan 27 20:21:40 bb systemd[1]: docker.service: Failed with result 'exit-code'.
Jan 27 20:21:40 bb systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support

           

上面日志中有一個關鍵錯誤語句：

controller: error obtaining controller instance: Enabling IP forwarding failed: open /proc/sys/net/ipv4/ip_forward: read-only file

得出問題應該是該檔案不能修改導緻的，查詢主控端檔案内容，發現該檔案内容為0，要改成1（原因參考：https://blog.csdn.net/li_101357/article/details/78415461)

解決辦法

在主控端中修改/proc/sys/net/ipv4/ip_forward内容為1

echo 1 > /proc/sys/net/ipv4/ip_forward

注意：将此屬性設定為0時，使用sudo設定不成功時，切換到root可以修改；

修改完成後重新開機lxd容器，發現docker起來了。