參考文檔:
etcd charm
etcdctl報錯x509
ETCD 使用
使用ubuntu charmed kubernetes 部署一套生産環境的叢集
因為etcd/2在安裝時使用了錯誤的節點,是以想将錯誤節點的etcd删除并重建。
删除節點
juju remove-unit etcd/2 --force --no-wait
重建節點
juju add-machine --constraints tags=etcd
machine 13
juju add-unit etcd --to 13
顯示狀态
juju status
etcd/0* active idle 1 10.0.4.139 2379/tcp UnHealthy with 3 known peers
etcd/3 waiting idle 17 10.0.4.153 Waiting to retry etcd registration
研究了下,應該是etcd的節點資訊中原來的etcd/2沒有除去,是以新增節點增加不進去。
擷取憑證資訊:
juju run-action --wait etcd/0 package-client-credentials
juju scp etcd/0:etcd_credentials.tar.gz etcd_credentials.tar.gz
解壓:
tar -zxvf etcd_credentials.tar.gz
etcd_credentials/
etcd_credentials/ca.crt
etcd_credentials/README.txt
etcd_credentials/client.crt
etcd_credentials/client.key
轉移到/root/etcd_credentials/目錄
cd /root/etcd_credentials/
手工輸入環境變量,因為etcd版本為3.4.5 ,是以格式如下:
juju expose etcd
export ETCDCTL_KEY=$(pwd)/client.key
export ETCDCTL_CERT=$(pwd)/client.crt
export ETCDCTL_CACERT=$(pwd)/ca.crt
export ETCDCTL_API=3 #否則會出現509錯誤
export ETCDCTL_ENDPOINT=https://10.0.4.139:2379 # etcd/0的ip
列出成員清單:
etcdctl member list
出現了如下錯誤:
Error: dial tcp 127.0.0.1:2379: connect: connection refused
查了下指令,原來在現在的k8s中,etcdctl指令格式變了,需要增加端點參數
--endpoints=https://10.0.4.139:2379
:
#10.0.4.139為處于leadership的etcd節點IP,目前為etcd/0節點。
如檢驗端點健康狀态:
etcdctl --endpoints=https://10.0.4.139:2379 endpoint health
https://10.0.4.139:2379 is healthy: successfully committed proposal: took = 8.609559ms
列出成員名單:
etcdctl --endpoints=https://10.0.4.139:2379 member list
bb605e8c9ebece4, started, etcd0, https://10.0.4.139:2380, https://10.0.4.139:2379
54bba7baf27ccef7, started, etcd1, https://10.0.4.140:2380, https://10.0.4.140:2379
defc4e8a9c8f25bc, started, etcd2, https://10.0.4.145:2380, https://10.0.4.145:2379
其中
defc4e8a9c8f25bc, started, etcd2, https://10.0.4.145:2380, https://10.0.4.145:2379
,就是已經删除的etcd節點,需要etcdctl删除。
删除多餘的etcd節點:
etcdctl --endpoints=https://10.0.4.139:2379 member remove defc4e8a9c8f25bc
Member defc4e8a9c8f25bc removed from cluster 22b26385f89f7fa8
在過一會兒,新增的etcd節點已經添加到etcd叢集中了
juju status
etcd/0* active idle 1 10.0.4.139 2379/tcp Healthy with 2 known peers
filebeat/2 active idle 10.0.4.139 Filebeat ready.
etcd/1 active idle 2 10.0.4.140 2379/tcp Healthy with 3 known peers
filebeat/1 active idle 10.0.4.140 Filebeat ready.
etcd/4 active idle 17 10.0.4.153 2379/tcp Healthy with 3 known peers
filebeat/10 active idle 10.0.4.153 Filebeat ready.