centos7使用kubeadm安裝部署kubernetes 1.14

應用背景：

截止目前為止，高熱度的kubernetes版本已經釋出至1.14，在此記錄一下安裝部署步驟和過程中的問題排查。

部署k8s一般兩種方式：kubeadm（官方稱目前已經GA，可以在生産環境使用）；二進制安裝（比較繁瑣）。

這裡暫且采用kubeadm方式部署測試。

測試環境：

System	Hostname	IP
CentOS 7.6	k8s-master	138.138.82.14
CentOS 7.6	k8s-node1	138.138.82.15
CentOS 7.6	k8s-node2	138.138.82.16

網絡插件：calico

具體步驟：

1. 環境預設（在所有主機上操作）

關閉firewalld：

systemctl stop firewalld && systemctl disable firewalld

關閉SElinux：

setenforce 0 && sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config

關閉Swap：

swapoff -a && sed -i "s/\/dev\/mapper\/centos-swap/\#\/dev\/mapper\/centos-swap/g" /etc/fstab

使用阿裡雲yum源：

wget -O /etc/yum.repos.d/CentOS7-Aliyun.repo http://mirrors.aliyun.com/repo/Centos-7.repo

更新 /etc/hosts 檔案：在每一台主機的該檔案中添加k8s所有節點的IP和對應主機名，否則初始化的時候回出現告警甚至錯誤。

2. 安裝docker引擎（在所有主機上操作）

安裝阿裡雲docker源：

wget -O /etc/yum.repos.d/docker-ce http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

安裝docker：

yum install docker-ce -y

啟動docker：

systemctl enable docker && systemctl start docker

調整docker部分參數：

mkdir -p /etc/docker
tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://5twf62k1.mirror.aliyuncs.com"],　　　// 改為阿裡鏡像
  "exec-opts": ["native.cgroupdriver=systemd"]　　// 預設cgroupfs，k8s官方推薦systemd，否則初始化出現Warning
}
EOF
systemctl daemon-reload
systemctl restart docker

檢查确認docker的Cgroup Driver資訊：

[[email protected] ~]# docker info |grep Cgroup
Cgroup Driver: systemd

3. 安裝kubernetes初始化工具（在所有主機上操作）

使用阿裡雲的kubernetes源：

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

安裝工具： yum install -y kubelet kubeadm kubectl 　　// 此時最新版本1.14.1

啟動kubelet： systemctl enable kubelet && systemctl start kubelet 　　// 此時啟動不成功正常，後面初始化的時候會變成功

4. 預下載下傳相關鏡像（在master節點上操作）

檢視叢集初始化所需鏡像及對應依賴版本号：

[[email protected] ~]# kubeadm config images list
……
k8s.gcr.io/kube-apiserver:v1.14.1
k8s.gcr.io/kube-controller-manager:v1.14.1
k8s.gcr.io/kube-scheduler:v1.14.1
k8s.gcr.io/kube-proxy:v1.14.1
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.3.10
k8s.gcr.io/coredns:1.3.1

因為這些重要鏡像都被牆了，是以要預先單獨下載下傳好，然後才能初始化叢集。

下載下傳腳本：

#!/bin/bash

set -e

KUBE_VERSION=v1.14.1
KUBE_PAUSE_VERSION=3.1
ETCD_VERSION=3.3.10
CORE_DNS_VERSION=1.3.1

GCR_URL=k8s.gcr.io
ALIYUN_URL=registry.cn-hangzhou.aliyuncs.com/google_containers

images=(kube-proxy:${KUBE_VERSION}
kube-scheduler:${KUBE_VERSION}
kube-controller-manager:${KUBE_VERSION}
kube-apiserver:${KUBE_VERSION}
pause:${KUBE_PAUSE_VERSION}
etcd:${ETCD_VERSION}
coredns:${CORE_DNS_VERSION})


for imageName in ${images[@]} ; do
  docker pull $ALIYUN_URL/$imageName
  docker tag  $ALIYUN_URL/$imageName $GCR_URL/$imageName
  docker rmi $ALIYUN_URL/$imageName
done

View Code

5. 初始化叢集（在master節點上操作）

kubeadm init --kubernetes-version=v1.14.1 --pod-network-cidr=192.168.0.0/16

注意：初始化之後會安裝網絡插件，這裡選擇了calico，是以修改 --pod-network-cidr=192.168.0.0/16

初始化輸出記錄樣例：

[[email protected] ~]# kubeadm init --kubernetes-version=v1.14.1 --pod-network-cidr=192.168.0.0/16
[init] Using Kubernetes version: v1.14.1
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 138.138.82.14]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [138.138.82.14 127.0.0.1 ::1]
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [138.138.82.14 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 16.002739 seconds
[upload-config] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.14" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --experimental-upload-certs
[mark-control-plane] Marking the node k8s-master as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: 57iu95.6narx7y8peauts76
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 138.138.82.14:6443 --token 57iu95.6narx7y8peauts76 \
    --discovery-token-ca-cert-hash sha256:5dc8beaa3b0e6fa26b97e2cc3b8ae776d000277fd23a7f8692dc613c6e59f5e4

View Code

以上輸出顯示初始化成功，并給出了接下來的必要步驟和節點加入叢集的指令，照着做即可。

[[email protected] ~]# mkdir -p $HOME/.kube
[[email protected]-master ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[[email protected]-master ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

檢視已經運作的pod

[[email protected] ~]# kubectl get pod -n kube-system -owide
NAME                                 READY   STATUS    RESTARTS   AGE     IP              NODE         NOMINATED NODE   READINESS GATES
coredns-fb8b8dccf-6mgks              0/1     Pending   0          9m6s    <none>          <none>       <none>           <none>
coredns-fb8b8dccf-cbtlx              0/1     Pending   0          9m6s    <none>          <none>       <none>           <none>
etcd-k8s-master                      1/1     Running   0          8m22s   138.138.82.14   k8s-master   <none>           <none>
kube-apiserver-k8s-master            1/1     Running   0          8m19s   138.138.82.14   k8s-master   <none>           <none>
kube-controller-manager-k8s-master   1/1     Running   0          8m30s   138.138.82.14   k8s-master   <none>           <none>
kube-proxy-c9xd2                     1/1     Running   0          9m7s    138.138.82.14   k8s-master   <none>           <none>
kube-scheduler-k8s-master            1/1     Running   0          8m6s    138.138.82.14   k8s-master   <none>           <none>

到這裡，會發現除了coredns未ready，這是正常的，因為還沒有網絡插件，接下來安裝calico後就變為正常running了。

6. 安裝calico（在master節點上操作）

Calico官網：https://docs.projectcalico.org/v3.6/getting-started/kubernetes/

kubectl apply -f \
https://docs.projectcalico.org/v3.5/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml

應用官方的yaml檔案之後，過一會檢視所有pod已經正常running狀态了，也配置設定出了對應IP：

[[email protected] ~]# kubectl get pod -n kube-system -owide 
NAME                                 READY   STATUS    RESTARTS   AGE   IP              NODE         NOMINATED NODE   READINESS GATES
calico-node-r5mlj                    1/1     Running   0          72s   138.138.82.14   k8s-master   <none>           <none>
coredns-fb8b8dccf-6mgks              1/1     Running   0          15m   192.168.0.7     k8s-master   <none>           <none>
coredns-fb8b8dccf-cbtlx              1/1     Running   0          15m   192.168.0.6     k8s-master   <none>           <none>
etcd-k8s-master                      1/1     Running   0          15m   138.138.82.14   k8s-master   <none>           <none>
kube-apiserver-k8s-master            1/1     Running   0          15m   138.138.82.14   k8s-master   <none>           <none>
kube-controller-manager-k8s-master   1/1     Running   0          15m   138.138.82.14   k8s-master   <none>           <none>
kube-proxy-c9xd2                     1/1     Running   0          15m   138.138.82.14   k8s-master   <none>           <none>
kube-scheduler-k8s-master            1/1     Running   0          14m   138.138.82.14   k8s-master   <none>           <none>

檢視節點狀态

[[email protected] ~]# kubectl get node -owide
NAME         STATUS   ROLES    AGE   VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION               CONTAINER-RUNTIME
k8s-master   Ready    master   22m   v1.14.1   138.138.82.14   <none>        CentOS Linux 7 (Core)   3.10.0-957.10.1.el7.x86_64   docker://18.9.5

至此，叢集初始化和主節點都準備就緒，接下來就是加入其他工作節點至叢集中。

7. 加入叢集（在非master節點上操作）

先在需要加入叢集的節點上下載下傳必要鏡像，下載下傳腳本如下：

#!/bin/bash

set -e

KUBE_VERSION=v1.14.1
KUBE_PAUSE_VERSION=3.1

GCR_URL=k8s.gcr.io
ALIYUN_URL=registry.cn-hangzhou.aliyuncs.com/google_containers

images=(kube-proxy-amd64:${KUBE_VERSION}
pause:${KUBE_PAUSE_VERSION})


for imageName in ${images[@]} ; do
  docker pull $ALIYUN_URL/$imageName
  docker tag  $ALIYUN_URL/$imageName $GCR_URL/$imageName
  docker rmi $ALIYUN_URL/$imageName
done

View Code

然後在主節點初始化輸出中擷取加入叢集的指令，複制到工作節點執行即可：

[[email protected] ~]# kubeadm join 138.138.82.14:6443 --token 57iu95.6narx7y8peauts76 \
>     --discovery-token-ca-cert-hash sha256:5dc8beaa3b0e6fa26b97e2cc3b8ae776d000277fd23a7f8692dc613c6e59f5e4 
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.14" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Activating the kubelet service
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

8. 在master節點上檢視各節點工作狀态

[[email protected] ~]# kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   26m   v1.14.1
k8s-node1    Ready    <none>   84s   v1.14.1
k8s-node2    Ready    <none>   74s   v1.14.1

至此，最簡單的叢集已經部署完成。

接下來，部署其他插件。

下一篇：calico用戶端工具calicoctl

結束.

轉載于:https://www.cnblogs.com/ding2016/p/10784620.html

centos7使用kubeadm安裝部署kubernetes 1.14

繼續閱讀

七牛雲-C#SDK-上傳-前期準備

作業系統（python）多程序學習

Shell程式設計——sort排序、uniq忽略重複、tr替換壓縮删除、cut指定删除字段、正規表達式元字元sort 指令uniq 指令tr 指令cut 指令正規表達式

Linxu常用指令技巧彙總

httpd服務的部署、啟動、配置和簡單優化一、部署二、啟動三、配置檔案

《Linux指令行與Shell腳本程式設計大全第2版.布盧姆》pdf

ACS基本配置-權限等級管理

nginx 安裝錯誤資訊解決

傳說FreeBSD等比Linux更穩定，更“健壯”

無人機--飛控科普

27 Best Free Eclipse Plug-ins for Java Developer to be ProductiveCode Quality PluginsText Editor PluginsDependency ManagementVersion Control Integration PluginsFramework Development Continuous Integration Related PluginsOther Utility Plugins

Ambari介紹和架構原理

vue-cli簡介（中文翻譯）

Ajax發送和擷取json資料到Spring mvc 1.spring mvc後端2.web前段

JSONObject包導入異常 java.lang.NoClassDefFoundErrorweb項目的導入包的問題