企業運維實戰--k8s學習筆記8.Volumes配置管理之動态pv、statefulset的使用

• 一、Nfs動态配置設定pv
• 二、StatefulSet如何通過Headless Service維持Pod的拓撲狀态

一、Nfs動态配置設定pv

有兩種PV提供的方式：靜态和動态。

靜态PV：叢集管理者建立多個PV，它們攜帶着真實存儲的詳細資訊，這些存儲對于叢集使用者是可用的。它們存在于Kubernetes API中，并可用于存儲使用。

動态PV：當管理者建立的靜态PV都不比對使用者的PVC時，叢集可能會嘗試專門地供給volume給PVC。這種供給基于StorageClass。

StorageClass提供了一種描述存儲類（class）的方法，不同的class可能會映射到不同的服務品質等級和備份政策或其他政策等。

每個 StorageClass 都包含 provisioner、parameters 和 reclaimPolicy 字段， 這些字段會在StorageClass需要動态配置設定 PersistentVolume 時會使用到。

自動建立PV和對應的PVC，本身不提供NFS存儲，需要外部先有一套NFS存儲服務。是以本章操作是基于實驗環境中nfs已部署完成才進行的。

建立操作目錄并進入，将檔案移入

mkdir nfs-client
cd nfs-client/
cp ~/nfs-client-provisioner.yaml .
           

編輯資源清單，參考官網：https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client

vim nfs-client-provisioner.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs-client-provisioner
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: nfs-client-provisioner
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs-client-provisioner
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: nfs-client-provisioner
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs-client-provisioner
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: nfs-subdir-external-provisioner:v4.0.0
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: westos.org/nfs
            - name: NFS_SERVER
              value: 172.25.9.1 #指定nfs server主機
            - name: NFS_PATH
              value: /mnt/nfs #指定nfs共享路徑
      volumes:
        - name: nfs-client-root
          nfs:
            server: 172.25.9.1 #同上
            path: /mnt/nfs
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: westos.org/nfs
parameters:
  archiveOnDelete: "true" #true為删除後自動打包備份，false為删除後不打包
           

拉起清單前，先建立好storageclass，準備好鏡像

建立sc，拉起清單

kubectl create namespace nfs-client-provisioner
kubectl apply -f nfs-client-provisioner.yaml
kubectl get sc

           

建立pvc

vim test-pvc.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nfs-pv1
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Mi

           

檢視建立好的pv和pvc狀态，顯示Bound

kubectl apply -f test-pvc.yaml
kubectl get pvc
kubectl get pv
           

Delete表示删除pvc後 pv也會删除，同時檔案會打包

測試：

首先檢視挂載目錄中的檔案

删除收回清單，可以看到pv和pvc均已經删除

kubectl delete -f test-pvc.yaml
kubectl get pvc
kubectl get pv
           

再次檢視挂載目錄，檔案已被打包。

測試完畢，拉起pvc

kubectl apply -f test-pvc.yaml
           

建立pod節點test-pd

vim pod.yaml

cat pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: test-pd
spec:
  containers:
  - image: nginx
    name: nginx
    volumeMounts:
    - mountPath: /usr/share/nginx/html
      name: pv1
  volumes:
  - name: pv1
    persistentVolumeClaim:
      claimName: nfs-pv1
           

需要注意的是，選取pvc名稱需與之前建立的pvc名稱一緻。

拉起pod

kubectl apply -f pod.yaml
           

初次通路報錯403，原因是預設釋出檔案被覆寫

kubectl get pod -o wide
curl 10.244.141.237
           

添加測試頁，再次測試，通路成功

kubectl get pod -o wide
curl 10.244.141.237
           

編輯資源清單nfs-client-provisioner.yaml，關閉删除後打包，删除pvc後檔案會消失

vim nfs-client-provisioner.yaml

parameters:
  archiveOnDelete: "false"

           

想要檔案更新修改内容，必須删除sc後再次生成

kubectl delete sc managed-nfs-storage
kubectl apply -f nfs-client-provisioner.yaml
kubectl get sc
           

删除pvc或pod節點

kubectl delete -f test-pvc.yaml
           

删除太慢可以之間删除節點

kubectl delete pod test-pd
kubectl get pvc
kubectl get pv
           

進入共享目錄下檢視

預設的 StorageClass 将被用于動态的為沒有特定 storage class 需求的PersistentVolumeClaims 配置存儲

為pvc設定預設sc狀态

編輯pvc檔案，不特指sc

vim test-pvc.yaml

cat test-pvc.yaml

apiVersion: v1
metadata:
  name: nfs-pv1
spec:
  #storageClassName: managed-nfs-storage
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Mi

           

設定預設storageclass

拉起清單，檢視sc

kubectl apply -f test-pvc.yaml
kubectl get sc
           

二、StatefulSet如何通過Headless Service維持Pod的拓撲狀态

建立工作目錄并進入

mkdir statefulset
cd statefulset/
           

編輯資源清單，建立服務nginx

vim nginx-svc.yaml

apiVersion: v1
kind: Service
metadata:
 name: nginx-svc
 labels:
  app: nginx
spec:
 ports:
 - port: 80
   name: web
 clusterIP: None
 selector:
  app: nginx

           

拉起并檢視服務

kubectl apply -f nginx-svc.yaml
kubectl get svc
           

此時服務還沒有後端endpoint

kubectl describe svc nginx-svc
           

編輯資源清單建立StatefulSet，設定初始副本數為二個

vim statefulset.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
 name: web
spec:
 serviceName: "nginx-svc"
 replicas: 2
 selector:
  matchLabels:
   app: nginx
 template:
  metadata:
   labels:
    app: nginx
  spec:
   containers:
   - name: nginx
     image: myapp:v1
     ports:
     - containerPort: 80
       name: web


           

拉起容器，檢視pod

kubectl apply -f statefulset.yaml
           

擴容pv，隻需将清黨副本數改為6

vim statefulset.yaml

```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
 name: web
spec:
 serviceName: "nginx-svc"
 replicas: 3
 selector:
  matchLabels:
   app: nginx
 template:
  metadata:
   labels:
    app: nginx
  spec:
   containers:
   - name: nginx
     image: myapp:v1
     ports:
     - containerPort: 80
       name: web
     volumeMounts:
       - name: www
         mountPath: /usr/share/nginx/html
 volumeClaimTemplates:
  - metadata:
     name: www
    spec:
     #storageClassName: nfs
     accessModes:
     - ReadWriteOnce
     resources:
      requests:
       storage: 1Gi
           

拉起容器并檢視

kubectl get pod
           

測試采用三個副本數，恢複副本數到3

vim statefulset.yaml
           

kubectl apply -f statefulset.yaml
           

檢視節點健康狀況

kubectl get pod
           

進入容器 ，通路檢視

進入容器，測試通路

kubectl run demo --image=busyboxplus -it
           

kubectl attach demo -it

将pvc與pod節點結合

建立測試index.html，進入/mnt/nfs各目錄中

echo web-0 > index.html

echo web-1 > index.html

echo web-2 > index.html
           

進入容器檢視修改内容

kubectl attach demo -it
           

通路檢視是否負載均衡

curl nginx-svc
           

删除副本後重新拉起，可繼續保留之前的内容

vim statefulset.yaml
           

将副本改為0即視為删除

kubectl apply -f statefulset.yaml
           

節點正在逐個回收

重新拉起三個副本

進入容器測試：

依舊負載均衡