K8S有狀态服務-動态雲盤使用最佳實踐

介紹

動态挂載方式是指在應用中顯式聲明PVC，并在PVC中聲明StorageClass；這時應用會通過Storageclass中指定的Provisioner來自動建立雲盤，并自動生成雲盤PV資源類型；

使用動态雲盤需要滿足以下條件：

叢集中要部署雲盤Provisioner服務，實作自動建立雲盤；
建立預期使用storageclass資源，并指定雲盤Provisioner；
在PVC中顯式聲明使用哪個storageclass；

無需顯式建立PV，而是通過Provisioner自動建立；
無需在ecs控制台購買雲盤，在應用部署時自動購買的情況；
           

雲盤Provisioner

使用雲盤動态卷的一個前提是系統中已經部署了雲盤Provisioner。

K8S叢集會預設部署Provisioner，Provisioner建立雲盤需要對雲盤有操作權限，可以通過AK、或STS token來擷取權限；

配置AK：在部署Provisioner的時候設定ACCESS_KEY_ID、ACCESS_KEY_SECRET環境變量，可以配置ak；
配置STS：為預設方式，可以給叢集（Master節點）授予RAM權限，詳情參看RAM權限管理；
           

下面yaml檔案為部署Provisioner的較長的描述：

---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: alicloud-disk-common
provisioner: alicloud/disk
parameters:
  type: cloud

---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: alicloud-disk-efficiency
provisioner: alicloud/disk
parameters:
  type: cloud_efficiency

---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: alicloud-disk-ssd
provisioner: alicloud/disk
parameters:
  type: cloud_ssd

---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: alicloud-disk-available
provisioner: alicloud/disk
parameters:
  type: available

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: alicloud-disk-controller-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["list", "watch", "create", "update", "patch"]

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: alicloud-disk-controller
  namespace: kube-system

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: run-alicloud-disk-controller
subjects:
  - kind: ServiceAccount
    name: alicloud-disk-controller
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: alicloud-disk-controller-runner
  apiGroup: rbac.authorization.k8s.io

---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: alicloud-disk-controller
  namespace: kube-system
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: alicloud-disk-controller
    spec:
      tolerations:
      - effect: NoSchedule
        operator: Exists
        key: node-role.kubernetes.io/master
      - effect: NoSchedule
        operator: Exists
        key: node.cloudprovider.kubernetes.io/uninitialized
      nodeSelector:
         node-role.kubernetes.io/master: ""
      serviceAccount: alicloud-disk-controller
      containers:
        - name: alicloud-disk-controller
          image: registry.cn-hangzhou.aliyuncs.com/acs/alicloud-disk-controller:v1.10.4-f431fd8
          volumeMounts:
            - name: cloud-config
              mountPath: /etc/kubernetes/
            - name: logdir
              mountPath: /var/log/alicloud/
      volumes:
        - name: cloud-config
          hostPath:
            path: /etc/kubernetes/
        - name: logdir
          hostPath:
            path: /var/log/alicloud/           

StorageClass

阿裡雲K8S系統初始化的時候會預設建立4個StorageClass，這4個StorageClass适合在叢集類型為單一zone的情況，若為多zone部署的叢集，則需要自己另行建立；

alicloud-disk-common：建立普通雲盤。
alicloud-disk-efficiency：建立高效雲盤。
alicloud-disk-ssd：建立SSD雲盤。
alicloud-disk-available：提供高可用選項，先試圖建立高效雲盤；如果相應AZ的高效雲盤資源售盡，再試圖建立SSD盤；如果SSD售盡，則試圖建立普通雲盤。
           

下面yaml描述了建立Storageclass的細節：

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: alicloud-disk-common-hangzhou-b
provisioner: alicloud/disk
reclaimPolicy: Retain
parameters:
  type: cloud_ssd
  regionid: cn-hangzhou
  zoneid: cn-hangzhou-b
  fstype: "ext4"
  readonly: "false"
  encrypted: "true"           

reclaimPolicy：表示建立pv的回收政策，支援Delete、Retain兩個類型，預設為Delete；這裡需要注意：配置為Delete時，删除PVC後雲盤一起被删除，資料不可恢複。

type: 表示建立什麼類型的雲盤，支援cloud、cloud_efficiency、cloud_ssd、available類型；

regionid：表示建立雲盤所在region；

zoneid：表示建立雲盤所在zone；

fstype：表示雲盤使用的檔案系統，可選項，預設為ext4；

readonly：表示挂載的讀寫權限是否為隻讀，可選項，預設為false；

encrypted：是否建立加密雲盤，可選項，預設為false；

使用動态卷建立應用

部署一下應用模闆，在PVC中顯式指定storageClassName為上述建立的StorageClass；

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: disk-ssd
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: alicloud-disk-ssd-beijing-b
  resources:
    requests:
      storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-dynamic
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        volumeMounts:
          - name: disk-pvc
            mountPath: "/data"
      volumes:
        - name: disk-pvc
          persistentVolumeClaim:
            claimName: disk-ssd           

驗證高可用

建立應用

# kubectl create -f dynamic.yaml

# kubectl get pod | grep dynamic
nginx-dynamic-69f9bd7b8c-58sbs   1/1       Running   0          3m

# kubectl exec nginx-dynamic-69f9bd7b8c-58sbs df | grep data
/dev/vdb        20511312   45080  19401272   1% /data
           

在雲盤中建立檔案：

# kubectl exec nginx-dynamic-69f9bd7b8c-58sbs ls /data
lost+found

# kubectl exec nginx-dynamic-69f9bd7b8c-58sbs touch /data/dynamic
# kubectl exec nginx-dynamic-69f9bd7b8c-58sbs ls /data
dynamic
lost+found
           

删除Pod，驗證檔案持久化：

# kubectl delete pod nginx-dynamic-69f9bd7b8c-58sbs
pod "nginx-dynamic-69f9bd7b8c-58sbs" deleted

# kubectl get pod
NAME                             READY     STATUS              RESTARTS   AGE
nginx-dynamic-69f9bd7b8c-58sbs   0/1       Terminating         0          5m
nginx-dynamic-69f9bd7b8c-ddcbb   0/1       ContainerCreating   0          2s

# kubectl exec nginx-dynamic-69f9bd7b8c-ddcbb ls /data
dynamic
lost+found