在虛拟機時代，我們經常會收到給磁盤擴容的需求，一般我們就是添加一塊磁盤，然後做LVM擴容操作，那在容器時代，我們可以給PVC擴容嗎？
之前發了持久化存儲相關的知識--PV和PVC，StorageClass，本文來給你介紹一下用NFS做後端存儲能不能擴容的問題，當然在生産上沒有哪個願意用NFS做後端存儲......
你首先需要準備好NFS存儲
首先我們介紹一下pv和pvc模式，也就是純手動建立的模式。
（1）、首先我們建立一個PV的YAML檔案（pv2.yaml）
apiVersion: v1
kind: PersistentVolume
metadata:
name: my-pv02
labels:
storage: pv
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 500Mi
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /data/k8s
server: 10.1.10.128 複制
建立pv
# kubectl apply -f pv2.yaml
persistentvolume/my-pv02 created
# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE 30s
my-pv02 500Mi RWO Recycle Bound default/pvc-test 30s 複制
（2）、建立PVC（pvc.yaml），我們這裡建立一個比較小的pvc，如下
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-test
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 500Mi 複制
然後建立pvc
# kubectl apply -f pvc.yaml
# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-test Bound my-pv02 500Mi RWO 58s 複制
我們其實可以看到，我在pvc裡定義的是500Mi，它會自動比對剛好合适的PV。
這時候如果我們修改my-pv02的YAML檔案，如下：
apiVersion: v1
kind: PersistentVolume
metadata:
name: my-pv02
labels:
storage: pv
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 2Gi
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /data/k8s
server: 10.1.10.128 複制
然後我們嘗試建立這個PV，如下：
# kubectl apply -f pv2.yaml
persistentvolume/my-pv02 configured
# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
my-pv02 2Gi RWO Recycle Bound default/pvc-test 116m
# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-test Bound my-pv02 500Mi RWO 116m 複制
我們看到我們my-pv02的CAPACITY變成了2Gi，我們的pvc還是500Mi，那麼我們能不能對PVC進行擴容呢？我嘗試的修改了pvc的配置檔案，如下：
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-test
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi 複制
然後我們執行這個配置檔案
]# kubectl apply -f pvc.yaml
Error from server (Forbidden): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"PersistentVolumeClaim\",\"metadata\":{\"annotations\":{},\"name\":\"pvc-test\",\"namespace\":\"default\"},\"spec\":{\"accessModes\":[\"ReadWriteOnce\"],\"resources\":{\"requests\":{\"storage\":\"1Gi\"}}}}\n"}},"spec":{"resources":{"requests":{"storage":"1Gi"}}}}
to:
Resource: "/v1, Resource=persistentvolumeclaims", GroupVersionKind: "/v1, Kind=PersistentVolumeClaim"
Name: "pvc-test", Namespace: "default"
Object: &{map["apiVersion":"v1" "kind":"PersistentVolumeClaim" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"PersistentVolumeClaim\",\"metadata\":{\"annotations\":{},\"name\":\"pvc-test\",\"namespace\":\"default\"},\"spec\":{\"accessModes\":[\"ReadWriteOnce\"],\"resources\":{\"requests\":{\"storage\":\"500Mi\"}}}}\n" "pv.kubernetes.io/bind-completed":"yes" "pv.kubernetes.io/bound-by-controller":"yes"] "creationTimestamp":"2019-12-24T10:46:05Z" "finalizers":["kubernetes.io/pvc-protection"] "name":"pvc-test" "namespace":"default" "resourceVersion":"15059" "selfLink":"/api/v1/namespaces/default/persistentvolumeclaims/pvc-test" "uid":"6a905af6-5e44-4940-ba87-e56f6e99df24"] "spec":map["accessModes":["ReadWriteOnce"] "resources":map["requests":map["storage":"500Mi"]] "volumeMode":"Filesystem" "volumeName":"my-pv02"] "status":map["accessModes":["ReadWriteOnce"] "capacity":map["storage":"500Mi"] "phase":"Bound"]]}
for: "pvc.yaml": persistentvolumeclaims "pvc-test" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize 複制
我們發現報錯了，錯誤資訊如下：
persistentvolumeclaims "pvc-test" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize 複制
它的意思就是說隻有動态供應的pvc可以調整大小，供應pvc的存儲類必須支援調整大小。也就是說我們手動建立PV，PVC模式是不支援調整大小的。
我們現在來驗證一下用storageclass建立的PVC。
（1）、因為我們使用nfs作為後端存儲，是以我們需要先建立一個nfs-client，YAML檔案如下：
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: nfs-client-provisioner-clusterrole
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: nfs-client-provisioner-clusterrolebinding
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-clusterrole
apiGroup: rbac.authorization.k8s.io
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nfs-client-prosioner
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-client-prosioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-prosioner
image: registry.cn-hangzhou.aliyuncs.com/rookieops/nfs-client-provisioner:v0.1
imagePullPolicy: IfNotPresent
volumeMounts:
- name: nfs-client-root
mountPath: /data/pv
env:
- name: PROVISIONER_NAME
value: rookieops/nfs
- name: NFS_SERVER
value: 10.1.10.128
- name: NFS_PATH
value: /data/k8s
volumes:
- name: nfs-client-root
nfs:
server: 10.1.10.128
path: /data/k8s 複制
建立YAML檔案
# kubectl apply -f !$
kubectl apply -f nfs-client.yaml
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-clusterrole created
clusterrolebinding.rbac.authorization.k8s.io/nfs-client-provisioner-clusterrolebinding created
deployment.extensions/nfs-client-prosioner created
# kubectl get pod
NAME READY STATUS RESTARTS AGE
nfs-client-prosioner-56f44c675b-t2ml9 1/1 Running 0 6s 複制
（1）我們建立一個普通的StorageClass（storageclass.yaml）
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs-client-storageclass
provisioner: rookieops/nfs 複制
建立這個YAML檔案
# kubectl apply -f storageclass.yaml
storageclass.storage.k8s.io/nfs-client-storageclass created
# kubectl get storageclasses.storage.k8s.io
NAME PROVISIONER AGE
nfs-client-storageclass rookieops/nfs 34s 複制
建立一個PVC（storageclass-pvc.yaml）
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-nfs-pvc2
annotations:
volume.beta.kubernetes.io/storage-class: "nfs-client-storageclass"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Mi 複制
建立這個YAML檔案
# kubectl apply -f storageclass-pvc.yaml
persistentvolumeclaim/test-nfs-pvc2 created
# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-test Bound my-pv02 500Mi RWO 26d
test-nfs-pvc2 Bound pvc-9dd9c608-6875-4609-829c-c92920aaa783 1Mi RWX nfs-client-storageclass 21s 複制
我們看到已經Bound了，我們現在直接修改pvc（storageclass-pvc.yaml）
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-nfs-pvc2
annotations:
volume.beta.kubernetes.io/storage-class: "nfs-client-storageclass"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 100Mi 複制
然後我們執行這個YAML檔案
# kubectl apply -f storageclass-pvc.yaml
Error from server (Forbidden): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"PersistentVolumeClaim\",\"metadata\":{\"annotations\":{\"volume.beta.kubernetes.io/storage-class\":\"nfs-client-storageclass\"},\"name\":\"test-nfs-pvc2\",\"namespace\":\"default\"},\"spec\":{\"accessModes\":[\"ReadWriteMany\"],\"resources\":{\"requests\":{\"storage\":\"100Mi\"}}}}\n"}},"spec":{"resources":{"requests":{"storage":"100Mi"}}}}
to:
Resource: "/v1, Resource=persistentvolumeclaims", GroupVersionKind: "/v1, Kind=PersistentVolumeClaim"
Name: "test-nfs-pvc2", Namespace: "default"
Object: &{map["apiVersion":"v1" "kind":"PersistentVolumeClaim" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"PersistentVolumeClaim\",\"metadata\":{\"annotations\":{\"volume.beta.kubernetes.io/storage-class\":\"nfs-client-storageclass\"},\"name\":\"test-nfs-pvc2\",\"namespace\":\"default\"},\"spec\":{\"accessModes\":[\"ReadWriteMany\"],\"resources\":{\"requests\":{\"storage\":\"1Mi\"}}}}\n" "pv.kubernetes.io/bind-completed":"yes" "pv.kubernetes.io/bound-by-controller":"yes" "volume.beta.kubernetes.io/storage-class":"nfs-client-storageclass" "volume.beta.kubernetes.io/storage-provisioner":"rookieops/nfs"] "creationTimestamp":"2020-01-20T06:04:20Z" "finalizers":["kubernetes.io/pvc-protection"] "name":"test-nfs-pvc2" "namespace":"default" "resourceVersion":"29483" "selfLink":"/api/v1/namespaces/default/persistentvolumeclaims/test-nfs-pvc2" "uid":"9dd9c608-6875-4609-829c-c92920aaa783"] "spec":map["accessModes":["ReadWriteMany"] "resources":map["requests":map["storage":"1Mi"]] "volumeMode":"Filesystem" "volumeName":"pvc-9dd9c608-6875-4609-829c-c92920aaa783"] "status":map["accessModes":["ReadWriteMany"] "capacity":map["storage":"1Mi"] "phase":"Bound"]]}
for: "storageclass-pvc.yaml": persistentvolumeclaims "test-nfs-pvc2" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize 複制
發現報與上面一樣的錯，這是因為要支援動态擴容需要滿足兩個條件：
- 後端底層存儲支援卷擴充（後端存儲保證足夠資源）
- 需要在StorageClass對象中設定allowVolumeExpansion為true
我們由于是測試，申請的資源比較少，我們直接對StorageClass對象進行修改（storageclass.yaml），如下：
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs-client-storageclass
provisioner: rookieops/nfs
allowVolumeExpansion: true 複制
然後我們重新申明一下
# kubectl apply -f storageclass.yaml
storageclass.storage.k8s.io/nfs-client-storageclass configured 複制
我們再次建立前面的storageclass-pvc.yaml檔案
# kubectl apply -f storageclass-pvc.yaml
persistentvolumeclaim/test-nfs-pvc2 configured 複制
我們發現這次沒有報錯，我們檢視一下這個PVC
# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-test Bound my-pv02 500Mi RWO 26d
test-nfs-pvc2 Bound pvc-9dd9c608-6875-4609-829c-c92920aaa783 1Mi RWX nfs-client-storageclass 14m 複制
我們一看，發現并沒有擴充成功，這是為什麼呢？我們describe一下這個pvc，發現如下提示：
# kubectl describe pvc test-nfs-pvc2
...
Warning ExternalExpanding 7m43s volume_expand
Ignoring the PVC: didn't find a plugin capable of expanding the volume; waiting for an external controller to process this PVC. 複制
報錯資訊為：沒有找到可擴充的插件。
我上官方網站一看，原來人家已經說的很清楚了：Although the feature is enabled by default, a cluster admin must opt-in to allow users to resize their volumes. Kubernetes v1.11 ships with volume expansion support for the following in-tree volume plugins: AWS-EBS, GCE-PD, Azure Disk, Azure File, Glusterfs, Cinder, Portworx, and Ceph RBD.
我們的NFS并不被支援（用NFS做後端存儲的小夥伴注意了哈）。