istio operator 運作錯誤 415: Unsupported Media Type

錯誤場景、環境

kubernetes version: v1.18.2
istio version: v1.10.0
           

運作之後 istio-operator 的日志就抛出下面錯誤，而且會一直重新開機

# kubectl get iop -A
NAMESPACE      NAME                          REVISION     STATUS   AGE
istio-system   iop-pro-cluster-8410-1-10-0   iop-1-10-0   ERROR    1h
           

# kubectl get pod -n istio-system
NAME                                         READY   STATUS    RESTARTS   AGE
istio-operator-iop-1-10-0-759f564df6-6rvbj   1/1     Running   19         1h
           

failed to update resource with server-side apply for obj ConfigMap/istio-system/istio-sidecar-injector-iop-1-10-0: 415: Unsupported Media Type
failed to update resource with server-side apply for obj MutatingWebhookConfiguration//istio-sidecar-injector-iop-1-10-0: 415: Unsupported Media Type
failed to update resource with server-side apply for obj Deployment/istio-system/istiod-iop-1-10-0: 415: Unsupported Media Type
failed to update resource with server-side apply for obj PodDisruptionBudget/istio-system/istiod-iop-1-10-0: 415: Unsupported Media Type
failed to update resource with server-side apply for obj HorizontalPodAutoscaler/istio-system/istiod-iop-1-10-0: 415: Unsupported Media Type
2021-08-31T09:42:12.369429Z info installer using server side apply to update obj: ServiceAccount/istio-system/istio-ingressgateway-service-account
2021-08-31T09:42:12.376827Z error installer failed to update resource with server-side apply for obj ServiceAccount/istio-system/istio-ingressgateway-service-account: 415: Unsupported Media Type
2021-08-31T09:42:12.382257Z info installer using server side apply to update obj: Deployment/istio-system/istio-ingressgateway
2021-08-31T09:42:12.384672Z error installer failed to update resource with server-side apply for obj Deployment/istio-system/istio-ingressgateway: 415: Unsupported Media Type
2021-08-31T09:42:12.385266Z info installer using server side apply to update obj: PodDisruptionBudget/istio-system/istio-ingressgateway
2021-08-31T09:42:12.386973Z error installer failed to update resource with server-side apply for obj PodDisruptionBudget/istio-system/istio-ingressgateway: 415: Unsupported Media Type
2021-08-31T09:42:12.387721Z info installer using server side apply to update obj: Role/istio-system/istio-ingressgateway-sds
2021-08-31T09:42:12.392418Z error installer failed to update resource with server-side apply for obj Role/istio-system/istio-ingressgateway-sds: 415: Unsupported Media Type
2021-08-31T09:42:12.393067Z info installer using server side apply to update obj: RoleBinding/istio-system/istio-ingressgateway-sds
2021-08-31T09:42:12.398916Z error installer failed to update resource with server-side apply for obj RoleBinding/istio-system/istio-ingressgateway-sds: 415: Unsupported Media Type
2021-08-31T09:42:12.401822Z info installer using server side apply to update obj: HorizontalPodAutoscaler/istio-system/istio-ingressgateway
2021-08-31T09:42:12.403878Z error installer failed to update resource with server-side apply for obj HorizontalPodAutoscaler/istio-system/istio-ingressgateway: 415: Unsupported Media Type
2021-08-31T09:42:12.404578Z info installer using server side apply to update obj: Service/istio-system/istio-ingressgateway
2021-08-31T09:42:12.405936Z error installer failed to update resource with server-side apply for obj Service/istio-system/istio-ingressgateway: 415: Unsupported Media Type
failed to update resource with server-side apply for obj Service/istio-system/istiod-iop-1-10-0: 415: Unsupported Media Type
✘ Ingress gateways encountered an error: failed to update resource with server-side apply for obj ServiceAccount/istio-system/istio-ingressgateway-service-account: 415: Unsupported Media Type
failed to update resource with server-side apply for obj Deployment/istio-system/istio-ingressgateway: 415: Unsupported Media Type
failed to update resource with server-side apply for obj PodDisruptionBudget/istio-system/istio-ingressgateway: 415: Unsupported Media Type
failed to update resource with server-side apply for obj Role/istio-system/istio-ingressgateway-sds: 415: Unsupported Media Type
failed to update resource with server-side apply for obj RoleBinding/istio-system/istio-ingressgateway-sds: 415: Unsupported Media Type
failed to update resource with server-side apply for obj HorizontalPodAutoscaler/istio-system/istio-ingressgateway: 415: Unsupported Media Type
failed to update resource with server-side apply for obj Service/istio-system/istio-ingressgateway: 415: Unsupported Media Type
2021-08-31T09:46:44.671636Z error klog error retrieving resource lock istio-system/istio-operator-lock-iop-1-10-0: etcdserver: request timed out
           

解決問題

仔細看就可以發現錯誤日志抛出的内容：

failed to update resource with server-side apply for obj

服務端更新

搜尋了一下 server side 找到 k8s 官方的文檔：服務端更新

注意這句話，這個特新是可以禁用的。這裡我就想到了 istio-operator 為什麼會出現這個問題。

在使用 v1.18.2 版本的 k8s 時，我注意到所有的資源都添加了

managedfields

字段并且配置非常多，看着也比較麻煩。是以就使用

--feature-gates ServerSideApply=false

禁用了

managedfields

問題現在比較清晰了：由于使用

--feature-gates ServerSideApply=false

禁用

managedfields

導緻 istio-operator 内部使用了 server side apply 而更新失敗，修複的話就直接把這個參數從 api-server 的啟動中去掉就可以了

如果是 kubeadm 部署的 k8s，kube-apiserver 配置在下面預設路徑中或者更新參數

/etc/kubernetes/manifests/kube-apiserver.yaml
           

讓 istio-operator 重新開機或者稍等 1 分鐘左右

NAMESPACE      NAME                          REVISION     STATUS    AGE
istio-system   iop-pro-cluster-8410-1-10-0   iop-1-10-0   HEALTHY   2h