阿裡雲容器服務Kubernetes使用者授權管理

阿裡雲容器服務Kubernetes使用者授權管理分為2個級别：一個是RAM通路控制，即設定使用者對k8s叢集的通路權限，比如檢視叢集清單，建立/删除叢集的權限，是否能夠對叢集進行擴容等等；另一個是需要在叢集内對使用者進行細粒度的授權，比如是否允許使用者部署應用，以及對叢集内資源的修改删除。本文将對這兩個部分進行簡單介紹，進而掌握對叢集的正确授權管理，特别示範了如何單獨建立叢集使用者并為其授權的過程。

Kubernetes 阿裡雲容器服務RAM通路控制

在RAM通路控制裡面，有2個系統政策：AliyunCSFullAccess和AliyunCSReadOnlyAccess。很明顯前者是具有對k8s叢集的完全通路控制，能夠在阿裡雲控制台建立删除叢集。而後這隻讀權限的内容是"cs:Get"，也就是說隻具有Get權限。這兩個系統政策的顆粒度是比較大的，要想精細化權限管理，需要進行自定義政策并賦給指定使用者。下表是所有叢集具有的操作（參考

自定義 RAM 授權政策

）：

Action	說明
CreateCluster	建立叢集
AttachInstances	向叢集中添加已有 ECS 執行個體
ScaleCluster	擴容叢集
GetClusters	檢視叢集清單
GetClusterById	檢視叢集詳情
ModifyClusterName	修改叢集名稱
DeleteCluster	删除叢集
UpgradeClusterAgent	更新叢集 Agent
GetClusterLogs	檢視叢集的記錄檔
GetClusterEndpoint	檢視叢集接入點位址
GetClusterCerts	下載下傳叢集證書
RevokeClusterCerts	吊銷叢集證書
BindSLB	為叢集綁定負載均衡執行個體
UnBindSLB	為叢集解綁負載均衡執行個體
ReBindSecurityGroup	為叢集重新綁定安全組
CheckSecurityGroup	檢測叢集現有的安全組規則
FixSecurityGroup	修複叢集的安全組規則
ResetClusterNode	重置叢集中的節點
DeleteClusterNode	移除叢集中的節點
GetClusterProjects	檢視叢集下的應用
CreateTriggerHook	為應用建立觸發器
GetTriggerHook	檢視應用的觸發器清單
RevokeTriggerHook	删除應用的觸發器
CreateClusterToken	建立 Token
UpdateClusterTags	編輯叢集标簽

阿裡雲環境如何用子賬号對使用者進行授權

具有阿裡雲RAM權限隻是具有管理阿裡雲叢集的第一步，要進行叢集的使用和運維，還需要利用Kubernetes RBAC進行單獨的通路控制。

其實這個跟阿裡雲的RAM通路控制Policy比較類似，就是描述誰可以對K8S的資源進行什麼樣的操作。不同的政策賦給不同的使用者，阿裡雲預置的一組政策可以關聯于角色Role，然後在阿裡雲叢集授權中就可以将相應的Role賦給具體某一使用者。這裡面看到的管理者，運維人員，開發人員等其實就是不同的角色，後面我們可以看到他們對應的具體Policy。

阿裡雲主賬号具有最大的管理權限，可以做叢集的授權操作，但一般我們不建議用主賬号做太多的操作，是以盡量用管理者賬戶操作就可以了。但對于K8S授權，即使是系統管理者，在對叢集做授權時，也會提示“目前子賬号不具備授權管理權限，請聯系主帳号或權限管理者授權。”，這是因為K8S授權需要先用主賬号對叢集的管理者先賦予叢集管理者角色，這樣這個管理者才能對其它子賬号進行授權操作。準确地說，授權需要滿足2個條件，一個是RAM通路控制讀權限（參考

link

），另一個是叢集中對apiGroups的rbac.authorization.k8s.io的通路權限。

Kubernetes RBAC（Role Based Access Control）基于角色的通路控制

我們用下面的指令可以檢視K8S中的ClusterRole，ClusterRoleBinding。

# kubectl get ClusterRole
NAME                                                                   AGE
admin                                                                  1y
ahas                                                                   245d
alibaba-log-controller                                                 1y
alicloud-disk-controller-runner                                        1y
arms-pilot-ack-arms-pilot-role                                         200d
cluster-admin                                                          1y
cs:admin                                                               340d
cs:dev                                                                 340d
cs:heapster                                                            1y
cs:ns:dev                                                              1y
cs:ops                                                                 337d
cs:restricted                                                          340d
deployment-reader                                                      1y
drain-node                                                             109d
edit                                                                   1y
flannel                                                                1y
kube-state-metrics                                                     1y
nginx-ingress-controller                                               320d
pod-reader                                                             1y
system:aggregate-to-admin                                              1y
system:aggregate-to-edit                                               1y
system:aggregate-to-view                                               1y
system:auth-delegator                                                  1y
system:aws-cloud-provider                                              1y
system:basic-user                                                      1y
system:certificates.k8s.io:certificatesigningrequests:nodeclient       1y
system:certificates.k8s.io:certificatesigningrequests:selfnodeclient   1y
system:cloud-controller-manager                                        1y
system:controller:attachdetach-controller                              1y
system:controller:certificate-controller                               1y
system:controller:clusterrole-aggregation-controller                   1y
system:controller:cronjob-controller                                   1y
system:controller:daemon-set-controller                                1y
system:controller:deployment-controller                                1y
system:controller:disruption-controller                                1y
system:controller:endpoint-controller                                  1y
system:controller:expand-controller                                    1y
system:controller:generic-garbage-collector                            1y
system:controller:horizontal-pod-autoscaler                            1y
system:controller:job-controller                                       1y
system:controller:namespace-controller                                 1y
system:controller:node-controller                                      1y
system:controller:persistent-volume-binder                             1y
system:controller:pod-garbage-collector                                1y
system:controller:pv-protection-controller                             1y
system:controller:pvc-protection-controller                            1y
system:controller:replicaset-controller                                1y
system:controller:replication-controller                               1y
system:controller:resourcequota-controller                             1y
system:controller:route-controller                                     1y
system:controller:service-account-controller                           1y
system:controller:service-controller                                   1y
system:controller:statefulset-controller                               1y
system:controller:ttl-controller                                       1y
system:coredns                                                         1y
system:csi-external-attacher                                           1y
system:csi-external-provisioner                                        1y
system:discovery                                                       1y
system:heapster                                                        1y
system:kube-aggregator                                                 1y
system:kube-controller-manager                                         1y
system:kube-dns                                                        1y
system:kube-scheduler                                                  1y
system:kubelet-api-admin                                               1y
system:node                                                            1y
system:node-bootstrapper                                               1y
system:node-problem-detector                                           1y
system:node-proxier                                                    1y
system:persistent-volume-provisioner                                   1y
system:volume-scheduler                                                1y
test-role                                                              2h
view                                                                   1y           

檢視ClusterRoleBinding：

# kubectl get clusterrolebinding
NAME                                                   AGE
202729146928739501-clusterrolebinding                  218d
202865947013061057-clusterrolebinding                  336d
203542248038995370-clusterrolebinding                  333d
204696546927216725-clusterrolebinding                  1d
admin                                                  1y
ahas                                                   245d
alibaba-log-controller                                 1y
aliyun-acr-credential-helper-rolebinding               1y
arms-pilot-ack-arms-pilot-role-binding                 200d
cluster-admin                                          1y
deployment-reader                                      1y
drain-node                                             109d
flannel                                                1y
global-job-controller-role-binding                     329d
kube-state-metrics                                     1y
kubeadm:kubelet-bootstrap                              1y
kubeadm:node-autoapprove-bootstrap                     1y
kubeadm:node-autoapprove-certificate-rotation          1y
kubeadm:node-proxier                                   1y
nginx-ingress-controller-starterservices-dev           224d
nginx-ingress-controller-starterservices-gadev         320d
node-masters-role-binding                              329d
pod-reader                                             1y
run-alicloud-disk-controller                           1y
system:aws-cloud-provider                              1y
system:basic-user                                      1y
system:cloud-controller-manager                        1y
system:cloud-node-controller                           1y
system:controller:attachdetach-controller              1y
system:controller:certificate-controller               1y
system:controller:clusterrole-aggregation-controller   1y
system:controller:cronjob-controller                   1y
system:controller:daemon-set-controller                1y
system:controller:deployment-controller                1y
system:controller:disruption-controller                1y
system:controller:endpoint-controller                  1y
system:controller:expand-controller                    1y
system:controller:generic-garbage-collector            1y
system:controller:horizontal-pod-autoscaler            1y
system:controller:job-controller                       1y
system:controller:namespace-controller                 1y
system:controller:node-controller                      1y
system:controller:persistent-volume-binder             1y
system:controller:pod-garbage-collector                1y
system:controller:pv-protection-controller             1y
system:controller:pvc-protection-controller            1y
system:controller:replicaset-controller                1y
system:controller:replication-controller               1y
system:controller:resourcequota-controller             1y
system:controller:route-controller                     1y
system:controller:service-account-controller           1y
system:controller:service-controller                   1y
system:controller:statefulset-controller               1y
system:controller:ttl-controller                       1y
system:coredns                                         1y
system:discovery                                       1y
system:kube-controller-manager                         1y
system:kube-dns                                        1y
system:kube-scheduler                                  1y
system:node                                            1y
system:node-proxier                                    1y
system:pvl-controller                                  1y
system:route-controller                                1y
system:shared-informers                                1y
system:volume-scheduler                                1y
test-rolebinding                                       2h
tiller-cluster-rule                                    1y           

cs:admin, cs:ops, cs:dev, cs:restricted分别代表阿裡雲預置的4個角色：管理者，運維人員，開發人員，受限使用者，它們的權限依次遞減，其中受限使用者隻對部分資源具有讀權限。

我們檢視其中的一個ClusterRoleBinding, ID為202729146928739501使用者賦予了cs:ops的角色。

# kubectl get clusterrolebinding/202729146928739501-clusterrolebinding -o yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  creationTimestamp: 2019-08-19T05:22:03Z
  name: 202729146928739501-clusterrolebinding
  resourceVersion: "111811902"
  selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/202729146928739501-clusterrolebinding
  uid: 46ec07c5-c241-11e9-b90f-00163e00519d
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cs:ops
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: "202729146928739501"           

接下來就可以繼續檢視cs:ops到底具有什麼樣的權限:

# kubectl get clusterrole/cs:ops -o yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: 2019-04-22T07:57:30Z
  name: cs:ops
  resourceVersion: "47812320"
  selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/cs%3Aops
  uid: 472ef445-64d4-11e9-bb7e-00163e06e73f
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - pods/attach
  - pods/exec
  - pods/portforward
  - pods/proxy
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - persistentvolumeclaims
  - replicationcontrollers
  - replicationcontrollers/scale
  - secrets
  - serviceaccounts
  - services
  - services/proxy
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - bindings
  - events
  - limitranges
  - namespaces/status
  - replicationcontrollers/status
  - pods/log
  - pods/status
  - resourcequotas
  - resourcequotas/status
  - componentstatuses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - namespaces
  - nodes
  - persistentvolumes
  verbs:
  - get
  - list
  - watch
  - patch
- apiGroups:
  - apps
  resources:
  - daemonsets
  - deployments
  - deployments/rollback
  - deployments/scale
  - replicasets
  - replicasets/scale
  - statefulsets
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - extensions
  resources:
  - daemonsets
  - deployments
  - deployments/rollback
  - deployments/scale
  - ingresses
  - replicasets
  - replicasets/scale
  - replicationcontrollers/scale
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - servicecatalog.k8s.io
  resources:
  - clusterserviceclasses
  - clusterserviceplans
  - clusterservicebrokers
  - serviceinstances
  - servicebindings
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - servicecatalog.k8s.io
  resources:
  - clusterservicebrokers/status
  - clusterserviceclasses/status
  - clusterserviceplans/status
  - serviceinstances/status
  - serviceinstances/reference
  - servicebindings/status
  verbs:
  - update
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - alicloud.com
  resources:
  - '*'
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch           

阿裡雲授權頁面中的自定義就是可以選取叢集中已經存在的一些角色，當然也可以建立自己的權限和角色，然後配置設定給指定的使用者。

如何為Kubernetes建立一個獨立于阿裡雲的使用者并進行授權

除了在阿裡雲容器服務控制台中對阿裡雲使用者進行授權外，叢集管理者也可以單獨建立一個使用者并對其進行授權，這樣就可以給一個沒有阿裡雲賬戶的使用者通過api或者kubectl指令來使用和管理叢集。以下的步驟舉例說明如何建立使用者bob，并給bob授權，以及bob如何配置使用。

建立使用者

登陸到master節點
# cd /etc/kubernetes/pki
建立bob 私鑰
# openssl genrsa -out bob.key 2048
建立bob CSR，并将bob配置設定到test這個組中。
# openssl req -new -key bob.key -out bob.csr -subj "/CN=bob/O=test"
通過叢集CA簽發bob證書。管理者将證書檔案和bob的私鑰一起發給bob使用者。
# openssl x509 -req -in bob.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out bob.crt -days 365           

上面在建立bob CSR的時候/CN=bob代表使用者名，/O=test代表使用者所在的組。system:masters是系統組，如果選擇這個組，使用者預設就具有了cluster admin的權限。

bob如何配置使用證書

bob得到CA簽發的證書和私鑰後，首先需要下載下傳kubectl，然後通過下面指令配置.kube/config檔案。

#添加叢集連接配接資訊。包括CA憑證。
C:\Users\mdw3sr\Desktop>kubectl config set-cluster sandbox --certificate-authority=ca.crt --embed-certs=true --server=https://<K8S叢集公網IP>:6443
Cluster "sandbox" set.
#将bob密鑰資訊加入配置中
C:\Users\mdw3sr\Desktop>kubectl config set-credentials bob --client-certificate=bob.crt --client-key=bob.key --embed-certs=true
User "bob" set.
#将新的context入口加入配置中
C:\Users\mdw3sr\Desktop>kubectl config set-context bob --cluster=sandbox --user=bob
Context "kubernetes" created.
#使用bob context
C:\Users\mdw3sr\Desktop>kubectl config use-context bob
Switched to context "bob".           

這樣運作kubectl cluster-info就能看到叢集資訊了，雖然還有些資訊因為權限問題不能正确顯示。

C:\Users\mdw3sr\Desktop>kubectl cluster-info
Kubernetes master is running at https://106.15.157.9:6443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Error from server (Forbidden): services is forbidden: User "bob" cannot list resource "services" in API group "" in the namespace "kube-system"
C:\Users\mdw3sr\Desktop>kubectl get pod
No resources found.
Error from server (Forbidden): pods is forbidden: User "bob" cannot list resource "pods" in API group "" in the namespace "default"           

為使用者bob設定權限

從上面結果可以看出，如果沒有設定任何權限，bob在叢集内是沒有任何資源通路能力，是以管理者需要對bob進行叢集内授權。

為bob建立ClusterRole和ClusterRoleBinding
# cat > test.yaml <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: test-role
rules:
- apiGroups: [""]
  resources: ["pods","services"]
  verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: test-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: test-role
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: bob
EOF

# kubectl apply -f test.yaml
clusterrole.rbac.authorization.k8s.io/test-role created
clusterrolebinding.rbac.authorization.k8s.io/test-rolebinding created           

然後，bob這邊再測試之前的指令，就可以正确執行了。結果如下:

C:\Users\mdw3sr\Desktop>kubectl cluster-info
Kubernetes master is running at https://106.10.157.9:6443
Heapster is running at https://106.10.157.9:6443/api/v1/namespaces/kube-system/services/heapster/proxy
KubeDNS is running at https://106.10.157.9:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
monitoring-influxdb is running at https://106.10.157.9:6443/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

C:\Users\mdw3sr\Desktop>kubectl get pod
NAME                              READY     STATUS    RESTARTS   AGE
nginx-deployment-bb54749c-426cb   1/1       Running   0          104d
nginx-deployment-bb54749c-qcvc5   1/1       Running   0          104d
nginx-v1-5bc8bf76db-7rnkf         1/1       Running   1          230d
nginx-v1-5bc8bf76db-r7k5r         1/1       Running   0          185d
patch-demo-7f54465dd5-jhr5c       2/2       Running   0          29d
patch-demo-7f54465dd5-w5hx2       2/2       Running   0          29d

# 因為并沒有賦給bob 擷取namespace的權限，是以bob仍然不能通路namespace資訊。
C:\Users\mdw3sr\Desktop>kubectl get ns
No resources found.
Error from server (Forbidden): namespaces is forbidden: User "bob" cannot list resource "namespaces" in API group "" at the cluster scope           

大家也可以通過對bob設定不同的權限來學習驗證具體的行為。