《OpenShift 4.x HOL教程彙總》

文章目錄

檢視Windows節點的Kubernete版本
安裝通路Windows節點的SSH Jumphost
登入到Windows節點
在Windows節點操作容器環境
向Windows節點部署應用
用MachineSet擴充Windows節點
其他參考

說明：

本文主要示範如何在OpenShift 4.6上操作部署和管理Windows Container。
本文所用OpenShift 4.6叢集中需要有運作Windows Server 2019 Datacenter節點，使用它來運作Windows Container，
需要提前安裝配置好 Windows Machine Config Operator。

檢視Windows節點的Kubernete版本

檢視叢集節點基本資訊。可以看到ip-10-0-156-110.ap-southeast-1.compute.internal節點的Kubernetes版本是v1.19.0-rc.2.1023+f5121a6a6a02dd，和其他節點不同。

$ oc get nodes
NAME                                              STATUS   ROLES    AGE   VERSION
ip-10-0-146-72.ap-southeast-1.compute.internal    Ready    master   63m   v1.19.0+db1fc96
ip-10-0-154-146.ap-southeast-1.compute.internal   Ready    worker   49m   v1.19.0+db1fc96
ip-10-0-156-110.ap-southeast-1.compute.internal   Ready    worker   15m   v1.19.0-rc.2.1023+f5121a6a6a02dd
ip-10-0-178-234.ap-southeast-1.compute.internal   Ready    master   63m   v1.19.0+db1fc96
ip-10-0-189-77.ap-southeast-1.compute.internal    Ready    worker   50m   v1.19.0+db1fc96
ip-10-0-195-53.ap-southeast-1.compute.internal    Ready    master   63m   v1.19.0+db1fc96

查詢叢集中運作Windows的節點，該節點就是ip-10-0-156-110.ap-southeast-1.compute.internal。我們後面用的<WINDOWS_NODE>就是你環境中的這個Windows節點名。

$ oc get nodes -l kubernetes.io/os=windows -o wide
NAME                                              STATUS   ROLES    AGE   VERSION                            INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION    CONTAINER-RUNTIME
ip-10-0-156-110.ap-southeast-1.compute.internal   Ready    worker   16m   v1.19.0-rc.2.1023+f5121a6a6a02dd   10.0.156.110   <none>        Windows Server 2019 Datacenter   10.0.17763.1457   docker://19.3.11

安裝通路Windows節點的SSH Jumphost

建立内容如下的deploy-sshproxy.sh檔案。

#!/bin/bash
 
set -e
 
# Configuration via env. variables:
# Namespace where the bastion should run. The namespace will be created.
SSH_BASTION_NAMESPACE="${SSH_BASTION_NAMESPACE:-openshift-ssh-bastion}"
 
# Directory with bastion yaml files. Can be either local directory or http(s) URL.
BASEDIR="${BASEDIR:-https://raw.githubusercontent.com/eparis/ssh-bastion/master/deploy}"
 
clean_up () {
    ARG=$?
    rm -f "${RSATMP}" "${RSATMP}.pub"
    rm -f "${ECDSATMP}" "${ECDSATMP}.pub"
    rm -f "${ED25519TMP}" "${ED25519TMP}.pub"
    rm -f "${CONFIGFILE}"
    exit $ARG
}
trap clean_up EXIT
 
create_host_keys() {
    RSATMP=$(mktemp -u)
    /usr/bin/ssh-keygen -q -t rsa -f "${RSATMP}" -C '' -N ''
    ECDSATMP=$(mktemp -u)
    /usr/bin/ssh-keygen -q -t ecdsa -f "${ECDSATMP}" -C '' -N ''
    ED25519TMP=$(mktemp -u)
    /usr/bin/ssh-keygen -q -t ed25519 -f "${ED25519TMP}" -C '' -N ''
 
    CONFIGFILE=$(mktemp)
    echo 'HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
SyslogFacility AUTHPRIV
PermitRootLogin no
AuthorizedKeysFile      /home/core/.ssh/authorized_keys
PasswordAuthentication no
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials no
UsePAM yes
X11Forwarding yes
PrintMotd no
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
Subsystem       sftp    /usr/libexec/openssh/sftp-server
' > "${CONFIGFILE}"
 
    oc -n "${SSH_BASTION_NAMESPACE}" create secret generic ssh-host-keys --from-file="ssh_host_rsa_key=${RSATMP},ssh_host_ecdsa_key=${ECDSATMP},ssh_host_ed25519_key=${ED25519
TMP},sshd_config=${CONFIGFILE}"
}
 
# Non-namespaced objects
oc apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: ${SSH_BASTION_NAMESPACE}
  labels:
    openshift.io/run-level: "0"
EOF
oc apply -f "${BASEDIR}/clusterrole.yaml"
# using oc apply to modifty any already existing clusterrolebinding
oc create clusterrolebinding ssh-bastion --clusterrole=ssh-bastion --user="system:serviceaccount:${SSH_BASTION_NAMESPACE}:ssh-bastion" -o yaml --dry-run=client | oc apply -f
-
 
# Namespaced objects
oc -n "${SSH_BASTION_NAMESPACE}" apply -f "${BASEDIR}/service.yaml"
oc -n "${SSH_BASTION_NAMESPACE}" get secret ssh-host-keys &>/dev/null || create_host_keys
oc -n "${SSH_BASTION_NAMESPACE}" apply -f "${BASEDIR}/serviceaccount.yaml"
oc -n "${SSH_BASTION_NAMESPACE}" apply -f "${BASEDIR}/role.yaml"
oc -n "${SSH_BASTION_NAMESPACE}" create rolebinding ssh-bastion --clusterrole=ssh-bastion --user="system:serviceaccount:${SSH_BASTION_NAMESPACE}:ssh-bastion" -o yaml --dry-ru
n=client | oc apply -f -
oc -n "${SSH_BASTION_NAMESPACE}" apply -f "${BASEDIR}/deployment.yaml"
 
retry=120
while [ $retry -ge 0 ]
do
    retry=$((retry-1))
    bastion_host=$(oc get service -n "${SSH_BASTION_NAMESPACE}" ssh-bastion -o jsonpath='{.status.loadBalancer.ingress[0].hostname}' || true)
    if [ -n "${bastion_host}" ]; then
        break
    fi
    bastion_ip=$(oc get service -n "${SSH_BASTION_NAMESPACE}" ssh-bastion -o jsonpath='{.status.loadBalancer.ingress[0].ip}' || true)
    if [ -n "${bastion_ip}" ]; then
        break
    fi
    sleep 1
done
 
if [ -n "${bastion_host}" ]; then
    echo "Waiting for ${bastion_host} to show up in DNS"
    retry=120
    while [ $retry -ge 0 ]
    do
        retry=$((retry-1))
        if ! ((retry % 10)); then
            echo "...Still waiting for DNS..."
        fi
        if nslookup "${bastion_host}" > /dev/null ; then
            break
        else
            sleep 2
        fi
    done
else
    bastion_host="${bastion_ip}"
fi
 
echo "The bastion address is ${bastion_host}"
echo "You may want to use https://raw.githubusercontent.com/eparis/ssh-bastion/master/ssh.sh to easily ssh through the bastion to specific nodes."

執行指令，部署可以通路Windows節點的SSH Jumphost相關資源，它們被部署到openshift-ssh-bastion項目中。

$ bash deploy-sshproxy.sh 
namespace/openshift-ssh-bastion created
clusterrole.rbac.authorization.k8s.io/ssh-bastion created
clusterrolebinding.rbac.authorization.k8s.io/ssh-bastion created
service/ssh-bastion created
secret/ssh-host-keys created
serviceaccount/ssh-bastion created
role.rbac.authorization.k8s.io/ssh-bastion created
rolebinding.rbac.authorization.k8s.io/ssh-bastion created
deployment.apps/ssh-bastion created
Waiting for a396bfd3fdd644f29bed71e530a1341d-1824562040.ap-southeast-1.elb.amazonaws.com to show up in DNS
...Still waiting for DNS...
...Still waiting for DNS...
...Still waiting for DNS...
...Still waiting for DNS...
...Still waiting for DNS...
...Still waiting for DNS...
The bastion address is a396bfd3fdd644f29bed71e530a1341d-1824562040.ap-southeast-1.elb.amazonaws.com
You may want to use https://raw.githubusercontent.com/eparis/ssh-bastion/master/ssh.sh to easily ssh through the bastion to specific nodes.

執行指令，檢視部署資源的狀态。

$ oc get all -n openshift-ssh-bastion
NAME                               READY   STATUS    RESTARTS   AGE
pod/ssh-bastion-5fcf8d7d9b-d5788   1/1     Running   0          2m28s
 
NAME                  TYPE           CLUSTER-IP     EXTERNAL-IP                                                                    PORT(S)        AGE
service/ssh-bastion   LoadBalancer   172.30.4.134   a396bfd3fdd644f29bed71e530a1341d-1824562040.ap-southeast-1.elb.amazonaws.com   22:30421/TCP   2m32s
 
NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ssh-bastion   1/1     1            1           2m28s
 
NAME                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/ssh-bastion-5fcf8d7d9b   1         1         1       2m28s

登入到Windows節點

說明：以下操作都需要在Terminal1的Linux Bash中執行。

建立内容如下的sshcmd.sh檔案，我們稍後會用它來登入Windows節點并進入PowerShell。

#!/bin/bash
 
ssh_key_param=''
if [ ! -z "$SSH_KEY_PATH" ]; then
    ssh_key_param="-i $SSH_KEY_PATH"
fi
 
ingress_host="$(oc get service --all-namespaces -l run=ssh-bastion -o go-template='{{ with (index (index .items 0).status.loadBalancer.ingress 0) }}{{ or .hostname .ip }}{{end}}')"
ssh $ssh_key_param -t -o StrictHostKeyChecking=no -o ProxyCommand="ssh $ssh_key_param -A -o StrictHostKeyChecking=no -o ServerAliveInterval=30 -W %h:%p [email protected]${ingress_host}"
[email protected]$1 "powershell"

執行指令，開啟一個ssh shell session。

加入目前使用者的ssh key

$ ssh-add  ${HOME}/.ssh/id_rsa

執行腳本，進入<WINDOWS_NODE>節點，此時就會自動進入Powershell中。

$ bash sshcmd.sh <WINDOWS_NODE>
Warning: Permanently added 'a396bfd3fdd644f29bed71e530a1341d-1824562040.ap-southeast-1.elb.amazonaws.com,13.229.174.64' (ECDSA) to the list of known hosts.
Warning: Permanently added 'ip-10-0-156-110.ap-southeast-1.compute.internal' (ECDSA) to the list of known hosts.
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
 
PS C:\Users\Administrator>

在Windows節點操作容器環境

說明：以下操作全部在Terminal2的Powershell中執行。

檢視Windows節點中運作容器。

PS C:\Users\Administrator> docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

檢視Windows節點中的容器服務程序。

PS C:\Users\Administrator> Get-Process | ?{ $_.ProcessName -match "kube|overlay|docker" }
Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    414      22    68952      46184     198.47   2544   0 dockerd
    205      16    25884      32348       3.20   3848   0 hybrid-overlay-node
    467      24    45244      70212      34.83   3032   0 kubelet
    235      21    29772      40076      11.30   4676   0 kube-proxy

檢視Windows節點中的鏡像。

PS C:\Users\Administrator> docker images
REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE

執行指令，把mcr.microsoft.com/windows/servercore鏡像pull到Windows節點本地。

PS C:\Users\Administrator> docker pull mcr.microsoft.com/windows/servercore:ltsc2019

再次檢視Windows節點本地的鏡像。

PS C:\Users\Administrator> docker images
REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE
mcr.microsoft.com/windows/servercore   ltsc2019            71d3f266f3af        13 days ago         5.1GB

向Windows節點部署應用

說明：以下操作如未說明，都需要在Terminal1的Linux Bash中執行。

檢視Windows節點中的“Taints”屬性。下面将用該屬性作為選擇條件将Pod隻部署在Windows節點中。

$ oc describe node <WINDOWS_NODE> |grep -i Taint
Taints:             os=Windows:NoSchedule

建立内容如下的WinWebServer.yaml檔案。注意：在部署中定義了“tolerations”，使用它和Windows節點上的“Taint”屬性，讓該部署隻運作到Windows節點。

apiVersion: v1
kind: Service
metadata:
  name: win-webserver
  labels:
    app: win-webserver
spec:
  ports:
    # the port that this service should serve on
  - port: 80
    targetPort: 80
  selector:
    app: win-webserver
  type: LoadBalancer
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: win-webserver
  name: win-webserver
spec:
  selector:
    matchLabels:
      app: win-webserver
  replicas: 1
  template:
    metadata:
      labels:
        app: win-webserver
      name: win-webserver
    spec:
      tolerations:
      - key: "os"
        value: "Windows"
        Effect: "NoSchedule"
      containers:
      - name: windowswebserver
        image: mcr.microsoft.com/windows/servercore:ltsc2019
        imagePullPolicy: IfNotPresent
        command:
        - powershell.exe
        - -command
        - $listener = New-Object System.Net.HttpListener; $listener.Prefixes.Add('http://*:80/'); $listener.Start();Write-Host('Listening at http://*:80/'); while ($listener.IsListening) { $context = $listener.GetContext(); $response = $context.Response; $content='<html><body><H1>Windows Container Web Server</H1></body></html>'; $buffer = [System.Text.Encoding]::UTF8.GetBytes($content); $response.ContentLength64 = $buffer.Length; $response.OutputStream.Write($buffer, 0, $buffer.Length); $response.Close(); };
      nodeSelector:
        beta.kubernetes.io/os: windows

在default項目中建立WinWebServer.yaml的對象。

$ oc create -n default -f WinWebServer.yaml
service/win-webserver created
deployment.apps/win-webserver created

檢視default項目中的pod狀态，直到狀态變為“Running”。确認該pod是運作在<WINDOWS_NODE>上了。

$ oc get pods -n default -o wide
NAME                             READY   STATUS    RESTARTS   AGE   IP           NODE                                              NOMINATED NODE   READINESS GATES
win-webserver-549cd7495d-gsgsl   1/1     Running   0          75s   10.132.0.2   ip-10-0-156-110.ap-southeast-1.compute.internal   <none>           <none>

在Terminal2中的Powershell中執行以下指令，确認此時在Windows節點中已經有運作的Pod了。

PS C:\Users\Administrator> docker ps
CONTAINER ID        IMAGE                                          COMMAND                  CREATED             STATUS              PORTS               NAMES
929bc257e78a        71d3f266f3af                                   "powershell.exe -com…"   2 minutes ago       Up 2 minutes                            k8s_windowswebserver_win-webserver-549cd7495d-gsgsl_default_c0967447-1e00-4791-8a95-8e154c6a5810_0
c48c9f36a728        mcr.microsoft.com/oss/kubernetes/pause:1.3.0   "cmd /S /C pauseloop…"   2 minutes ago       Up 2 minutes                            k8s_POD_win-webserver-549cd7495d-gsgsl_default_c0967447-1e00-4791-8a95-8e154c6a5810_0

重新在Terminal1中執行指令，根據service建立route。

$ oc expose svc/win-webserver
route.route.openshift.io/win-webserver exposed

通過route通路win-webserver應用，确認可以正常通路運作在Window節點中的Pod。

$  curl -s http://$(oc get route win-webserver -n default -o jsonpath='{.spec.host}')
<html><body><H1>Windows Container Web Server</H1></body></html>[[email protected] ~]$

執行指令，進入運作在Window節點中的容器。

$ oc exec -it $(oc get pods -l app=win-webserver -o name) powershell
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
 
PS C:\>

從Window節點中的容器中退出。

PS C:\> exit

用MachineSet擴充Windows節點

檢視OpenShift中包含Windows的machinesets，以下将用<WINDOWS_MACHINE_SET>代表它。

$ oc get machinesets -n openshift-machine-api | grep windows
cluster-pek-2b3a-qxnvj-windows-ap-southeast-1a   1         1         1       1           54m

檢視OpenShift中運作Windows的machine。

$ oc get machines -n openshift-machine-api  -l machine.openshift.io/os-id=Windows
NAME                                                   PHASE     TYPE        REGION           ZONE              AGE
cluster-pek-2b3a-qxnvj-windows-ap-southeast-1a-lv7rq   Running   m5a.large   ap-southeast-1   ap-southeast-1a   56m

執行指令擴充<WINDOWS_MACHINE_SET>。

$ oc scale machineset <WINDOWS_MACHINE_SET> --replicas=2 -n openshift-machine-api
machineset.machine.openshift.io/cluster-pek-2b3a-qxnvj-windows-ap-southeast-1a scaled

檢視node的變化情況。需要一段時間部署新的node，最後确認OpenShift叢集會有2個Windows節點。

$ oc get nodes -l kubernetes.io/os=windows -w
NAME                                              STATUS   ROLES    AGE   VERSION
ip-10-0-156-110.ap-southeast-1.compute.internal   Ready    worker   64m   v1.19.0-rc.2.1023+f5121a6a6a02dd
ip-10-0-158-52.ap-southeast-1.compute.internal    NotReady   <none>   0s    v1.19.0-rc.2.1023+f5121a6a6a02dd
ip-10-0-158-52.ap-southeast-1.compute.internal    NotReady   <none>   0s    v1.19.0-rc.2.1023+f5121a6a6a02dd
ip-10-0-158-52.ap-southeast-1.compute.internal    NotReady   <none>   0s    v1.19.0-rc.2.1023+f5121a6a6a02dd
ip-10-0-158-52.ap-southeast-1.compute.internal    NotReady   <none>   0s    v1.19.0-rc.2.1023+f5121a6a6a02dd
ip-10-0-158-52.ap-southeast-1.compute.internal    NotReady   <none>   0s    v1.19.0-rc.2.1023+f5121a6a6a02dd
ip-10-0-158-52.ap-southeast-1.compute.internal    Ready      <none>   0s    v1.19.0-rc.2.1023+f5121a6a6a02dd
ip-10-0-158-52.ap-southeast-1.compute.internal    Ready      <none>   0s    v1.19.0-rc.2.1023+f5121a6a6a02dd
ip-10-0-158-52.ap-southeast-1.compute.internal    Ready      worker   1s    v1.19.0-rc.2.1023+f5121a6a6a02dd
 
$ oc get nodes -l kubernetes.io/os=windows
NAME                                              STATUS   ROLES    AGE   VERSION
ip-10-0-156-110.ap-southeast-1.compute.internal   Ready    worker   66m   v1.19.0-rc.2.1023+f5121a6a6a02dd
ip-10-0-158-52.ap-southeast-1.compute.internal    Ready    worker   17s   v1.19.0-rc.2.1023+f5121a6a6a02dd

其他參考

https://www.openshift.com/blog/announcing-the-community-windows-machine-config-operator-on-openshift-4.6

OpenShift 4.6 新特性 - 用 Windows MachineConfig Operator 管理 Windows Container檢視Windows節點的Kubernete版本安裝通路Windows節點的SSH Jumphost登入到Windows節點在Windows節點操作容器環境向Windows節點部署應用用MachineSet擴充Windows節點其他參考

文章目錄

檢視Windows節點的Kubernete版本

安裝通路Windows節點的SSH Jumphost

登入到Windows節點

在Windows節點操作容器環境

向Windows節點部署應用

用MachineSet擴充Windows節點

其他參考

繼續閱讀

k8s/openshift部署Hyperledger burrow

openshift 學習筆記-3 核心元件和概念

openshift 學習筆記-5 template使用

openshift 安裝

okd 3.11 安裝、使用Istio1.資源準備2.安裝okd 3.113. 安裝Istio4.部署helloworld

OpenShift 3.11 OKD安裝部署

部署OpenShift3.10版基礎配置搭建叢集OpenShift的配置

運作在筆記本上的OpenShift4環境

OpenShift推送自定義鏡像到本地倉庫并建立項目和應用

openshift介紹及基礎使用看看具體的指令：準備第1步：建立一個JBoss EAP的應用部署到OpenShift

OPENSHIFT-280-7-為項目建立定額與限額

容器入門（1） - 安裝和使用Docker Registry部署拓撲準備主控端安裝Docker Registry容器操作Docker Registry操作停止Docker Registry一些常見問題參考

OpenShift 4 - 部署Mirror Registry并複制Image安裝Mirror Registry運作環境複制Image到Mirror Registry參考

OpenShift 4 之在不同的項目之間推送Image

OpenShift 4 - 下載下傳OpenShift項目中的ImageStream

Hands-on Lab (10) - 增加管理者使用者說明和環境準備增加管理者使用者