雲原生CI/CD:Tekton/pipelin之pipeline概念篇
本節介紹下tekton中pipeline概念。作為雲原生的CI/CD神器在之前介紹的task和taskrun之後,還有什麼強大的功能呢?
Pipeline
用于定義一系列完成特定建構或傳遞目标的任務。pipeline的運作是由事件觸發或從PipelineRun調用。pipeline和task的差別在于,task隻能執行一個task,而pipeline中可以編排多個task,注意是編排,并不隻是簡單運作。pipeline的spec.tasks定義了需要編排的task,是個數組,而這個數組中的task的順序并不一定是執行順序,pipeline中task的執行順序是可以指定的。下面講下pipeline的使用:
1.1 Declared resources
為了使pipeline能與外界互動,可能需要PipelineResources,PipelineResources作為輸入和輸出提供給Tasks。下面就是pipeline中使用pipelineResource:
spec:
resources:
- name: my-repo
type: git
- name: my-image
type: image
1.2 Workspaces
工作空間是一種為執行中的管道及其任務提供可用的共享卷的方法。在pipeline中定義worksapce作為共享卷傳遞個相關的task。在tekton中定義workspace的用途有以下幾點:
- 存儲輸入和/或輸出
- 在task之間共享資料
- secret認證的挂載點
- ConfigMap中儲存的配置的挂載點
- 組織共享的常用工具的挂載點
-
高速緩存的建構工件可加快工作速度,簡而言之,用于緩存建構時的包,例如作為Maven倉庫存儲
使用方式如下:
spec:
workspaces:
- name: pipeline-ws1 # workspace的名稱
tasks:
- name: use-ws-from-pipeline
taskRef:
name: gen-code # 使用的Task
workspaces:
- name: output
workspace: pipeline-ws1
- name: use-ws-again
taskRef:
name: commit # 使用的task
runAfter:
- use-ws-from-pipeline # 定義任務的執行順序,該task在use-ws-from-pipeline之後執行
workspaces:
- name: src
workspace: pipeline-ws1
1.3 使用task和pipelineresource
pipeline中使用已經定義好的task和pipelineResource:
spec:
tasks:
- name: build-the-image
taskRef:
name: build-push # 使用定義好的task
resources:
inputs:#輸入資源 代碼
- name: workspace
resource: my-repo
outputs:#輸出資源 鏡像
- name: image
resource: my-image
1.4 from
你可能需要将先前任務的輸出作為輸入,舉個例子:
- name: build-app
taskRef:
name: build-push
resources:
outputs: #定義任務輸出
- name: image
resource: my-image
- name: deploy-app
taskRef:
name: deploy-kubectl
resources:
inputs:#定義任務輸入
- name: image
resource: my-image
from:
- build-app #任務輸入源,也意味着任務deploy-kubectl要在任務build-app之後執行
resource my-image将會從build-app的輸出作為deploy-app的輸入,是以my-image必須是build-app任務的輸出結果,當然這也意味着build-app必須先于deloy-app運作完,無論它們在定義中出現的順序如何。
1.5 runAfter
有時,您需要具有按特定順序運作的pipeline task,但它們沒有明确的輸出來輸入依賴項(通過from表示)。在這種情況下,可以使用runAfter訓示應在一個或多個先前的管道任務之後執行管道任務。
- name: test-app
taskRef:
name: make-test
resources:
inputs:
- name: workspace
resource: my-repo
- name: build-app
taskRef:
name: kaniko-build
runAfter:
- test-app #build-app任務在test-app任務之後執行
resources:
inputs:
- name: workspace
resource: my-repo
build-app任務會在test-app之後執行
1.6 retries
有時,你需要一項重試政策以應對可能會遇到的網絡錯誤、缺少依賴或者上傳問題等.
預設retries确省值為0,不會重試,自定義重試政策如下:
tasks:
- name: build-the-image
retries: 1
taskRef:
name: build-push
build-the-image任務在第一次失敗後,馬上就會啟動第二個。目前設定隻能重試1次。
1.7 conditions
有時你需要在某些條件為true的情況下才去執行task,condition字段允許您列出對在任務運作之前運作的條件的一系列引用。如果所有條件都為真,則運作任務。要是有一個條件不滿足,那任務就不會運作,同時任務的status标志位會被置為ConditionCheckFailed。注意正常來講,一個task不能運作,不會影響整個pipelinerun。pipeline中使用condition:
tasks:
- name: conditional-task
taskRef:
name: build-push
conditions:
- conditionRef: my-condition #使用已有condition
params:
- name: my-param
value: my-value
resources:
- name: workspace
resource: source-repo
關于如何定義condition,後面找個機會詳細找下。
1.8 Timeout
Pipeline Task的Timeout屬性允許為PipelineRun的一部分TaskRun定義逾時。如果TaskRun超過指定的時間,則TaskRun将失敗,并且與Pipeline關聯的PipelineRun也将失敗。Pipeline的Tasks沒有預設逾時設定,是以在定義pipeline時必須使用pipeline task指定逾時。帶有逾時的管道任務示例如下所示:
spec:
tasks:
- name: build-the-image
taskRef:
name: build-push
Timeout: "0h1m30s"
timeout的完整例子:
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
name: task-echo-message
spec:
inputs:
params:
- name: MESSAGE
type: string
default: "Hello World"
steps:
- name: echo
image: ubuntu
command:
- sleep 90s
args:
- "$(inputs.params.MESSAGE)"
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
name: pipelinerun-timeout
spec:
# 1 hour and half timeout
timeout: 1h30m
pipelineSpec:
params:
- name: MORNING_GREETINGS
description: "morning greetings, default is Good Morning!"
type: string
default: "Good Morning!"
- name: NIGHT_GREETINGS
description: "Night greetings, default is Good Night!"
type: string
default: "Good Night!"
tasks:
# Task to display morning greetings
- name: echo-good-morning
taskRef:
name: task-echo-message
params:
- name: MESSAGE
value: $(params.MORNING_GREETINGS)
# Task to display night greetings
- name: echo-good-night
taskRef:
name: task-echo-message
params:
- name: MESSAGE
value: $(params.NIGHT_GREETINGS)
params:
- name: MORNING_GREETINGS
value: "Good Morning, Bob!"
- name: NIGHT_GREETINGS
value: "Good Night, Bob!"
注意:設定逾時時間是很有必要的,不然task 的pod會一直運作不完,浪費k8s叢集資源,對于逾時的任務确實應該kill掉。我之前建構Java項目時,使用的maven倉庫沒有配置阿裡雲的源,拉包很慢,跑了一個小時,最後到了預設的逾時時間才強制關掉任務。
1.8 Results
piple中可以使用task的運作結果作為其他Task的輸入,即task可在執行過程中生成一些result,這些result可用作pipeline後續task中的參數值,此外Tekton将根據輸入參數來推斷tasks的執行順序,以確定生成result的task在那些消耗其結果的task之前運作。
通過變量替換将Task結果用作另一個Task參數的值:
params:
- name: foo
value: "$(tasks.previous-task-name.results.bar-result)"
"previous-task-name"産生的result被用于參數值。完整的例子如下:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: sum-and-multiply-pipeline
spec:
params:
- name: a
type: string
default: "1"
- name: b
type: string
default: "1"
tasks:
- name: sum-inputs
taskRef:
name: sum
params:
- name: a
value: "$(params.a)"
- name: b
value: "$(params.b)"
- name: multiply-inputs
taskRef:
name: multiply
params:
- name: a
value: "$(params.a)"
- name: b
value: "$(params.b)"
- name: sum-and-multiply
taskRef:
name: sum
params:
- name: a
value: "$(tasks.multiply-inputs.results.product)$(tasks.sum-inputs.results.sum)" #該任務在multiply-inputs之後執行
- name: b
value: "$(tasks.multiply-inputs.results.product)$(tasks.sum-inputs.results.sum)"
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: sum
annotations:
description: |
A simple task that sums the two provided integers
spec:
params:
- name: a
type: string
default: "1"
description: The first integer
- name: b
type: string
default: "1"
description: The second integer
results:
- name: sum
description: The sum of the two provided integers
steps:
- name: sum
image: bash:latest
script: |
#!/usr/bin/env bash
echo -n $(( "$(params.a)" + "$(params.b)" )) | tee $(results.sum.path)
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: multiply
annotations:
description: |
A simple task that multiplies the two provided integers
spec:
params:
- name: a
type: string
default: "1"
description: The first integer
- name: b
type: string
default: "1"
description: The second integer
results:
- name: product
description: The product of the two provided integers
steps:
- name: product
image: bash:latest
script: |
#!/usr/bin/env bash
echo -n $(( "$(params.a)" * "$(params.b)" )) | tee $(results.product.path)
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: sum-and-multiply-pipeline-run-
spec:
pipelineRef:
name: sum-and-multiply-pipeline
params:
- name: a
value: "2"
- name: b
value: "10"
1.9 Ordering
前面有講過單獨使用from和runAfter這邊講下,這兩者聯合使用能到達指定順序執行任務:
- name: lint-repo
taskRef:
name: pylint
resources:
inputs:
- name: workspace
resource: my-repo
- name: test-app
taskRef:
name: make-test
resources:
inputs:
- name: workspace
resource: my-repo
- name: build-app
taskRef:
name: kaniko-build-app
runAfter:
- test-app #build-app在test-app之後執行
resources:
inputs:
- name: workspace
resource: my-repo
outputs:
- name: image
resource: my-app-image
- name: build-frontend
taskRef:
name: kaniko-build-frontend
runAfter:
- test-app #build-frontend在test-app之後執行
resources:
inputs:
- name: workspace
resource: my-repo
outputs:
- name: image
resource: my-frontend-image
- name: deploy-all
taskRef:
name: deploy-kubectl
resources:
inputs:
- name: my-app-image
resource: my-app-image
from:
- build-app # 在build-app之後執行
- name: my-frontend-image
resource: my-frontend-image
from:
- build-frontend #在build-frontend之後執行
執行過程圖:
| |
v v
test-app lint-repo
/ \
v v
build-app build-frontend
\ /
v v
deploy-all
總結
對于比較複雜的CI/CD任務或者需要指定執行順序時,可以選擇使用pipeline來運作,pipeline定義好了就建立,然後不用管了。在運作pipelineRun時指定必要的參數,每次運作建構任務時,隻要運作pipelineRun就行,pipelineRun的使用我們下一次再講。總體來說,pipeline的功能相對于task來說還比較全,之後我會找一些場景進行示範。
歡迎關注“南君手記”公衆号,歡迎評論指正。技術之路,我們一起成長!