前言
Kong is a cloud-native, fast, scalable, and distributed Microservice Abstraction Layer (also known as an API Gateway or API Middleware). Made available as an open-source project in 2015, its core values are high performance and extensibility.
Actively maintained, Kong is widely used in production at companies ranging from startups to Global 5000 as well as government organizations.
Kong是目前社群最流行的雲原生API網關。
高性能
和
可擴充
兩大特性使得Kong被各大廠商廣泛使用
在深入Kong使用前有必要對Kong的作用進行概述：
If you are building for the web, mobile, or IoT (Internet of Things) you will likely end up needing common functionality to run your actual software. Kong can help by acting as a gateway (or a sidecar) for microservices requests while providing load balancing, logging, authentication, rate-limiting, transformations, and more through plugins.
也即在進行微服務開發時，我們需要一些公共的特性和功能，例如：日志、負載均衡、認證以及
Rate limiting
等。而Kong(API網關)便充當着這個角色，使服務與這些公共功能解耦，讓開發者更加專注于自身的服務開發和運維，從這些繁瑣的外圍事情中解脫出來。更直覺的對比如下：
在舊的服務管理體制下，各個服務需要各自開發具有相同功能的諸如日志、認證以及
Rate limiting
等子產品，不僅增加了開發者負擔也增加了整個系統的備援度；而對比Kong(API網關)作為這些公共服務的統一接入層，所有外圍服務均由Kong實作，整個系統結構清晰且易維護
Kong
這裡我們從Kong Admin API為切入點深入Kong的使用
一、Kong Admin API
By default Kong listens on the following ports:
-
:8000
-
:8443
-
:8001
-
:8444
如圖：
- 1、Proxy端口(
8000
8443
- 2、Admin端口(
8001
8444
二、Kong Configuration Mode
在詳細介紹Kong具體使用之前，我們先介紹一下Kong的兩種使用模式：
- DB-less mode：使用declarative configuration，所有配置存放于一個配置檔案中(YAML or JSON格式），不需要使用資料庫，而修改配置的方法有兩種：
- 1、靜态——在kong初始化時指定
declarative_config
$ export KONG_DATABASE=off $ export KONG_DECLARATIVE_CONFIG=kong.yml $ kong start -c kong.conf - 2、動态——在kong運作期間，調用Kong Admin API：
$ http :8001/config [email protected]
- 1、靜态——在kong初始化時指定
- DB mode: 使用imperative configuration，需要使用資料庫(PostgreSQL or Cassandra)，并通過Kong Admin API對配置進行CRUD操作
這兩種模式各有優缺點，如下：
- DB-less mode
- Pros:
- 1、無需使用資料庫，減少了對資料庫的依賴，減少部署&運維成本
- Cons:
- 1、由于采用declarative configuration設計，更新規則必須全量更新，重置整個配置檔案，無法做到局部更新(調用Kong Admin API/config)
- 2、不支援Konga對Kong的管理
- 3、插件相容性較差，無法支援所有Kong插件，詳情見Plugin Compatibility
- Pros:
- DB mode
- Pros:
- 1、支援調用Kong Admin API CRUD，支援局部更新
- 2、支援Konga對Kong的管理
- 3、插件相容性好，可以支援所有Kong插件
- Cons:
- 1、需要使用資料庫，增加了對資料庫的依賴，增加部署&運維成本
- Pros:
三、Kong Used As HTTP Proxy
由于Kong DB mode更加便于舉例說明，這裡我們采用Kong DB mode展示如何使用Kong代理HTTP請求
首先介紹一下Kong Proxy幾個關鍵概念：
- client: Refers to the downstream client making requests to Kong’s proxy port.
- upstream service: Refers to your own API/service sitting behind Kong, to which client requests/connections are forwarded.
- Service: Service entities, as the name implies, are abstractions of each of your own upstream services. Examples of Services would be a data transformation microservice, a billing API, etc.
- Route: This refers to the Kong Routes entity. Routes are entrypoints into Kong, and defining rules for a request to be matched, and routed to a given Service.
- Target: A target is an ip address/hostname with a port that identifies an instance of a backend service. Every upstream can have many targets, and the targets can be dynamically added. Changes are effectuated on the fly.
- Plugin: This refers to Kong “plugins”, which are pieces of business logic that run in the proxying lifecycle. Plugins can be configured through the Admin API - either globally (all incoming traffic) or on specific Routes and Services.
舉一個例子對上述概念進行說明：
一個典型的 Nginx 配置：
upstream testUpstream {
server localhost:3000 weight=100;
}
server {
listen 80;
location /test {
proxy_pass http://testUpstream;
}
}
轉換為Kong Admin API請求如下：
# configurate service
curl -X POST http://localhost:8001/services --data "name=test" --data "host=testUpstream"
# configurate route
curl -X POST http://localhost:8001/routes --data "paths[]=/test" --data "service.id=92956672-f5ea-4e9a-b096-667bf55bc40c"
# configurate upstream
curl -X POST http://localhost:8001/upstreams --data "name=testUpstream"
# configurate target
curl -X POST http://localhost:8001/upstreams/testUpstream/targets --data "target=localhost:3000" --data "weight=100"
從這個例子可以看出：
- Service：Kong服務抽象層，可以直接映射到一個實體服務，也可以指向一個Upstream來做到負載均衡
- Route：Kong路由抽象層，負責将實際請求映射到相應的Service
- Upstream：後端服務抽象，主要用于負載均衡
- Target：代表了Upstream中的一個後端服務，是 ip(hostname) + port 的抽象
也即通路鍊路：Route => Service => Upstream => Target
下面給一個
Kong Used As HTTP Proxy
的例子，如下：
# step1: create nginx service
$ cat << EOF > nginx-svc.yml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.15
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
EOF
$ kubectl apply -f nginx-svc.yml
deployment.apps/nginx created
service/nginx created
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP 172.28.255.197 <none> 80/TCP 5h18m
# step2: create kong nginx service
$ curl -s -X POST --url http://172.28.255.207:8001/services/ \
> -d 'name=nginx' \
> -d 'protocol=http' \
> -d 'host=nginxUpstream' \
> -d 'port=80' \
> -d 'path=/' \
> | python -m json.tool
{
"client_certificate": null,
"connect_timeout": 60000,
"created_at": 1580560293,
"host": "nginxUpstream",
"id": "14100336-f5d2-48ef-a720-d341afceb466",
"name": "nginx",
"path": "/",
"port": 80,
"protocol": "http",
"read_timeout": 60000,
"retries": 5,
"tags": null,
"updated_at": 1580560293,
"write_timeout": 60000
}
# step3: create kong nginx route
$ curl -s -X POST --url http://172.28.255.207:8001/services/nginx/routes \
> -d 'name=nginx' \
> -d 'hosts[]=nginx-test.duyanghao.com' \
> -d 'paths[]=/' \
> -d 'strip_path=true' \
> -d 'preserve_host=true' \
> -d 'protocols[]=http' \
> | python -m json.tool
{
"created_at": 1580560619,
"destinations": null,
"headers": null,
"hosts": [
"nginx-test.duyanghao.com"
],
"https_redirect_status_code": 426,
"id": "bb678485-0b3e-4e8a-9a46-3e5464fedffc",
"methods": null,
"name": "nginx",
"paths": [
"/"
],
"preserve_host": true,
"protocols": [
"http"
],
"regex_priority": 0,
"service": {
"id": "14100336-f5d2-48ef-a720-d341afceb466"
},
"snis": null,
"sources": null,
"strip_path": true,
"tags": null,
"updated_at": 1580560619
}
# step4: create kong nginx upstream
$ curl -s -X POST --url http://172.28.255.207:8001/upstreams \
> -d 'name=nginxUpstream' \
> | python -m json.tool
{
"algorithm": "round-robin",
"created_at": 1580560763,
"hash_fallback": "none",
"hash_fallback_header": null,
"hash_on": "none",
"hash_on_cookie": null,
"hash_on_cookie_path": "/",
"hash_on_header": null,
"healthchecks": {
"active": {
"concurrency": 10,
"healthy": {
"http_statuses": [
200,
302
],
"interval": 0,
"successes": 0
},
"http_path": "/",
"https_sni": null,
"https_verify_certificate": true,
"timeout": 1,
"type": "http",
"unhealthy": {
"http_failures": 0,
"http_statuses": [
429,
404,
500,
501,
502,
503,
504,
505
],
"interval": 0,
"tcp_failures": 0,
"timeouts": 0
}
},
"passive": {
"healthy": {
"http_statuses": [
200,
201,
202,
203,
204,
205,
206,
207,
208,
226,
300,
301,
302,
303,
304,
305,
306,
307,
308
],
"successes": 0
},
"type": "http",
"unhealthy": {
"http_failures": 0,
"http_statuses": [
429,
500,
503
],
"tcp_failures": 0,
"timeouts": 0
}
}
},
"id": "a4c88440-bd50-48f1-8926-527d02abc4a2",
"name": "nginxUpstream",
"slots": 10000,
"tags": null
}
# step5: create kong nginx target
$ curl -s -X POST --url http://172.28.255.207:8001/upstreams/nginxUpstream/targets -d 'target=172.28.255.197:80' | python -m json.tool
{
"created_at": 1580570363.097,
"id": "09871dc5-5ede-4ea3-b232-e52501be071a",
"target": "172.28.255.197:80",
"upstream": {
"id": "a4c88440-bd50-48f1-8926-527d02abc4a2"
},
"weight": 100
}
# Test forward through Kong
$ curl -s -X GET \
--url http://172.28.255.9:8000/ \
--header 'Host: nginx-test.duyanghao.com'
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
可以看到通過Kong Admin API建立Service、Route、Upstream以及Target等配置後，Kong成功地代理了nginx service的請求
另外，Kong除了可以代理HTTP請求，還可以作為WebSocket、TCP以及gRPC Proxy，詳見Kong proxy
四、Kong Plugins
You’ve probably heard that Kong is built on Nginx, leveraging its stability and efficiency. But how is this possible exactly?
To be more precise, Kong is a Lua application running in Nginx and made possible by the lua-nginx-module. Instead of compiling Nginx with this module, Kong is distributed along with OpenResty, which already includes lua-nginx-module. OpenResty is not a fork of Nginx, but a bundle of modules extending its capabilities.
This sets the foundations for a pluggable architecture, where Lua scripts (referred to as ”plugins”) can be enabled and executed at runtime. Because of this, we like to think of Kong as a paragon of microservice architecture: at its core, it implements database abstraction, routing and plugin management. Plugins can live in separate code bases and be injected anywhere into the request lifecycle, all in a few lines of code.
由于Kong是OpenResty 應用程式，天然支援Lua插件，我們可以給Kong Service或者Route等屬性添加插件，來達到更多定制化需求，如下：
# add plugin for kong nginx service
$ curl -X POST http://172.28.255.207:8001/services/nginx/plugins \
--data "name=rate-limiting" \
--data "config.second=50" \
| python -m json.tool
{
"config": {
"day": null,
"fault_tolerant": true,
"hide_client_headers": false,
"hour": null,
"limit_by": "consumer",
"minute": null,
"month": null,
"policy": "cluster",
"redis_database": 0,
"redis_host": null,
"redis_password": null,
"redis_port": 6379,
"redis_timeout": 2000,
"second": 50,
"year": null
},
"consumer": null,
"created_at": 1580567002,
"enabled": true,
"id": "ce629c6f-046a-45fa-bb0a-2e6aaea70a83",
"name": "rate-limiting",
"protocols": [
"grpc",
"grpcs",
"http",
"https"
],
"route": null,
"run_on": "first",
"service": {
"id": "14100336-f5d2-48ef-a720-d341afceb466"
},
"tags": null
}
# Test forward through Kong
$ curl -v -s -X GET \
--url http://172.28.255.9:8000/ \
--header 'Host: nginx-test.duyanghao.com'
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=UTF-8
< Content-Length: 612
< Connection: keep-alive
< Server: nginx/1.15.12
< Date: Sat, 01 Feb 2020 14:33:29 GMT
< Vary: Accept-Encoding
< Last-Modified: Tue, 16 Apr 2019 13:08:19 GMT
< ETag: "5cb5d3c3-264"
< Accept-Ranges: bytes
< X-RateLimit-Remaining-second: 49
< X-RateLimit-Limit-second: 50
< X-Kong-Upstream-Latency: 1
< X-Kong-Proxy-Latency: 11
< Via: kong/1.4.2
<
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
# add plugin for kong nginx route
$ curl -X POST http://172.28.255.207:8001/routes/nginx/plugins \
--data "name=rate-limiting" \
--data "config.second=50" \
| python -m json.tool
{
"config": {
"day": null,
"fault_tolerant": true,
"hide_client_headers": false,
"hour": null,
"limit_by": "consumer",
"minute": null,
"month": null,
"policy": "cluster",
"redis_database": 0,
"redis_host": null,
"redis_password": null,
"redis_port": 6379,
"redis_timeout": 2000,
"second": 50,
"year": null
},
"consumer": null,
"created_at": 1580567880,
"enabled": true,
"id": "f4e18187-b24e-437c-95e3-485589f0e326",
"name": "rate-limiting",
"protocols": [
"grpc",
"grpcs",
"http",
"https"
],
"route": {
"id": "bb678485-0b3e-4e8a-9a46-3e5464fedffc"
},
"run_on": "first",
"service": null,
"tags": null
}
# Test forward through Kong
$ curl -v -s -X GET \
--url http://172.28.255.9:8000/ \
--header 'Host: nginx-test.duyanghao.com'
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=UTF-8
< Content-Length: 612
< Connection: keep-alive
< Server: nginx/1.15.12
< Date: Sat, 01 Feb 2020 14:54:14 GMT
< Vary: Accept-Encoding
< Last-Modified: Tue, 16 Apr 2019 13:08:19 GMT
< ETag: "5cb5d3c3-264"
< Accept-Ranges: bytes
< X-RateLimit-Remaining-second: 49
< X-RateLimit-Limit-second: 50
< X-Kong-Upstream-Latency: 1
< X-Kong-Proxy-Latency: 6
< Via: kong/1.4.2
<
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
Kong-ingress-controller
上述
Kong Used As HTTP Proxy
給出了使用Admin API配置Kong來通路Kubernetes nginx service的例子，這種方法屬于Kong原生的方式，并沒有針對Kubernetes進行定制。如果像上述例子一樣采用Kong Admin API對Kubernetes service進行通路設定，則需要我們自己開發對應的Kong API CRUD邏輯。而Kong官方為了解決這個共性問題(也是為了擴充Kong Cloud-Native的使用場景)建立了kubernetes-ingress-controller項目
一、Kong-ingress-controller Principle
Kong-ingress-controller采用标準的Kubernetes Operator(CRDs + Controller)模式編寫，按元件分為：
- Kong, the core proxy that handles all the traffic
- Controller, a process that syncs the configuration from Kubernetes to Kong
整個架構類似nginx-ingress-controller(Nginx + Controller)，原理如下：
Controller會監控Kubernetes叢集中資源的變化，然後更新Kong相應的配置使其代理Kubernetes服務，這裡面就存在Kubernetes資源與Kong資源對應的關系，如下：
- An Ingress resource in Kubernetes defines a set of rules for proxying traffic. These rules corresponds to the concept or Route in Kong.
- A Service inside Kubernetes is a way to abstract an application that is running on a set of pods. This maps to two objects in Kong: Service and Upstream. The service object in Kong holds the information on the protocol to use to talk to the upstream service and various other protocol specific settings. The Upstream object defines load balancing and healthchecking behavior.
- Pods associated with a Service in Kubernetes map as a Target belonging to the Upstream (the upstream corresponding to the Kubernetes Service) in Kong. Kong load balances across the Pods of your service. This means that all requests flowing through Kong are not directed via kube-proxy but directly to the pod.
也即Kong巧妙地将Kubernetes Ingress轉化為Kong Route；Kubernetes Service轉化為Kong Service&Upstream；Kubernetes Pod轉化為Kong Target。最終實作Kong代理Kubernetes服務
二、Kong-ingress-controller Install&Usage
我們按照官方指引安裝kong-ingress-controller，如下：
# install kong-ingress-controller
$ kubectl apply -f https://bit.ly/k4k8s
namespace/kong created
customresourcedefinition.apiextensions.k8s.io/kongconsumers.configuration.konghq.com created
customresourcedefinition.apiextensions.k8s.io/kongcredentials.configuration.konghq.com created
customresourcedefinition.apiextensions.k8s.io/kongingresses.configuration.konghq.com created
customresourcedefinition.apiextensions.k8s.io/kongplugins.configuration.konghq.com created
serviceaccount/kong-serviceaccount created
clusterrole.rbac.authorization.k8s.io/kong-ingress-clusterrole created
clusterrolebinding.rbac.authorization.k8s.io/kong-ingress-clusterrole-nisa-binding created
configmap/kong-server-blocks created
service/kong-admin created
service/kong-proxy created
service/kong-validation-webhook created
deployment.apps/ingress-kong created
# kubectl get all -nkong
NAME READY STATUS RESTARTS AGE
pod/ingress-kong-7875999c56-cbfzs 2/2 Running 1 2m59s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kong-admin ClusterIP 172.71.71.166 <none> 8001/TCP 2m59s
service/kong-proxy NodePort 172.71.138.94 <none> 80:8000/TCP,443:8443/TCP 2m59s
service/kong-validation-webhook ClusterIP 172.71.9.238 <none> 443/TCP 2m59s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ingress-kong 1/1 1 1 2m59s
NAME DESIRED CURRENT READY AGE
replicaset.apps/ingress-kong-7875999c56 1 1 1 2m59s
安裝完kong-ingress-controller後，我們建立nginx svc&ingress，如下：
# step1: create nginx svc&ingress
$ cat << EOF > nginx-svc.yml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.15
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: nginx
spec:
rules:
- host: nginx-test.duyanghao.com
http:
paths:
- path: /
backend:
serviceName: nginx
servicePort: 80
EOF
$ kubectl apply -f nginx-svc.yml
deployment.apps/nginx created
service/nginx created
ingress.networking.k8s.io/nginx created
建立完nginx service&ingress後，我們通路nginx如下：
# Test forward through Kong
$ curl -s -X GET \
--url http://172.71.138.94:8000/ \
--header 'Host: nginx-test.duyanghao.com'
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
可以看到，相比Kong原生的用法，這裡沒有調用Kong Admin API建立各種Kong屬性，而隻是添加了一個Kubernetes Ingress，就可以直接通路服務了，簡單且實用
三、Kong-ingress-controller CRDs
Kong Ingress Controller performs more than just proxying the traffic coming into a Kubernetes cluster. It is possible to configure plugins, load balancing, health checking and leverage all that Kong offers in a standalone installation.
Kong-ingress-controller原則上可以完成Kong所涵蓋的所有特性。為此，Kong-ingress-controller給出了四種CRDs對功能進行擴充，如下：
- KongIngress
- KongPlugin
- KongConsumer
- KongCredential (Deprecated)
這裡主要介紹KongIngress與KongPlugin CRDs的用法：
Kong-ingress-controller CRDs-KongIngress
The Ingress resource in Kubernetes is a fairly narrow and ambiguous API, and doesn’t offer resources to describe the specifics of proxying. To overcome this limitation,
KongIngress
Custom Resource is used as an “extension" to the existing Ingress API to provide fine-grained control over proxy behavior. In other words,
KongIngress
works in conjunction with the existing Ingress resource and extends it. It is not meant as a replacement for the
Ingress
resource in Kubernetes. Using
KongIngress
, all properties of Upstream, Service and Route entities in Kong related to an Ingress resource can be modified.
Once a
KongIngress
resource is created, you can use the
configuration.konghq.com
annotation to associate the
KongIngress
resource with an
Ingress
or a
Service
resource:
- When the annotation is added to the
Ingress
resource, the routing
configurations are updated, meaning all routes associated with the annotated
Ingress
KongIngress
's route
section.
- When the annotation is added to a
Service
resource in Kubernetes,
the corresponding
Service
Upstream
proxy
upstream
KongIngress
The below diagram shows how the resources are linked
with one another:
也即：
KongIngress
用于擴充Kubernetes Ingress資源功能，進而更加細粒度控制代理行為。這裡舉例進行說明：
# Example1: Use KongIngress with Ingress resource
# Install a dummy service
$ cat << EOF > echo-service.yaml
---
apiVersion: v1
kind: Service
metadata:
labels:
app: echo
name: echo
spec:
ports:
- port: 8080
name: high
protocol: TCP
targetPort: 8080
- port: 80
name: low
protocol: TCP
targetPort: 8080
selector:
app: echo
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: echo
name: echo
spec:
replicas: 1
selector:
matchLabels:
app: echo
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: echo
spec:
containers:
- image: gcr.io/kubernetes-e2e-test-images/echoserver:2.2
name: echo
ports:
- containerPort: 8080
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
resources: {}
EOF
$ kubectl apply -f echo-service.yaml
service/echo created
deployment.apps/echo created
# Setup Ingress
$ echo "
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: demo
spec:
rules:
- http:
paths:
- path: /foo
backend:
serviceName: echo
servicePort: 80
" | kubectl apply -f -
ingress.extensions/demo created
# Let's test
$ curl -i http://172.71.138.94:8000/foo
HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Date: Sun, 02 Feb 2020 07:42:28 GMT
Server: echoserver
X-Kong-Upstream-Latency: 0
X-Kong-Proxy-Latency: 1
Via: kong/1.4.2
Hostname: echo-85fb7989cc-kk7r6
Pod Information:
node name: vm-xxx-centos
pod name: echo-85fb7989cc-kk7r6
pod namespace: default
pod IP: 172.70.0.21
Server values:
server_version=nginx: 1.12.2 - lua: 10010
Request Information:
client_address=172.70.0.17
method=GET
real path=/
query=
request_version=1.1
request_scheme=http
request_uri=http://172.71.138.94:8080/
# Kong will strip the path defined in the Ingress rule before proxying the request to the service. This can be seen in the real path value in the above response.
# We can configure Kong to not strip out this path and to only respond to GET requests for this particular Ingress rule.
# create a KongIngress resource
$ echo "apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
name: sample-customization
route:
methods:
- GET
strip_path: false" | kubectl apply -f -
kongingress.configuration.konghq.com/sample-customization created
# Now, let's associate this KongIngress resource with our Ingress resource using the configuration.konghq.com annotation.
$ kubectl patch ingress demo -p '{"metadata":{"annotations":{"configuration.konghq.com":"sample-customization"}}}'
ingress.extensions/demo patched
# Now, Kong will proxy only GET requests on /foo path and not strip away /foo:
$ curl -s http://172.71.138.94:8000/foo -X POST
{"message":"no Route matched with those values"}
$ curl -i http://172.71.138.94:8000/foo
HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Date: Sun, 02 Feb 2020 07:54:39 GMT
Server: echoserver
X-Kong-Upstream-Latency: 0
X-Kong-Proxy-Latency: 0
Via: kong/1.4.2
Hostname: echo-85fb7989cc-kk7r6
Pod Information:
node name: vm-xxx-centos
pod name: echo-85fb7989cc-kk7r6
pod namespace: default
pod IP: 172.70.0.21
Server values:
server_version=nginx: 1.12.2 - lua: 10010
Request Information:
client_address=172.70.0.17
method=GET
real path=/foo
query=
request_version=1.1
request_scheme=http
request_uri=http://172.71.138.94:8080/foo
# Example2: Use KongIngress with Service resource
# KongIngress can be used to change load-balancing, health-checking and other proxy behaviours in Kong.
# Next, we are going to tweak two settings:
# 1、Configure Kong to hash the requests based on IP address of the client.
# 2、Configure Kong to proxy all the request on /foo to /bar.
# Let's create a KongIngress resource with these settings
$ echo 'apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
name: demo-customization
upstream:
hash_on: ip
proxy:
path: /bar/' | kubectl apply -f -
kongingress.configuration.konghq.com/demo-customization created
# Now, let's associate this KongIngress resource to the echo service.
$ kubectl patch service echo -p '{"metadata":{"annotations":{"configuration.konghq.com":"demo-customization"}}}'
service/echo patched
# Let's test this now
$ curl -i http://172.71.138.94:8000/foo
HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Date: Sun, 02 Feb 2020 08:11:42 GMT
Server: echoserver
X-Kong-Upstream-Latency: 1
X-Kong-Proxy-Latency: 0
Via: kong/1.4.2
Hostname: echo-85fb7989cc-kk7r6
Pod Information:
node name: vm-xxx-centos
pod name: echo-85fb7989cc-kk7r6
pod namespace: default
pod IP: 172.70.0.21
Server values:
server_version=nginx: 1.12.2 - lua: 10010
Request Information:
client_address=172.70.0.17
method=GET
real path=/bar/foo
query=
request_version=1.1
request_scheme=http
request_uri=http://172.71.138.94:8080/bar/foo
# Real path received by the upstream service (echo) is now changed to /bar/foo
# Also, now all the requests will be sent to the same upstream pod:
$ curl -s 172.71.138.94:8000/foo | grep "pod IP"
pod IP: 172.70.0.21
$ curl -s 172.71.138.94:8000/foo | grep "pod IP"
pod IP: 172.70.0.21
$ curl -s 172.71.138.94:8000/foo | grep "pod IP"
pod IP: 172.70.0.21
Kong-ingress-controller CRDs-KongPlugin
Kong is designed around an extensible plugin architecture and comes with a wide variety of plugins already bundled inside it.
These plugins can be used to modify the request/response or impose restrictions
on the traffic.
Once this resource is created, the resource needs to be associated with an
Ingress
,
Service
, or
KongConsumer
resource in Kubernetes. For more details, please read the reference documentation on
KongPlugin
.
The below diagram shows how the
KongPlugin
resource can be linked to an
Ingress
,
Service
, or
KongConsumer
:
也即：
KongPlugin
用于給Kubernetes服務提供Kong插件功能。這裡舉例進行說明：
# start httpbin service.
$ cat << EOF > httpbin.yaml
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
labels:
app: httpbin
spec:
ports:
- name: http
port: 80
targetPort: 80
selector:
app: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
spec:
replicas: 1
selector:
matchLabels:
app: httpbin
template:
metadata:
labels:
app: httpbin
spec:
containers:
- image: docker.io/kennethreitz/httpbin
name: httpbin
ports:
- containerPort: 80
EOF
$ kubectl apply -f httpbin.yaml
service/httpbin created
deployment.apps/httpbin created
# Setup Ingress rules
$ echo "
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: demo2
spec:
rules:
- http:
paths:
- path: /baz
backend:
serviceName: httpbin
servicePort: 80
" | kubectl apply -f -
ingress.extensions/demo2 created
# Let's test
$ curl -i http://172.71.138.94:8000/baz/status/200
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
Server: gunicorn/19.9.0
Date: Sun, 02 Feb 2020 09:19:11 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
X-Kong-Upstream-Latency: 1
X-Kong-Proxy-Latency: 0
Via: kong/1.4.2
# Configuring plugins on Ingress resource
# First, we will create a KongPlugin resource
$ echo '
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: add-response-header
config:
add:
headers:
- "demo: injected-by-kong"
plugin: response-transformer
' | kubectl apply -f -
kongplugin.configuration.konghq.com/add-response-header created
# Next, we will associate it with our Ingress rules
$ kubectl patch ingress demo2 -p '{"metadata":{"annotations":{"plugins.konghq.com":"add-response-header"}}}'
ingress.extensions/demo2 patched
# Here, we are asking Kong Ingress Controller to execute the response-transformer plugin whenever a request matching the Ingress rule is processed
# Let's test it out
$ curl -i http://172.71.138.94:8000/baz/status/200
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
Server: gunicorn/19.9.0
Date: Sun, 02 Feb 2020 09:22:44 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
demo: injected-by-kong
X-Kong-Upstream-Latency: 1
X-Kong-Proxy-Latency: 0
Via: kong/1.4.2
# As can be seen in the output, the demo header is injected by Kong when the request matches the Ingress rules defined in the demo Ingress resource.
注意區分與原生Kong添加插件在使用上的差别，更多例子詳見using-kongplugin-resource
四、 Kong-ingress-controller High-Availability
Kong Ingress Controller is designed to be reasonably easy to operate and be highly available, meaning, when some expected failures do occur, the Controller should be able to continue to function with minimum possible service disruption.Kong Ingress Controller is composed of two parts: 1. Kong, which handles the rquests, 2. Controller, which configures Kong dynamically.Kong itself can be deployed in a Highly available manner by deploying multiple instances (or pods). Kong nodes are state-less, meaning a Kong pod can be terminated and restarted at any point of time.The controller itself can be stateful or stateless, depending on if a database is being used or not.If a database is not used, then the Controller and Kong are deployed as colocated containers in the same pod and each controller configures the Kong container that it is running with.For cases when a database is necessary, the Controllers can be deployed on multiple zones to provide redudancy. In such a case, a leader election process will elect one instance as a leader, which will manipulate Kong’s configuration.
根據Kong-ingress-controller High-availability and Scaling給出的說明，可以得出高可用方案如下：
Kong DB-less Mode
在DB-less Mode下，各個Pod(Controller + Kong)之間資料獨立(
state-less
)，是以部署Kong-ingress-controller副本數(Replicas) > 1即可實作高可用，如下：
$ kubectl get deploy -nkong
NAME READY UP-TO-DATE AVAILABLE AGE
ingress-kong 1/1 1 1 8h
$ kubectl scale --replicas=2 deploy/ingress-kong -nkong
deployment.apps/ingress-kong scaled
$ kubectl get deploy -nkong
NAME READY UP-TO-DATE AVAILABLE AGE
ingress-kong 2/2 2 2 8h
$ kubectl get pods -nkong
NAME READY STATUS RESTARTS AGE
ingress-kong-7875999c56-cbfzs 2/2 Running 1 8h
ingress-kong-7875999c56-r44bn 2/2 Running 1 34s
方案圖如下：
Kong DB Mode
在DB Mode下，Kong需要共享資料庫。而Kong本身無狀态，可以擴充；Controller實作了Leader選舉機制保證同一時刻隻有一個副本維護Kong配置(調用Kong Admin API)，是以高可用方案如下：
- 1、Kong&Controller副本數(Replicas) > 1
- 2、DB高可用
方案圖如下：
Conclusion
API Gateway是雲原生應用中必不可少的元件，而Kong由于其具有的高性能和高可擴充性成為目前社群中最流行的API網關。本文從Kong API為切入點介紹了Kong的基本使用——Used As HTTP Proxy。而Kong雖然可以通過調用Admin API來完成代理行為，但是對于Kubernetes應用來說還是顯得太過繁瑣與笨重，Kong officials為了解決這個共性問題建立了Kong-ingress-controller項目。Kong-ingress-controller采用标準的Kubernetes Operator設計開發，以Kubernetes-Native方式使Kong代理Kubernetes服務，同時利用四種CRDs對功能進行擴充，以完成Kong原生所涵蓋的所有特性，可以說是API Gateway Kubernetes-Native的最佳實踐。最後本文給出了Kong-ingress-controller的高可用方案，這在企業生産環境中是非常重要且有必要的
Refs
- 選擇Kong作為你的API網關
- High-availability and Scaling
- kong-ingress-controller實踐
- kong proxy
- Kong Ingress Controller Design
歡迎關注我的公衆号