kubernetes實戰-基于redis和docker的留言簿案例

一、介紹

本案例基于Kubernetes和Docker，其中包括

1、web前端

2、redis master

3、redis slave

其中web前端通過javascript redis api和redis master互動

kubernetes體系架構

二、配置

0、先決條件

Kubernetes 叢集

1、啟動redis master

使用replication controller確定隻有一個pod在運作（當某個節點down了，rc會在另一個健康的node啟動redis master），但可能會有資料丢失。

[root@centos1 example]# kubectl create -f redis-master-controller.json replicationcontrollers/redis-master[root@centos1 example]# kubectl get rcCONTROLLER     CONTAINER(S)   IMAGE(S)   SELECTOR            REPLICASredis-master   master         redis      name=redis-master   1      

驗證master運作成功，如下展示了pod運作在centos2/192.168.1.112這台機器上    

[root@centos1 example]# kubectl get podsPOD                  IP           CONTAINER(S)   IMAGE(S)   HOST                    LABELS              STATUS    CREATED      MESSAGEredis-master-svar7   172.17.0.9                             centos2/192.168.1.112   name=redis-master   Running   55 seconds                                     master         redis                                                  Running   55 seconds      

SSH到centos2/192.168.1.112檢視docker狀态

[root@centos2 yum.repos.d]# docker psCONTAINER ID        IMAGE                                  COMMAND                CREATED             STATUS              PORTS               NAMES91689ce56668        redis:latest                           "/entrypoint.sh redi   3 minutes ago       Up 3 minutes                            k8s_master.52732b08_redis-master-svar7_default_5b6d5485-1894-11e5-b3ad-000c293c8c19_97e79b7b   38c3180813c3        gcr.io/google_containers/pause:0.8.0   "/pause"               3 minutes ago       Up 3 minutes                            k8s_POD.49eee8c2_redis-master-svar7_default_5b6d5485-1894-11e5-b3ad-000c293c8c19_298e038f      

注意：kubectl create執行後，如果鏡像不存在，會執行docker pull，根據網絡情況，下載下傳中的pods 在kubertnetes UI上會顯示pending狀态

2、啟動master service

一個kubernetes

service會對一個或多個container進行負載均衡，這是通過我們上面redis-master中定義的labels中繼資料實作的，值得注意的是，在redis中隻有一個master，但是我們依然為它建立一個service，這是因為這樣我們就能使用一個elastic

IP來路由到具體某一個master。

kubernetes叢集中的service是通過container中的環境變量實作服務發現的，service基于pod label實作container的負載均衡。

在第一步中建立的pod包含了一個label“name=redis-master”，service的selector字段決定了service将流量轉發給哪個pod，port和targetPort資訊定義了service

proxy運作在什麼端口。

[root@centos1 example]# kubectl create -f redis-master-service.json services/redis-master[root@centos1 example]# kubectl get servicesNAME            LABELS                                    SELECTOR            IP(S)           PORT(S)redis-master    name=redis-master                         name=redis-master   10.254.154.90   6379/TCP      

上面的運作成功後，所有pods都能發現redis master運作在6379端口，從salve到master流量走向會有以下兩步：

1、一個redis slave會連接配接到redis master service的port上

2、流量會從service節點上的port到targetPort

如果targetPort未指定，預設和port一緻

3、啟動replicated slave pod

雖然redis master是一個單獨的pod，redis slaves是一個replicated

pod，在Kubernetes中，一個Replication Controller負責管理一個replicated

pod的多個執行個體，RC會自動拉起down掉的replica（可以通過殺死docker 程序方式簡單測試）

[root@centos1 example]# kubectl create -f redis-slave-controller.json replicationcontrollers/redis-slave[root@centos1 example]# kubectl get rcCONTROLLER     CONTAINER(S)   IMAGE(S)                    SELECTOR            REPLICASredis-master   master         redis                       name=redis-master   1redis-slave    slave          kubernetes/redis-slave:v2   name=redis-slave    2[root@centos1 example]# kubectl get podsPOD                  IP            CONTAINER(S)   IMAGE(S)                    HOST                    LABELS              STATUS    CREATED      MESSAGEredis-master-svar7   172.17.0.9                                               centos2/192.168.1.112   name=redis-master   Running   41 minutes                                      master         redis                                                                   Running   41 minutes   redis-slave-31tkb    172.17.0.10                                              centos2/192.168.1.112   name=redis-slave    Running   29 seconds                                      slave          kubernetes/redis-slave:v2                                               Running   28 seconds   redis-slave-uk8nu    172.17.0.11                                              centos2/192.168.1.112   name=redis-slave    Running   29 seconds                                      slave          kubernetes/redis-slave:v2                                               Running   28 seconds      

可以看到一個master pod和兩個slave pod

4、啟動slave service

和master一樣，我們希望有一個代理服務連接配接到redis slave，除了服務發現之外，slave service還為web app client提供了透明代理。

這次service 的selector是name=redis-slave，我們可以友善的使用kubectl

get services -l "label=value"指令來定位這些服務

[root@centos1 example]# kubectl create -f redis-slave-service.json services/redis-slave[root@centos1 example]# kubectl get servicesNAME            LABELS                                    SELECTOR            IP(S)            PORT(S)redis-master    name=redis-master                         name=redis-master   10.254.154.90    6379/TCPredis-slave     name=redis-slave                          name=redis-slave    10.254.159.145   6379/TCP      

5、建立frontend pod

這是一個簡單的PHP 服務，用來和master service（寫請求）或slave service（讀請求）互動

[root@centos1 example]# kubectl create -f frontend-controller.json replicationcontrollers/frontend[root@centos1 example]# kubectl get rcCONTROLLER     CONTAINER(S)   IMAGE(S)                                    SELECTOR            REPLICASfrontend       php-redis      kubernetes/example-guestbook-php-redis:v2   name=frontend       3redis-master   master         redis                                       name=redis-master   1redis-slave    slave          kubernetes/redis-slave:v2                   name=redis-slave    2      

運作成功後，檢視目前pod運作狀态

[root@centos1 example]# kubectl get podsPOD                  IP            CONTAINER(S)   IMAGE(S)                                    HOST                    LABELS              STATUS    CREATED      MESSAGEfrontend-fr5z1       172.17.0.13                                                              centos2/192.168.1.112   name=frontend       Running   2 minutes                                       php-redis      kubernetes/example-guestbook-php-redis:v2                                               Running   2 minutes    frontend-gjx3t       172.17.0.14                                                              centos2/192.168.1.112   name=frontend       Running   2 minutes                                       php-redis      kubernetes/example-guestbook-php-redis:v2                                               Running   2 minutes    frontend-v608r       172.17.0.12                                                              centos2/192.168.1.112   name=frontend       Running   2 minutes                                       php-redis      kubernetes/example-guestbook-php-redis:v2                                               Running   2 minutes    redis-master-svar7   172.17.0.9                                                               centos2/192.168.1.112   name=redis-master   Running   53 minutes                                      master         redis                                                                                   Running   53 minutes   redis-slave-31tkb    172.17.0.10                                                              centos2/192.168.1.112   name=redis-slave    Running   12 minutes                                      slave          kubernetes/redis-slave:v2                                                               Running   12 minutes   redis-slave-uk8nu    172.17.0.11                                                              centos2/192.168.1.112   name=redis-slave    Running   12 minutes                                      slave          kubernetes/redis-slave:v2                                                               Running   12 minutes      

可以看到一個redis master，兩個redis slave和三個frontend pods

6、建立guestbook service

和其他service一樣，你可以建立一個service管理frontend pods

[root@centos1 example]# kubectl create -f frontend-service.json services/frontend[root@centos1 example]# kubectl get servicesNAME            LABELS                                    SELECTOR            IP(S)            PORT(S)frontend        name=frontend                             name=frontend       10.254.154.111   80/TCPredis-master    name=redis-master                         name=redis-master   10.254.154.90    6379/TCPredis-slave     name=redis-slave                          name=redis-slave    10.254.159.145   6379/TCP      

我們可以通過frontend service（10.254.154.111）通路pods，但是這個IP明顯是無法在外部通路的，下一節講解如何在外部網絡通路guestbook

7、外部網絡通路guestbook

kubernetes 支援兩種通路暴露一個服務到外部IP位址

NodePort

s

和

LoadBalancer

s，https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/services.md#external-services

另外可以檢視防火牆，找到service對應的端口，如下

[root@centos1 example]# kubectl get pods,servicesPOD                  IP            CONTAINER(S)   IMAGE(S)                                    HOST                    LABELS              STATUS    CREATED         MESSAGEfrontend-fr5z1       172.17.0.13                                                              centos2/192.168.1.112   name=frontend       Running   22 minutes                                         php-redis      kubernetes/example-guestbook-php-redis:v2                                               Running   22 minutes      frontend-gjx3t       172.17.0.14                                                              centos2/192.168.1.112   name=frontend       Running   22 minutes                                         php-redis      kubernetes/example-guestbook-php-redis:v2                                               Running   22 minutes      frontend-v608r       172.17.0.12                                                              centos2/192.168.1.112   name=frontend       Running   22 minutes                                         php-redis      kubernetes/example-guestbook-php-redis:v2                                               Running   22 minutes      redis-master-svar7   172.17.0.9                                                               centos2/192.168.1.112   name=redis-master   Running   About an hour                                      master         redis                                                                                   Running   About an hour   redis-slave-31tkb    172.17.0.10                                                              centos2/192.168.1.112   name=redis-slave    Running   32 minutes                                         slave          kubernetes/redis-slave:v2                                                               Running   32 minutes      redis-slave-uk8nu    172.17.0.11                                                              centos2/192.168.1.112   name=redis-slave    Running   32 minutes                                         slave          kubernetes/redis-slave:v2                                                               Running   32 minutes      NAME            LABELS                                    SELECTOR            IP(S)            PORT(S)frontend        name=frontend                             name=frontend       10.254.154.111   80/TCPkubernetes      component=apiserver,provider=kubernetes   <none>              10.254.0.2       443/TCPkubernetes-ro   component=apiserver,provider=kubernetes   <none>              10.254.0.1       80/TCPredis-master    name=redis-master                         name=redis-master   10.254.154.90    6379/TCPredis-slave     name=redis-slave                          name=redis-slave    10.254.159.145   6379/TCP[root@centos1 example]# kubectl get servicesNAME            LABELS                                    SELECTOR            IP(S)            PORT(S)frontend        name=frontend                             name=frontend       10.254.154.111   80/TCPkubernetes      component=apiserver,provider=kubernetes   <none>              10.254.0.2       443/TCPkubernetes-ro   component=apiserver,provider=kubernetes   <none>              10.254.0.1       80/TCPredis-master    name=redis-master                         name=redis-master   10.254.154.90    6379/TCPredis-slave     name=redis-slave                          name=redis-slave    10.254.159.145   6379/TCP      

發現redis-master是在10.254.154.90上，登入到centos2上，執行iptables-save，發現其中有這樣一條規則

-A KUBE-PORTALS-HOST -d 10.254.154.90/32 -p tcp -m comment --comment "default/redis-master:" -m tcp --dport 6379 -j DNAT --to-destination 192.168.1.112:49038      

說明，centos2本機的49038端口映射到master container内的6379了，當然我們就能執行在本機通路redis了

[root@centos2 yum.repos.d]# redis-cli -p 49038127.0.0.1:49038> set a bOK127.0.0.1:49038> get a"b"      

8、使用curl簡單測試

送出資料

curl "localhost:8000/index.php?cmd=set&key=messages&value=jay_sais_hi"      

查詢資料

curl "localhost:8000/index.php?cmd=get&key=messages"      

附本案例用到的6個.json檔案

1、redis-master-controller.json

{   "kind":"ReplicationController",   "apiVersion":"v1beta3",   "metadata":{      "name":"redis-master",      "labels":{         "name":"redis-master"      }   },   "spec":{      "replicas":1,      "selector":{         "name":"redis-master"      },      "template":{         "metadata":{            "labels":{               "name":"redis-master"            }         },         "spec":{            "containers":[               {                  "name":"master",                  "p_w_picpath":"redis",                  "ports":[                     {                        "containerPort":6379                     }                  ]               }            ]         }      }   }}      

2、redis-master-service.json

{   "kind":"Service",   "apiVersion":"v1beta3",   "metadata":{      "name":"redis-master",      "labels":{         "name":"redis-master"      }   },   "spec":{      "ports": [        {          "port":6379,          "targetPort":6379        }      ],      "selector":{         "name":"redis-master"      }   }}      

3、redis-slave-controller.json

{   "kind":"ReplicationController",   "apiVersion":"v1beta3",   "metadata":{      "name":"redis-slave",      "labels":{         "name":"redis-slave"      }   },   "spec":{      "replicas":2,      "selector":{         "name":"redis-slave"      },      "template":{         "metadata":{            "labels":{               "name":"redis-slave"            }         },         "spec":{            "containers":[               {                  "name":"slave",                  "p_w_picpath":"kubernetes/redis-slave:v2",                  "ports":[                     {                        "containerPort":6379                     }                  ]               }            ]         }      }   }}      

4、redis-slave-service.json

{   "kind":"Service",   "apiVersion":"v1beta3",   "metadata":{      "name":"redis-slave",      "labels":{         "name":"redis-slave"      }   },   "spec":{      "ports": [        {          "port":6379        }      ],      "selector":{         "name":"redis-slave"      }   }}      

{   "kind":"ReplicationController",   "apiVersion":"v1beta3",   "metadata":{      "name":"frontend",      "labels":{         "name":"frontend"      }   },   "spec":{      "replicas":3,      "selector":{         "name":"frontend"      },      "template":{         "metadata":{            "labels":{               "name":"frontend"            }         },         "spec":{            "containers":[               {                  "name":"php-redis",                  "p_w_picpath":"kubernetes/example-guestbook-php-redis:v2",                  "ports":[                     {                        "containerPort":80                     }                  ]               }            ]         }      }   }}      

{   "kind":"Service",   "apiVersion":"v1beta3",   "metadata":{      "name":"frontend",      "labels":{         "name":"frontend"      }   },   "spec":{      "ports": [        {          "port":80        }      ],      "selector":{         "name":"frontend"      }   }}