minio筆記02--基于swarm搭建minio叢集

• ​​1 介紹​​
• ​​2 初始化swarm叢集​​
• ​​3 搭建minio叢集​​
• ​​4 說明​​

1 介紹

minio 有多種搭建方式，​​minio筆記01–部署與測試minio​​已經介紹了單機上搭建測試叢集和常見使用方式，本文将進一步介紹如何基于swarm 搭建多機叢集。

2 初始化swarm叢集

1. 初始化 master 節點

   master節點執行init，初始化叢集

# docker swarm init      

1. 添加 worker節點：

   在work節點執行join，加入叢集

在每個work上依次執行該指令：
# docker swarm join --token SWMTKN-1-61sdyxdby3ofx0s3p4x44hmyahfrcvijcxf7v2fgbhf3yd007c-5d5lt1vhkr28aw4koith8ya8j 192.168.2.131:237      

1. 叢集節點初始狀态：



2. 建立測試服務

# docker service create --name nginx --network host nginx:1.19.6 
擴容到9個副本：
# docker service scale nginx=4 -d
預設情況下master節點也是worker，是以也會部署服務；但是可以通過drain是master隻充當管理節點，不部署服務：
# docker node update --availability drain kmaster
kmaster

# docker node ls # 此時狀态為Drain，master節點上的nginx飄到其它節點上了。
ID                            HOSTNAME                          STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
x6krw762e277z3cg9jcty17rh *   kmaster    Ready     Drain          Leader           20.10.1
cv2qe94em477zz5kx92a9qt1m     knode01    Ready     Active                          20.10.1

測試完畢，删除服務，nginx服務
# docker service rm nginx 

筆者此處剛剛好4個節點，是以需要設定master節點可用；線上環境可以專門用3個低配機器當作master，不部署其它服務。
# docker node update --availability active kmaster      

3 搭建minio叢集

1. 建立secret_key 和 access_ke

echo "minio" | docker secret create access_key -
echo "minio123" |      

1. 對節點打标簽，以将minio執行個體排程到不同節點上

docker node update --label-add minio1=true x6krw762e277z3cg9jcty17rh
docker node update --label-add minio2=true cv2qe94em477zz5kx92a9qt1m
docker node update --label-add minio3=true utp02b0firo331snlz3jvgexy
docker node update --label-add minio4=true jl0wocdk31hik8xlgkovaoz44
注意：打标簽必須使用node的ID來辨別節點，否則會失敗      

1. 依次在4個節點 建立/home/minio/data 目錄

# adduser --home /home/minio minio
# su - minio
$ mkdir      

1. 下載下傳并修改配置檔案

# wget https://raw.githubusercontent.com/minio/minio/master/docs/orchestration/docker-swarm/docker-compose-secrets.yaml
# vim docker-compose-secrets.yaml
version: '3.7'

services:
  minio1:
    image: minio/minio:RELEASE.2021-01-16T02-19-44Z
    hostname: minio1
    volumes:
      - /home/minio/data:/export
    ports:
      - "9001:9000"
    networks:
      - minio_distributed
    deploy:
      restart_policy:
        delay: 10s
        max_attempts: 10
        window: 60s
      placement:
        constraints:
          - node.labels.minio1==true
    command: server http://minio{1...4}/export
    secrets:
      - secret_key
      - access_key
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio2:
    image: minio/minio:RELEASE.2021-01-16T02-19-44Z
    hostname: minio2
    volumes:
      - /home/minio/data:/export
    ports:
      - "9002:9000"
    networks:
      - minio_distributed
    deploy:
      restart_policy:
        delay: 10s
        max_attempts: 10
        window: 60s
      placement:
        constraints:
          - node.labels.minio2==true
    command: server http://minio{1...4}/export
    secrets:
      - secret_key
      - access_key
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio3:
    image: minio/minio:RELEASE.2021-01-16T02-19-44Z
    hostname: minio3
    volumes:
      - /home/minio/data:/export
    ports:
      - "9003:9000"
    networks:
      - minio_distributed
    deploy:
      restart_policy:
        delay: 10s
        max_attempts: 10
        window: 60s
      placement:
        constraints:
          - node.labels.minio3==true
    command: server http://minio{1...4}/export
    secrets:
      - secret_key
      - access_key
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio4:
    image: minio/minio:RELEASE.2021-01-16T02-19-44Z
    hostname: minio4
    volumes:
      - /home/minio/data:/export
    ports:
      - "9004:9000"
    networks:
      - minio_distributed
    deploy:
      restart_policy:
        delay: 10s
        max_attempts: 10
        window: 60s
      placement:
        constraints:
          - node.labels.minio4==true
    command: server http://minio{1...4}/export
    secrets:
      - secret_key
      - access_key
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

networks:
  minio_distributed:
    driver: overlay

secrets:
  secret_key:
    external: true
  access_key:
    external: true      

1. 啟動叢集

# docker stack deploy --compose-file=docker-compose-secrets.yaml minio_stack
Creating network minio_stack_minio_distributed
Creating service minio_stack_minio2
Creating service minio_stack_minio3
Creating service minio_stack_minio4
Creating service      

1. 服務建立成功後如下：



2. 在master節點添加叢集資訊

# mc alias set minio http://localhost:9001 minio minio123
Added `minio` successfully
# mc mb test01
Bucket created successfully `test01`.
# mc ls minio 
[2021-01-22 17:00:33 CST]      

1. 删除叢集

在master上删除minio_stack
# docker stack rm minio_stack
在每個work節點執行清除卷的功能
# docker volume prune      

1. minio 叢集資訊：

2. 通路minio叢集：

   ​​​http://ip:9001/minio/​​

4 說明

1. 軟體環境

   筆者系統為ubuntu18.04

   docker 20.10.1

   minio鏡像：minio/minio:RELEASE.2021-01-16T02-19-44Z