minio分布式裸機安裝（圖文并茂）

背景&架構

因為單機的minio無法擴充節點，無法使用版本功能，于是我們邊開始使用minio的分布式版本，minio的分布式版本可以使用docker、kubernetes、裸機三種方式，這裡我們使用裸機安裝，架構如下圖所示

1準備工作

4台ubuntu18的電腦，每台電腦的系統，CPU，記憶體，硬碟空間大小均一緻。給minio用的硬碟需使用XFS格式化。挂載給minio用的硬碟到/mnt/disk目錄。分别按順序配置了4個域名

minio1.sfere.local  minio2.sfere.local minio3.sfere.local minio4.sfere.local

編者注：這裡我與官網略有不同，我每個伺服器隻有一塊硬碟給挂載，官網是每個伺服器給4塊硬碟挂載

1個安裝了nginx的伺服器，域名是minio.sfere.local

編者注：如果你沒有域名，你可以在這5台機器裡的hosts檔案裡把5個位址加上，再在你的測試機器的hosts裡上加上這5個位址

2.安裝minio程式（4台電腦均進行一樣的操作）

1.進入官網的下載下傳連結，下載下傳一個最新的deb檔案

https://dl.min.io/server/minio/release/linux-amd64/

例如我下載下傳的 是

https://dl.min.io/server/minio/release/linux-amd64/minio_20211124231933.0.0_amd64.deb

2.把最新檔案放到4台伺服器上，使用dpkg指令安裝 

3.sudo vi /etc/systemd/system/minio.service 注釋掉ProtectProc=invisible 。這個是kernel 5.8之後才加入的，我們的ubuntu18系統不支援

4.添加minio-user使用者和使用者組。注意：此處與官網略有不同，官網打錯字了把minio-user打成了miniouser

sudo groupadd -r minio-user
sudo useradd -M -r -g minio-user minio-user
sudo chown minio-user:minio-user /mnt/disk      

5.建立環境變量檔案

sudo nano /etc/default/minio

# Set the hosts and volumes MinIO uses at startup
# The command uses MinIO expansion notation {x...y} to denote a
# sequential series.
#
# The following example covers four MinIO hosts
# with 4 drives each at the specified hostname and drive locations.
 
MINIO_VOLUMES="http://minio{1...4}.sfere.local/mnt/disk/minio"
 
# Set all MinIO server options
#
# The following explicitly sets the MinIO Console listen address to
# port 9001 on all network interfaces. The default behavior is dynamic
# port selection.
 
MINIO_OPTS="--console-address :9001"
 
# Set the root username. This user has unrestricted permissions to
# perform S3 and administrative API operations on any resource in the
# deployment.
#
# Defer to your organizations requirements for superadmin user name.
 
MINIO_ROOT_USER=minioadmin
 
# Set the root password
#
# Use a long, random, unique string that meets your organizations
# requirements for passwords.
 
MINIO_ROOT_PASSWORD=sfere!lzw!2021
 
# Set to the URL of the load balancer for the MinIO deployment
# This value *must* match across all MinIO servers. If you do
# not have a load balancer, set this value to to any *one* of the
# MinIO hosts in the deployment as a temporary measure.
# nginx伺服器位址
MINIO_SERVER_URL="http://minio.sfere.local"
 
MINIO_IDENTITY_LDAP_TLS_SKIP_VERIFY=on
MINIO_IDENTITY_LDAP_SERVER_INSECURE=on
MINIO_IDENTITY_LDAP_STS_EXPIRY=24h
MINIO_IDENTITY_LDAP_SERVER_ADDR=${LDAP域名}
MINIO_IDENTITY_LDAP_LOOKUP_BIND_DN=${LDAP隻讀賬戶}
MINIO_IDENTITY_LDAP_LOOKUP_BIND_PASSWORD=${LDAP隻讀賬戶的密碼}
MINIO_IDENTITY_LDAP_USER_DN_SEARCH_BASE_DN=${LDAP使用者搜尋域}
MINIO_IDENTITY_LDAP_USER_DN_SEARCH_FILTER=(&(objectClass=inetOrgPerson)(uid=%s))
MINIO_IDENTITY_LDAP_GROUP_SEARCH_BASE_DN=${LDAP組搜尋域}
MINIO_IDENTITY_LDAP_GROUP_SEARCH_FILTER=(&(objectclass=groupOfUniqueNames))      

6. 運作minio服務，檢查運作是否成功

sudo systemctl start minio.service
sudo systemctl status minio.service
journalctl -f -u minio.service      

nginx配置

在/etc/nginx/conf.d目錄下添加一個minio.conf

upstream minio {
    server minio1.sfere.local:9000;
    server minio2.sfere.local:9000;
    server minio3.sfere.local:9000;
    server minio4.sfere.local:9000;
}
 
upstream console {
    ip_hash;
    server minio1.sfere.local:9001;
    server minio2.sfere.local:9001;
    server minio3.sfere.local:9001;
    server minio4.sfere.local:9001;
}
 
server {
        listen       80;
        listen  [::]:80;
        server_name  minio.sfere.local;
 
        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;
 
        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
 
            proxy_connect_timeout 300;
            # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            chunked_transfer_encoding off;
 
            proxy_pass http://minio;
        }
}
server {
        listen       9001;
        listen  [::]:9001;
        server_name  minio.sfere.local;
 
        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;
 
        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-NginX-Proxy true;
 
            # This is necessary to pass the correct IP to be hashed
            real_ip_header X-Real-IP;
 
            proxy_connect_timeout 300;
 
            # To support websocket
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
 
            chunked_transfer_encoding off;
 
            proxy_pass http://console;
        }
}      

使用mc用戶端添加ldap超管，普通使用者

docker run --rm -it --entrypoint=/bin/sh minio/mc
 
mc config host add minio http://minio.sfere.local minioadmin 'sfere!lzw!2021' --api S3v4
  
mc admin policy list minio
  
mc admin policy set minio consoleAdmin user=cn=李鎮偉,ou=test-department,ou=NJ-Dev,ou=SFERE-RD,dc=sfere-elec,dc=com
mc admin policy set minio readwrite group=cn=jira-software-users,dc=sfere-elec,dc=com
mc admin policy set minio consoleAdmin group=cn=超級使用者,dc=sfere-elec,dc=com      

通路頁面

通路

http://minio.sfere.local/

會自動跳轉到

http://minio.sfere.local:9001/login

參考文章

https://docs.min.io/minio/baremetal/installation/deploy-minio-distributed.html