saltstack mutilple master 高可用研究

multiple masters

As of Salt 0.16.0, the ability to connect minions to multiple masters has been made available. The multi-master system allows for redundancy of Salt masters and facilitates multiple points of communication out to minions. When using a multi-master setup, all masters are running hot, and any active master can be used to send commands out to the minions.

備援的master的數量是沒有限制的.

Summary of Steps

    1.建立多台新的master伺服器

    2.複制原來的master key master.pem,master.pub 到新的master伺服器

    3.完成key的複制後才可以安全的啟動新master伺服器

    4.配置minions的配置檔案master:添加new master

    5.重新開機minions

    6.接受key認證.

預設的master證書pki存儲在

# Directory used to store public key data:

#pki_dir: /etc/salt/pki/master

root@saltmaster:/etc/salt/pki/master# ls

master.pem  master.pub  minions  minions_autosign  minions_denied  minions_pre  minions_rejected

# Set the location of the salt master server. If the master server cannot be

# resolved, then the minion will fail to start.

#master: salt

master:

  - 192.168.50.10

  - 192.168.50.100 #在此添加多台master

  - 192.168.50.101

# If multiple masters are specified in the 'master' setting, the default behavior

# is to always try to connect to them in the order they are listed. If random_master is

# set to True, the order will be randomized instead. This can be helpful in distributing

# the load of many minions executing salt-call requests, for example, from a cron job.

# If only one master is listed, this setting is ignored and a warning will be logged.

#random_master: False

#如果master配置指定使用多master,預設總是按連接配接清單中的順序連接配接.如果啟用random_master,将使用随機連接配接.

Minions can automatically detect failed masters and attempt to reconnect to reconnect to them quickly.

minions 能夠自動檢測失敗的master,重新連接配接也是很迅速的.

To enable this functionality, set master_alive_interval in the minion config and specify a number of seconds to poll the masters for connection status.

在minion端配置master_alive_interval指定時間輪詢master的連接配接狀态.

If this option is not set, minions will still reconnect to failed masters but the first command sent after a master comes back up may be lost while the minion authenticates.

如果不配置這個選項,minions仍将重連失敗的master但第一次發送到master的指令會丢失

Sharing Files Between Masters

在masters之間共享檔案

Salt does not automatically share files between multiple masters. A number of files should be shared or sharing of these files should be strongly considered.

Minion Keys

salt不會在masters之間自動共享檔案.是以一些檔案必須慎重考慮.

Minion keys can be accepted the normal way using salt-key on both masters. Keys accepted, deleted, or rejected on one master will NOT be automatically managed on redundant masters; this needs to be taken care of by running salt-key on both masters or sharing the /etc/salt/pki/master/{minions,minions_pre,minions_rejected} directories between masters.

使用salt-key接受,删除等操作并不會在masters之間傳遞,是以此類操作必須謹慎.

Note

While sharing the /etc/salt/pki/master directory will work, it is strongly discouraged, since allowing access to the master.pem key outside of Salt creates a SERIOUS security risk.

File_Roots

The file_roots contents should be kept consistent between masters. Otherwise state runs will not always be consistent on minions since instructions managed by one master will not agree with other masters.

The recommended way to sync these is to use a fileserver backend like gitfs or to keep these files on shared storage.

Important

If using gitfs/git_pillar with the cachedir shared between masters using GlusterFS, nfs, or another network filesystem, and the masters are running Salt 2015.5.9 or later, it is strongly recommended not to turn off gitfs_global_lock/git_pillar_global_lock as doing so will cause lock files to be removed if they were created by a different master.

Pillar_Roots

Pillar roots should be given the same considerations as file_roots.

Master Configurations

While reasons may exist to maintain separate master configurations, it is wise to remember that each master maintains independent control over minions. Therefore, access controls should be in sync between masters unless a valid reason otherwise exists to keep them inconsistent.

These access control options include but are not limited to:

    external_auth

    client_acl

    peer

    peer_run

使用共享存儲,修改/etc/init.d/salt-master參數指定從共享讀取master配置檔案實作master配置同步.些方案待驗證.

另外salt自身也可以使用salt進行管理,這是另一個方案.

具體操作檢視

Multi Master Tutorial

https://docs.saltstack.com/en/latest/topics/tutorials/multimaster.html

Multi-Master-PKI Tutorial With Failover

https://docs.saltstack.com/en/latest/topics/tutorials/multimaster_pki.html