OpenStack+Ceph+熱遷移+++四、建立存儲池

四、建立存儲池

1.建立Cinder，Glance，Nova的存儲池（node1節點）

ceph osd pool create volumes 128

ceph osd pool create images 128

ceph osd pool create vms 128

[[email protected] ~]# ceph osd pool create volumes 128
pool 'volumes' created
[[email protected] ~]# ceph osd pool create images 128
pool 'images' created
[[email protected] ~]# ceph osd pool create vms 128
pool 'vms' created
[[email protected] ~]# 
           

2.将ceph.conf的檔案傳給controller，compute1，compute2，cinder節點

也就是運作着glance-api 、 cinder-volume 、 nova-compute 和 cinder的節點

ssh controller tee /etc/ceph/ceph.conf </etc/ceph/ceph.conf

ssh cinder tee /etc/ceph/ceph.conf </etc/ceph/ceph.conf

ssh compute1 tee /etc/ceph/ceph.conf </etc/ceph/ceph.conf

ssh compute2 tee /etc/ceph/ceph.conf </etc/ceph/ceph.conf

[[email protected] ~]# ssh controller tee /etc/ceph/ceph.conf </etc/ceph/ceph.conf
[email protected]'s password: 
[global]
fsid = 46ac86e8-1efe-403c-b735-587f9d76a905
mon_initial_members = node1
mon_host = 10.0.0.41
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx

[[email protected] ~]# ssh cinder tee /etc/ceph/ceph.conf </etc/ceph/ceph.conf
[email protected]'s password: 
[global]
fsid = 46ac86e8-1efe-403c-b735-587f9d76a905
mon_initial_members = node1
mon_host = 10.0.0.41
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx

[[email protected] ~]# ssh compute1 tee /etc/ceph/ceph.conf </etc/ceph/ceph.conf
[email protected]'s password: 
[global]
fsid = 46ac86e8-1efe-403c-b735-587f9d76a905
mon_initial_members = node1
mon_host = 10.0.0.41
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx

[[email protected] ~]# ssh compute2 tee /etc/ceph/ceph.conf </etc/ceph/ceph.conf
[email protected]'s password: 
[global]
fsid = 46ac86e8-1efe-403c-b735-587f9d76a905
mon_initial_members = node1
mon_host = 10.0.0.41
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
           

3.安裝ceph用戶端

在controller（glance-api）節點上安裝librbd的Python 綁定

yum install python-rbd

yum install python-rbd
           

在compute1（nova-compute）、compute2（nova-compute）、cinder（cinder-volume）節點上安裝 Python 綁定和用戶端指令行工具

yum install ceph

yum install ceph
           

4.配置ceph用戶端認證

啟用cephx認證，為Nova/Cinder和Glance建立新使用者

ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images'

ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images'

[[email protected] ~]# ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images'
[client.cinder]
	key = AQCVXx1aUZErNRAAeVUTAhLU4r4xaiqDbHqVhw==
[[email protected] ~]# ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images'
[client.glance]
	key = AQCzXx1a7g5BKxAAHECANQ7E6rxTR9S2xzN5bQ==
           

5.把 client.cinder（cinder） 、 client.glance(controller) 的密鑰環複制到适當的節點，并更改所有權：

client.glance（controller）

ceph auth get-or-create client.glance | ssh controller tee /etc/ceph/ceph.client.glance.keyring

ssh controller chown glance:glance /etc/ceph/ceph.client.glance.keyring

[[email protected] ~]# ceph auth get-or-create client.glance | ssh controller tee /etc/ceph/ceph.client.glance.keyring
[email protected]'s password: 
[client.glance]
	key = AQCzXx1a7g5BKxAAHECANQ7E6rxTR9S2xzN5bQ==
[[email protected] ~]# ssh controller chown glance:glance /etc/ceph/ceph.client.glance.keyring
[email protected]'s password: 
           

clientcinder(cinder)

ceph auth get-or-create client.cinder | ssh cinder tee /etc/ceph/ceph.client.cinder.keyring

ssh cinder chown cinder:cinder /etc/ceph/ceph.client.cinder.keyring

[[email protected] ~]# ceph auth get-or-create client.cinder | ssh cinder tee /etc/ceph/ceph.client.cinder.keyring
[email protected]'s password: 
[client.cinder]
	key = AQCVXx1aUZErNRAAeVUTAhLU4r4xaiqDbHqVhw==
[[email protected] ~]# ssh cinder chown cinder:cinder /etc/ceph/ceph.client.cinder.keyring
[email protected]'s password: 
           

6.配置compute1與compute2的密匙環檔案

運作nova-compute 的節點，其程序需要密鑰環檔案：

compute1

ceph auth get-or-create client.cinder | ssh compute1 tee /etc/ceph/ceph.client.cinder.keyring

ceph auth get-key client.cinder | ssh compute1 tee client.cinder.key

[[email protected] ceph]# ceph auth get-or-create client.cinder | ssh compute1 tee /etc/ceph/ceph.client.cinder.keyring
[email protected]'s password: 
[client.cinder]
	key = AQCVXx1aUZErNRAAeVUTAhLU4r4xaiqDbHqVhw==
[[email protected] ceph]# ceph auth get-key client.cinder | ssh compute1 tee client.cinder.key
root@compute1's password: 
AQCVXx1aUZErNRAAeVUTAhLU4r4xaiqDbHqVhw==
           

compute2

ceph auth get-or-create client.cinder | ssh compute2 tee /etc/ceph/ceph.client.cinder.keyring

ceph auth get-key client.cinder | ssh compute2 tee client.cinder.key

[[email protected] ceph]# ceph auth get-or-create client.cinder | ssh compute2 tee /etc/ceph/ceph.client.cinder.keyring
[email protected]'s password: 
[client.cinder]
	key = AQCVXx1aUZErNRAAeVUTAhLU4r4xaiqDbHqVhw==
[[email protected] ceph]# ceph auth get-key client.cinder | ssh compute2 tee client.cinder.key
root@compute2's password: 
AQCVXx1aUZErNRAAeVUTAhLU4r4xaiqDbHqVhw==
           

任選計算節點1或者2生成一個uuid

[[email protected] ~]# uuidgen
9d9cbdda-532a-4441-b11d-eb540dec9df9
           

所有計算節點執行執行下面指令

下面那一長串直接複制進去就能執行改為自己的uuid

cat > secret.xml <<EOF
<secret ephemeral='no' private='no'>
  <uuid>9d9cbdda-532a-4441-b11d-eb540dec9df9</uuid>
  <usage type='ceph'>
        <name>client.cinder secret</name>
  </usage>
</secret>
EOF
           

在計算節點上把密鑰加進libvirt、然後删除臨時副本

virsh secret-define --file secret.xml

[[email protected] ~]# virsh secret-define --file secret.xml
Secret 9d9cbdda-532a-4441-b11d-eb540dec9df9 created
           

virsh secret-set-value --secret 9d9cbdda-532a-4441-b11d-eb540dec9df9 --base64 $(cat client.cinder.key) && rm client.cinder.key secret.xml

Secret value set

[[email protected] ~]# virsh secret-set-value --secret 9d9cbdda-532a-4441-b11d-eb540dec9df9 --base64 $(cat client.cinder.key) && rm client.cinder.key secret.xml
Secret value set

rm: remove regular file ‘client.cinder.key’? y
rm: remove regular file ‘secret.xml’? y