openstack與ceph整合安裝指導(dǎo)文檔

openstack與ceph整合安裝指導(dǎo)文檔目錄TOC\o"1-5"\h\z\o"CurrentDocument"概述 3\o"CurrentDocument"版本配套表 3\o"CurrentDocument"系統(tǒng)架構(gòu)圖 3物理結(jié)構(gòu)圖 3邏輯結(jié)構(gòu)圖 4\o"CurrentDocument"openstack安裝 5\o"CurrentDocument"ceph安裝 5\o"CurrentDocument"ip規(guī)劃 5\o"CurrentDocument"安裝步驟 6\o"CurrentDocument"controller點和 compute 肖點安ceph客戶端 7\o"CurrentDocument"controller 節(jié)點酉I置glance使用 ceph 8controller紡點配置cinder 使用 ceph 10\o"CurrentDocument"compute節(jié)點配宜 nova 使用 ceph 121概述木文檔描述openstack在glance、cinder^nova組件后端如何配置使用ceph來進(jìn)行存儲。2版本配套表軟件版本說明openstackicehouseceph集群0.80.5ceph客戶端0.81Libvirt1.1.4qemu1.6.1ceph操作系統(tǒng)Centos6.5DVD大小：4Gopenstack操作系統(tǒng)Centos6.5Mini3系統(tǒng)架構(gòu)圖3.1物理結(jié)構(gòu)圖

3.2邏輯結(jié)構(gòu)圖

3.3openstack安裝使用趙子顧的自動部署，3節(jié)點部署。3.4ceph安裝3.4.1ip規(guī)劃角色I(xiàn)p配置說明controller40見自動部署說明虛擬機(jī),cinder、glance、nova-apinetwork41見自動部署說明虛擬機(jī)，neutroncompute42見自動部署說明虛擬機(jī)，nova-computerCephnodd48/dev/sdb/dev/sdc虛擬機(jī)，deploy,mon,osdCephnode249/dev/sdb/dev/sdc虛擬機(jī)，deploy,mon,osdCephnode350/dev/sdb虛擬機(jī)，deploy,mon,osd/dev/sdc3.4.2安裝步驟修改3臺機(jī)器的主機(jī)名分別為:cephl48.cephl49、cephl50編輯3臺機(jī)器/etc/hosts內(nèi)容如下：cephl48cephl49cephl50將ceph.zip@錄拷貝到/home/ceph目錄下并且解壓，生成ceph和deploy兩個目錄。編輯/etc/yum.repos.d/ceph.repo文件內(nèi)容如卜:[ceph?noarch]name=Cephnoarchpackagesbaseurl=file:///home/ceph/cephenabled=lgpgcheck=O[ceph-deply]name=Cephdeploypackagesbaseurl=file:///home/ceph/deployenabled=lgpgcheck=O三個節(jié)點增加相互信任：cephl48上執(zhí)行：ssh-keygenssh-copy-idcephl48ssh-copy-idcephl49ssh-copy-idcephl50cephl49上執(zhí)行：ssh-keygenssh-copy-idcephl48ssh-copy-idcephl50cephl50上執(zhí)行：ssh-keygenssh-copy-idcephl48ssh-copy-idcephl49三個節(jié)點均關(guān)閉selinux和防火墻：serviceiptablesstopchkconfigiptablesoff將/etc/sysconfig/seIinux中SELINUX=enforcing改為SELINUX=disabled重啟機(jī)器reboot安裝ceph,三臺機(jī)器均執(zhí)行如下命令：yuminstallceph-y

8.9.在cephl48上執(zhí)行如卜命令安裝ceph-deploy：8.9.執(zhí)行如下命令：cd/etc/cephceph-deploynewcephl48cephl49cephl50部署mon節(jié)點，執(zhí)行如下命令：ceph-deploymoncreatecephl48cephl49cephl50ceph-deploygatherkeyscephl48〃收集密鑰部署osd節(jié)點，執(zhí)行如下命令：ceph-deployosdpreparecephl48:/dev/sdbcephl48:/dev/sdccephl49:/dev/sdbcephl49:/dev/sdccephl50:/dev/sdbcephl50:/dev/sdc如果有需要，部署mds,執(zhí)行如下命令：重啟服務(wù)/etc/init.d/ceph重啟服務(wù)/etc/init.d/ceph?arestart查看ceph狀態(tài)是否正常：ceph-s13.24.顯示如下：cluster4fa8cb32-feal-4d68-a341-ebddab2f3e0fhealthHEALTH_WARNclockskewdetectedonmon.cephl50monmap e2: 3 mons at{cephl48=48:6789/0/cephl49=49:6789/0,cephl50=50:6789/0},electionepoch8,quorum0,1/2cephl48,cephl49，cephl50osdmape41:6osds:6up,6inpgmapv76:192pgs,3pools,0bytesdata,0objects215MBused,91878MB/92093MBavail192active+clean15.配置148為ntp的server,貝他節(jié)點泄時向148同步時間3.5controller節(jié)點和compute節(jié)點安ceph客戶端（不需要,在openstack上執(zhí)行ceph-version能看到版本表示ceph已經(jīng)安裝）執(zhí)行如下命令rpm?-import,/git/?p=ceph.git;a=blob_plain;f=keys/release.asc,rpm?-import,/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.ascl增加如下文件：vi/etc/yum?repos?d/ceph-extras內(nèi)容如下：[=CephExtrasPackagesbaseurl=http://ceph?com/packages/ceph-extras/rpm/centos6/Sbasearchenabled=lpriority=2gpgcheck=ltype=rpm-mdgpgkey=https://ceph?com/git/?p=ceph?git;a=blob_plain;f=keys/release?asc[ceph-extras-noarch]name=CephExtrasnoarchbaseurl=http://ceph?com/packages/ceph-extras/rpm/centos6/noarchenabled=lpriority=2gpgcheck=ltype=rpm-mdgpgkey=https://ceph?com/git/?p=ceph?git;a=blob_plain:f=kejrs/release?asc[ceph-extras-source]name=CephExtrasSourcesbaseurl=http://ceph?com/packages/ceph-extras/rpm/centos6/SRPMSenabled=lpriority=2gpgcheck=ltype=rpm-mdgpgkey=https://ceph?com/git/?p=ceph?git;a=blob_plain:f=keys/release?asc添加ceph庫rpm-Uvhhttp：//ceph?com/rpms/el6/noarch/ceph-release-l一0?el6?noarch?rpm添加epel庫rpm~Uvhhttp：//d1?fedoraproject?org/pub/epel/6/x86_64/epel-release-6-8?noarch?rpm安裝cephyumupdate-yyuminstallceph-y3.6controller節(jié)點配置glance使用ceph將cephl48點/etc/ceph0錄下的兩個文件拷貝到controller盯點和compute肖點cd/etc/ceph/scpceph?confceph?client?admin?keyring192?168?1?142:/etc/ceph/scpceph?confceph?client?admin.keyring192?168?1?140:/etc/ceph/修改ceph.client,admin,keyring的權(quán)限chmod+r/etc/ceph/ceph.client.admin.keyring在cephl48上創(chuàng)建glance的存儲池radosmkpoolglance編輯140上glance的配置文件/etc/glance/glance-api.conf中如卜配置項rbd_store_ceph_conf=/etc/ceph/ceph.confdefaultstore=rbd

rbd_store_user=adminrbd_store_pool=glance重啟glance-api進(jìn)程/etc/init?d/openstack-glance-apirestart測試上傳本地鏡像，首先將測試鏡像cirros-0.3.2-x86_64-disk?img放到140的/home/1然后執(zhí)行如下上傳命令：glanceimage-create一一name?，cirros-0?3?2-x86_64一10〃—disk-formatqcow2一-container-formatbare ―is-public True 一一progress </home/cirros-0?3?2-x86_64一disk?img顯示如F：[ >]100%Property1Value |checksum64d7clcd2b6f60c92cl4662941cb7913 |container_formatbarecreated_atI2014-09-16T08:15:46 |deletedFalsedeleted_atINone |disk_formatqcow21id149a71de0-0842-4a7a-b756-edfcb0b86153is_publicITrue |minidisk101min_ram101namecirros-0.3.2-x86_64T0ownerI3636a6e92daf4991beb64643bcl45fab |protectedFalsesizeI13167616 |statusactiveupdated_atI2014-09-16T08:15:51 |virtual^size1None I查看上傳的鏡像glanceimage-list顯示如下：TOC\o"1-5"\h\z + + F +ID Name DiskFormatContainerFormatSize Status? W49a71de0-0842-4a7a-b756-edfcb0b86153cirros-0.3.2-x86_64T0qcow2bare 13167616active

測試網(wǎng)頁上傳鏡像，在網(wǎng)頁上傳一個鏡像，然后查看鏡像文件glanceimage-list顯示如下：IDNameDiskFormatContainerFormatSizeStatusda28a635-2336-4603-a596-30879f4716f4asdadadabare13167616active49a71de0-0842-4a7a-b756-edfcb0b86153cirros-0.3.2-IDNameDiskFormatContainerFormatSizeStatusda28a635-2336-4603-a596-30879f4716f4asdadadabare13167616active49a71de0-0842-4a7a-b756-edfcb0b86153cirros-0.3.2-x86_64T0bare13167616activeqcow2qcow2查看ceph中g(shù)lance查看ceph中g(shù)lance池中的對象:rbdIsglance顯示如下:49a71de0-0842-la7a-b756-edfcb0b86153da28a635-2336-4603-a596-30879f4716f43.7controller節(jié)點配「???3.7controller節(jié)點配「???l￥i[cinder使用ceph在cephl48在cephl48上創(chuàng)建cinder的存儲池radosmkpoolcinder編輯140上cinder的配置文件/etc/cinder/cinder.conf中如卜配置項volume_driver=cinder?volume?drivers?rbd.RBDDriverrbd_pool=cinderrbd_user=adminrbd_ceph_conf=/etc/ceph/ceph?conf重啟/etc/init?d/openstack-cinder-volume進(jìn)程/etc/init?d/openstack-cinder-volumerestart命令行創(chuàng)建一個1G的磁盤cindercreate—display-namedevl1顯示如F：cinder1ist"?- H Property I ValueH H attachmentsI []availability_zone novabootablecreatedatfalse2014-09-16T08:48:50.3679765.display_description

display_name

encryptedidmetadatasize

snapshot_id

sourcevolid5.display_description

display_name

encryptedidmetadatasize

snapshot_id

sourcevolidstatusvolume_type查看創(chuàng)建的磁盤狀態(tài)Nonedevl |Falseld8f3416-fbl5-44a9-837f-7724a9034ble|{} I1 INone |None |creatingNonecinderlist顯示如下：IDVolumeTypeBootableAttachedtoIDVolumeTypeBootableAttachedto+StatusDisplayNameSizeNone■ W? MW1d8f3416-fb15-44a9-837f-7724a9034b1ecreatingfalseNone■ W? MW1d8f3416-fb15-44a9-837f-7724a9034b1ecreatingfalsedevl一+ + 界面創(chuàng)建一個2G磁盤6.查看創(chuàng)建的磁盤狀態(tài)6.查看創(chuàng)建的磁盤狀態(tài)cinderlist顯示如下：IDVolumeTypeBootableAttachedtoIDVolumeTypeBootableAttachedtoStatusDisplayNameSizeld8f3416-fbl5-44a9-837f-7724a9034ble eg訂ableld8f3416-fbl5-44a9-837f-7724a9034ble eg訂ableNone falsee53efe68-5d3b-438d-84cl-fa4c68bd9582availableNoneifalse idevldev210.查看10.查看ceph中cinder池中的對象:rbdIscinder顯示如下:volume-ld8f3416-fbl5-44a9-837f-7724a9034ble

volume-e53efe68-5d3b-438d-84cl-fa4c68bd95823.8compute節(jié)點配置nova使用ceph升級libvirtl.1.0,參考《qemu-1ibvirt更新步驟?doct》在cephl48上創(chuàng)建nova的存儲池radosmkpoolnova生成一個uuiduuidgen編譯qemu-1.6?1,參考在cephl48上創(chuàng)建nova的存儲池radosmkpoolnova生成一個uuiduuidgen3.4.顯示如F：c245elef-d340-4d02-9dcf-fd091cdlfe47執(zhí)行如下命令cat>secret?xml?E0F<secretephemeral二'no'private二'no'><uuid>c245elef-d340-4d02-9dcf-fd091cdlfe47</uuid><usagetype二'ceph'><name>client?cindersecret</name></usage></secret>EOFvirshsecret-define―filesecret?xml顯示如F：Secretc245elef-d340-4d02-9dcf-fd091cdlfe47created執(zhí)行如下命令：cat/etc/ceph/ceph?執(zhí)行如下命令：cat/etc/ceph/ceph?client?admin?keyring顯示如下：[client?admin.key=AQAXrRdU807uHRAAvYit51h4Dgiz6jkAtq8GLA=將“AQAXrRdU807uHRAAvYit51h4Dgiz6jkAtq8GLA二二”放到一個臨時文件echo"AQAXrRdU807uHRAAvYit51h4Dgiz6jkAtq8GLA==，>key執(zhí)行如下命令：7.8.virshsecret-set-value--secretc245elef-d340-4d02-9dcf-fd091cdlfe47--base64$(catkey)9.編輯142上nova的配置文件/etc/nova/nova,conf中如卜配置項images_type=rbdimages_rbd_pool=novaimages_rbd_ceph_conf=/etc/c