SUSE12SP4-HA-部署文檔-v1.0文檔

SUSE12SP4HAGuide日期(年月日)版本描述作者審閱記錄分發(fā)記錄2018-12-251.0創(chuàng)建目錄目錄 3環(huán)境 4軟件需求 4系統(tǒng)倉庫需求 4網絡架構 4安裝部署HA 4安裝HA軟件 4Hosts文件 6設置SSH互信 6NTP設置 7配置集群 7設置IP地址資源 11配置SBD屏蔽磁盤 13開啟集群 15集群配置文件 17快速部署HA 18快速部署概述 18前提條件 19HA01主機部署 19HA02主機部署 21集群基礎配置 22設置pacemker開機不啟動 23使用Csync2同步更改 23使用Hawk2配置和管理群集資源 24集群腳本模板 27集群配置文件路徑 28集群運維命令 28在線升級Livepatching 30環(huán)境軟件需求SUSELinuxEnterpriseServer12SP4x86_64SUSELinuxEnterpriseHighAvailabilityExtension12SP4系統(tǒng)倉庫需求##zypperlr#|Alias|Name|Enabled|GPGCheck|Refresh+ + + + + 1|SLEHA-12SP4-Pool|SLEHA-12SP4-Pool|Yes|(r)Yes|No2|SLEHA-12SP4-Updates|SLEHA-12SP4-Updates|Yes|(r)Yes|No3|SLES12SP4_Pool|SLES12SP4_Pool|Yes|(r)Yes|No4|SLES12SP4_Updates|SLES12SP4_Updates|Yes|(r)Yes|No網絡架構主機名心跳地址業(yè)務地址虛擬地址ha01446ha0255安裝部署HAHA#zypper#zypperpt##zypper-nin--auto-agree-with-licenses-tpatternha_sles圖形化方式:Hosts#vim/etc/hosts4 #vim/etc/hosts4 ha01 心跳網絡5ha01 業(yè)務網絡ha02SSH##ssh-keygen#cd.ssh/#ssh-copy-id-iid_rsa.pubha02NTP#vim/etc/ntp.confserver#vim/etc/ntp.confserver#localclock(LCL)fudgestratum10 #LCLisunsynchronized#systemctlstartntpd.service#systemctlenablentpd.service##ntpq-premoterefidsttwhenpollreach delay offsetjitter==============================================================================*LOCAL(0).LOCL.10l 13 64377 0.000 0.000 0.000#vim#vim/etc/ntp.confserverha01iburst#systemctlstartntpd.service#systemctlenablentpd.service配置集群##yast2clusterpublic）＿感＿.

YaST2-cluster@haOl _. xSecurityConUeconntrackdService

Cluster-CommunicationChannelsChannelReChannelRedundantChannelMulticastAddressMulticastAddress·Port:5407192.168,1.455RedundantIP NodeID2尸P-ClusteName巨0AutoGenerateNodeID

巨pectedVoteslr;-

—_ll—吐[三_lc'Pmode巨 lI_I 二三YaST2-cluster@nodeOOl 一 口 XCluster-Security0 n_<bleeurityAu『,Tojoinanexistingclustepleasecopy/etc/corosync/authkeyfomothernodesmanually.|§enemteAuthkeyFile|l旦elp | IAbo_r:tII旦ackII抖ext |感SUSE.YaST2-cluster@nodeOOl 一 口 XCluster-ConfigureCsync2SyncHost SyncFileI "'-dd II Del ll EdIt l I Agd II Del II |尸§u99-I GeneratePre-Shared-Keys ][ Tμ「ncsync2ON |l 旦elp 1

IAbo_r:tIIgackII抖ext |YaST2-cluster@nodeOOl 一 口 XCluster-ConfigureconntrackdnrckdisadaemonhihhelpsodupefiralsbwnnodYaSTcanhelptoconfiguresomebasicaspectsofconntrackduneedotitwitheocf:heaannckd,DedicatedInteface:[ethO TlIP:3I IroupNumbe--Generate/etc/conn!ackd/conntrack--l Help | IAbo_r:tllBackII抖ext I#scp/etc/corosync/corosync.conf#scp/etc/corosync/corosync.confha02:/etc/corosync/corosync.conf#rcpacemaker#rcpacemakerstart#crm_mon-1#crm_mon-1Stack:corosyncCurrentDC:ha01(version1.1.16-4.8-77ea74d)-partitionwithquorumLastupdated:TueNov723:13:062017Lastchange:TueNov723:13:052017byhaclusterviacrmdonha012nodesconfigured0resourcesconfiguredOnlineOnlineha01ha02 online狀態(tài)Noactiveresources##corosync-cfgtool-sPrintingringstatus.LocalnodeID2RINGID0id=5status=ring0activewithnofaultsRINGID1id=5status=ring1activewithnofaults#crmcrm(live)##crmcrm(live)#configurecrm(live)configure#propertystonith-enabled=falsecrm(live)configure#commitIP##crmconfigurecrm(live)configure#primitiveres_vip_testocf:heartbeat:IPaddr2paramsip=6cidr_netmask=24crm(live)configure#verifycrm(live)configure#commitcrm(live)configure#exit查看集群狀態(tài)##crm_mon-1Stack:corosyncCurrentDC:ha01(version1.1.16-4.8-77ea74d)-partitionwithquorumLastupdated:WedNov800:03:352017Lastchange:WedNov800:02:492017byrootviacibadminonha012nodesconfigured1resourceconfiguredOnline:[ha01ha02]Activeresources:res_vip_test (ocf::heartbeat:IPaddr2):Startedha01#ipaddr#ipaddrshoweth0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscpfifo_faststateUPgroupdefaultqlen1000link/ether00:0c:29:66:3f:ddbrdff:ff:ff:ff:ff:ffinet3/24brd55scopeglobaleth0valid_lftforeverpreferred_lftforeverinet6/24brd55scopeglobalsecondaryeth0valid_lftforeverpreferred_lftforever#ping5PING5(5)56(84)bytesofdata.64bytesfrom5:icmp_seq=1ttl=64time=0.021ms64bytesfrom5:icmp_seq=2ttl=64time=0.050msIP##crm_mon-1Online:[ha01ha02]Activeresources:res_vip_test (ocf::heartbeat:IPaddr2):Startedha01ha01:~ha01:~#crmresourcemigratesap_ipha02INFO:Moveconstraintcreatedforsap_iptoha02#crm_mon-1Online:[ha01ha02]Activeresources:res_vip_test (ocf::heartbeat:IPaddr2):Startedha02配置SBD#rcpacemakerstop1、關閉集群(#rcpacemakerstop23（設備名按實際情況設置）ha01:~#lsblkNAME MAJ:MINRMSIZEROTYPEMOUNTPOINTsdb8:1601G0disksdc8:3201G0disksdd8:4801G0disk3、設置檢查包##echosoftdog>/etc/modules-load.d/watchdog.conf#systemctlrestartsystemd-modules-load##systemctlstatussystemd-modules-load##lsmod|grepdogsoftdog163840sbd-d/dev/sdbsbd-d/dev/sdb–d/dev/sdc–d/dev/sdd-460-130create#sbd-d/dev/sdbdump==Dumpingheaderondisk/dev/sdbHeaderversion:2.1UUID:94d700ee-837b-46c7-95cc-27f3d1ffcf9fNumberofslots:255Sectorsize:512Timeout(watchdog):30Timeout(allocate):2Timeout(loop):1Timeout(msgwait):60==Headerondisk/dev/sdbisdumped5、兩個timeout解釋-4選項用于指定msgwait超時。在以上示例中，超時設置為180秒。-1選項用于指定watchdog超時。在以上示例中，超時設置為60秒。模擬檢查包的最小允許值為15秒。如果SBD設備駐留在多路徑組上，則需要調整SBD所用的超時，因為MPIO的沿路徑檢測可能導致一些等待時間。msgwait超時后，將假定此消息已傳遞到節(jié)點。對于多路徑，這應是MPIO檢測路徑故障并切換到下一個路徑所需的時間?？赡苄枰谀沫h(huán)境SBD守護程序未足夠快速地更新檢查包計時器，則節(jié)點會自行終止。watchdog超時必須短于msgwait超時-前者是后者的一半是較好的估計值。以下“公式”大致表達了這三個值之間的關系：Timeout(msgwait)=(Timeout(watchdog)*2)stonith-timeout=Timeout(msgwait)+20%更多信息請mansbd獲取#vim/etc/init.d/boot.localmodprobe#vim/etc/init.d/boot.localmodprobesoftdog#vim/etc/sysconfig/sbdSBD_DELAY_START=yes說明：啟用或禁用延遲。設置SBD_DELAY_START為yes（如果msgwait相對較長，而群集節(jié)點引導很快。將此參數(shù)設置為yes可在引導時延遲SBD要此項延遲。(虛擬機中時常會遇見這種狀況)#systemctlenable#systemctlenablesbd開啟集群#rcpacemakerstart啟動集群(#rcpacemakerstart##crmconfigurecrm(live)configure#propertystonith-enabled="true"crm(live)configure#propertystonith-timeout="80s"crm(live)configure#propertyno-quorum-policy="ignore"crm(live)configure#rsc_defaultsresource-stickiness=1000crm(live)configure#commitcrm(live)configure#exit注意：stonith-timeout設置為220秒比較合適，是根據(jù)前面公式計算而出。#crm#crmconfigureprimitiveres_stonith_sbdstonith:external/sbd\paramspcmk_delay_max=30\opstartinterval=0timeout=20\opmonitorinterval=3600timeout=60\opstopinterval=0timeout=15commitexit參數(shù)添加到STONITH資源的配置中。\h#ps#ps-ef|grepsbdroot1022:44?00:00:01sbd:inquisitor59475946022:44?00:00:00sbd:watcher:/dev/sdb-slot:0-uuid:94d700ee-837b-46c7-95cc-27f3d1ffcf9froot70214root7021470209013:13?00:00:00sbd:watcher:/dev/mapper/sdb-slot:1-uuid:c036a147-ac99-49aa-b270-feada81737bcroot7021570209013:13?00:00:00sbd:watcher:/dev/mapper/sdc-slot:1-uuid:9aa3bec4-6ed6-4a5f-b656-796f1aac9e8froot7021670209013:13?00:00:00sbd:watcher:/dev/mapper/sdd-slot:1-uuid:467c0af6-eacf-4126-9969-bf68c3285839#systemctl#systemctlenablesbdSBD#sbd#sbd-d/dev/sdblist0ha01clear1ha02clear#sbd-d#sbd-d/dev/sdbmessageha01test2018-12-17T12:55:50.082675+08:002018-12-17T12:55:50.082675+08:00ha01sbd:[44807]:info:Receivedcommandtestfromha02ondisk/dev/sdb#sbd#sbd-d/dev/sdbmessageha02reset集群配置文件##crmconfigureshownode1084752417:ha01nodenode1084752418:ha02primitiveres_vip_testIPaddr2\paramsip=6cidr_netmask=24\metatarget-role=Startedprimitivestonith_sbdstonith:external/sbd\paramspcmk_delay_max=30propertycib-bootstrap-options:\have-watchdog=true\dc-version=1.1.16-4.8-77ea74d\cluster-infrastructure=corosync\cluster-name=cluster\last-lrm-refresh=1510625994\stonith-enabled=true\stonith-timeout=80s\no-quorum-policy=ignore\rsc-options\help\default-resource-stickiness=1000快速部署HA快速部署概述所有ha-cluster-bootstrap極少。使用ha-cluster-init可定義群集通訊所需的基本參數(shù)。這將為您提供一個運行中的單節(jié)點群集。使用ha-cluster-join向群集添加更多節(jié)點。使用ha-cluster-remove從群集中去除節(jié)點。前提條件NTP設置了網關路由3SBD具有root關閉防火墻HA01#zypper-nin--auto-agree-with-licenses-tpatternha_sles##zypper-nin--auto-agree-with-licenses-tpatternha_sles#echosoftdog>/etc/modules-load.d/watchdog.conf#systemctlrestartsystemd-modules-load#echo"modprobesoftdog">>/etc/init.d/boot.local#modprobesoftdog#sbd-d/dev/mapper/mpathc-d/dev/mapper/mpatha-d/dev/mapper/mpathb-460-130create#cp/etc/sysconfig/sbd/etc/sysconfig/sbd.bak#sed-i's/SBD_DELAY_START=no/SBD_DELAY_START=yes/'/etc/sysconfig/sbd#systemctlenablesbd#routeadddefaultgw0ha-cluster-init命令參數(shù)-u采用單播模式，默認采用組播模式-i指定網卡接口-M使用冗余心跳-S啟動SBDSBD#ha-cluster-init#ha-cluster-init-u-ieth1-M–S!ntpd.serviceisnotconfiguredtostartatsystemboot.Doyouwanttocontinueanyway(y/n)?yGeneratingSSHkeyConfiguringcsync2Generatingcsync2sharedkey(thismaytakeawhile)...donecsync2checkingfiles...doneConfigureCorosync(unicast):Thiswillconfiguretheclustermessaginglayer.Youwillneedtospecifyanetworkaddressoverwhichtocommunicate(defaultiseth1'snetwork,butyoucanusethenetworkaddressofanyactiveinterface).Addressforring0[4]Portforring0[5405]Addanotherheartbeatline(y/n)?yAddressforring1[4]Portforring1[5407]InitializingdisklessSBD.doneHawkclusterinterfaceisnowrunning.Toseeclusterstatus,open:4:7630/Loginwithusername'hacluster',password'linux'!Youshouldchangethehaclusterpasswordtosomethingmoresecure!WaitingWaitingforcluster doneLoadinginitialclusterconfigurationConfigureAdministrationIPAddress:OptionallyconfigureanadministrationvirtualIPaddress.ThepurposeofthisIPaddressistoprovideasingleIPthatcanbeusedtointeractwiththecluster,ratherthanusingtheIPaddressofanyspecificclusternode.DoyouwishtoconfigureavirtualIPaddress(y/n)?nDone(logsavedto/var/log/ha-cluster-bootstrap.log)#sed-i'9i/etc/sysconfig/sbd#sed-i'9i/etc/sysconfig/sbdHA02echosoftdogechosoftdog>/etc/modules-load.d/watchdog.confsystemctlrestartsystemd-modules-loadecho"modprobesoftdog">>/etc/init.d/boot.localmodprobesoftdogcp/etc/sysconfig/sbd/etc/sysconfig/sbd.baksystemctlenablesbdrouteadddefaultgw0Ha-cluster-join-I指定網卡-c指定對端主機名-y自動應答yes#ha-cluster-join#ha-cluster-join-ieth1-cha01-y!ntpd.serviceisnotconfiguredtostartatsystemboot.RetrievingSSHkeys-Thismaypromptforroot@ha01:Password:OnenewSSHkeyinstalledConfiguringcsync2...doneMergingknown_hostsProbingfornewpartitions...doneHawkclusterinterfaceisnowrunning.Toseeclusterstatus,open:5:7630/Loginwithusername'hacluster',password'linux'!Youshouldchangethehaclusterpasswordtosomethingmoresecure!Waitingforcluster doneReloadingclusterconfiguration.doneDone(logsavedto/var/log/ha-cluster-bootstrap.log)集群基礎配置##crmconfigurepropertystonith-enabled=truepropertystonith-timeout=80spropertyno-quorum-policy=ignorersc_defaultsresource-stickiness=1000primitiveres_stonith_sbdstonith:external/sbd\paramsparamspcmk_delay_max=30\opstartinterval=0timeout=20\opmonitorinterval=3600timeout=60\opstopinterval=0timeout=15primitiveres_vip_test02ocf:heartbeat:IPaddr2paramsip=6cidr_netmask=24\opstarttimeout=20sinterval=0\opstoptimeout=20sinterval=0\opmonitortimeout=20sinterval=10s\metatarget-role=StartedgroupGrp_testres_vip_test02verifycommitshow設置pacemker#rcpacemakerstop#rcpacemakerstop#rcpacemakerstart#systemctldisablepacemaker.service#systemctldisablepacemaker.service使用Csync2使用快速部署后，所有群集節(jié)點上都運行Csync2，當部分配置文件修改后可通過Csync2。檢查服務狀態(tài)##systemctlstatuscsync2.socket##csync2–f/etc/sysconfigure/sbd#csync2-x##csync2-xv使用Hawk2配置和管理群集資源#passwdhacluster#systemctlstart#passwdhacluster#systemctlstarthawk.service#systemctlstatushawk.serviceWEB3:7630/，用戶名和密碼hacluster/linux全局參數(shù)配置IP添加組資源，并勾選資源為該組成員集群腳本模板#!/bin/bash#/etc/init.d/mq_scriptcase"$1"instart)#echo"test">/opt/demosu#!/bin/bash#/etc/init.d/mq_scriptcase"$1"instart)#echo"test">/opt/demosu-mqm-c"/home/mqm/startMQ.shAML_QM"RET=$?exit$RET;;stop)#rm/opt/demosu-mqm-c"/home/mqm/stopMQ.shAML_QM120"RET=$?exit$RET;;status)echoecho-n"checking "su-mqm-c"/home/mqm/monitorMQ.shAML_QM"#cat/opt/demoRET=$?exit$RET;;*)echo"Usage:$0{start|stop|status}"exit1;;esac集群配置文件路徑CIB路徑：/var/lib/pacemaker/cib資源腳本路徑：/usr/lib/ocf/resource.d/stonith路徑：/usr/lib64/stonith/plugins/<name>集群運維命令##crm_mon##crm_mon-1##crm_mon-1–r-f##crmresourcemigrategroup_namenode_name清理資源限制命令##crmresourceunmigreategroup_name##crmconfigureshow##crmconfigureedit##crmconfigurepropertymaintenance-mode=true#crmconfigurepropertymaintenance-mode=false##crmnodemaintenanceNODENAME#crmnodereadyNODENAME##crmresourceunmanageRESOURCE_ID##crmresourcemanageRESOURCE_ID#crm#crmcorosyncdiff##crmconfigureloadpushmycluster-config.txt##crmresourcecleanupresource_name在線升級LivepatchingLinuxEnterpriseLivePatchingkGraft增補read_copyupdate(RCU)等技術，使用它可在運行時增補Linux內核，而無需停止常重要。##cat/etc/os-releaseNAME="SLES"VERSION="12-SP3"VERSION_ID="12.3"PRETTY_NAME="SUSELinuxEnterpriseServer12SP3"ID="sles"ANSI_COLOR="0;32"CPE_NAME="cpe:/o:suse:sles_sap:12:sp3"##uname-r4.4.73-7-default##zypperlrRepositoryprioritiesarewithouteffect.Allenabledrepositoriessharethesamepriority.#|Alias|Name|Enabled|GPGCheck|Refresh+ + +++1|SLE-Live-Patching12-SP3-Pool|SLE-Live-Patching12-SP3-Pool|Yes|(r)Yes|Yes2|SLE-Live-Patching12-SP3-Updates|SLE-Live-Patching12-SP3-Update