高級篇第20節(jié)消息中間件在分布式系統(tǒng)中的作用介紹對比來源于

MQ對比基本信息對比主要關注前三個（標紅）ActiveMQRabbitMQRocketMqJoramHornetQOpenMQMuleMQSonicMQZeroMQ關注度高高中中中中低低中成熟度成熟成熟比較成熟比較成熟比較成熟比較成熟新產品無成功案例成熟不成熟所屬社區(qū)/公司ApacheMozillaPublicLicenseAlibabaOW2JbossSunMuleProgress社區(qū)活躍度高高中中中低高低低文檔多多中多中中少少中特點功能齊全，被大量開源項目使用由于Erlang語言的并發(fā)能力，性能很好各個環(huán)節(jié)分布式擴展設計，主從HA；支持上萬個隊列；多種消費模式；性能很好在Linux平臺上直接調用操作系統(tǒng)的AIO，性能得到很大的提升性能非常好，與MuleESB無縫整合性能優(yōu)越的商業(yè)MQ低延時，高性能，最高43萬條消息每秒授權方式開源開源開源開源開源開源商業(yè)商業(yè)開源開發(fā)語言JavaErlangJavaJavaJavaJavaJavaJavaC支持的協議OpenWire、STOMP、REST、XMPP、AMQPAMQP自己定義的一套(社區(qū)提供JMS--不成熟)JMSJMSJMSJMSJMSTCP、UDP客戶端支持語言Java、C、C++、Python、PHP、Perl、.net等Java、C、C++、Python、PHP、Perl等JavaC++（不成熟）JavaJavaJavaJavaJava、C、C++、.netpython、java、php、.net等持久化內存、文件、數據庫內存、文件磁盤文件內存、文件內存、文件內存、文件內存、文件內存、文件、數據庫在消息發(fā)送端保存事務支持不支持支持支持支持支持支持支持不支持集群支持支持支持支持支持支持支持支持不支持負載均衡支持支持支持支持支持支持支持支持不支持管理界面一般好無社區(qū)有webconsole實現一般無一般一般好無部署方式獨立、嵌入獨立獨立獨立、嵌入獨立、嵌入獨立、嵌入獨立獨立獨立評價優(yōu)點：成熟的產品，已經在很多公司得到應用（非大規(guī)模場景）。有較多的文檔。各種協議支持較好，有多重語言的成熟的客戶端；缺點：根據其他用戶反饋，會出莫名其妙的問題，切會丟失消息。其重心放到activemq6.0產品—apollo上去了，目前社區(qū)不活躍，且對5.x維護較少；Activemq不適合用于上千個隊列的應用場景優(yōu)點：由于erlang語言的特性，mq性能較好；管理界面較豐富，在互聯網公司也有較大規(guī)模的應用；支持amqp系誒，有多中語言且支持amqp的客戶端可用缺點：erlang語言難度較大。集群不支持動態(tài)擴展。優(yōu)點：模型簡單，接口易用（JMS的接口很多場合并不太實用）。在阿里大規(guī)模應用。目前支付寶中的余額寶等新興產品均使用rocketmq。集群規(guī)模大概在50臺左右，單日處理消息上百億；性能非常好，可以大量堆積消息在broker中；支持多種消費，包括集群消費、廣播消費等。開發(fā)度較活躍，版本更新很快。缺點：產品較新，文檔比較缺乏。沒有在mq核心中去實現JMS等接口，對已有系統(tǒng)而言不能兼容。阿里內部還有一套未開源的MQAPI，這一層API可以將上層應用和下層MQ的實現解耦（阿里內部有多個mq的實現，如notify、metaq1.x，metaq2.x，rocketmq等），使得下面mq可以很方便的進行切換和升級而對應用無任何影響，目前這一套東西未開源支持的Protocols對比Thefollowingtableliststhesupportedprotocolsofeachbroker.ActiveMQRabbitMQRocketMQHornetQQpidZeroMQAMQP1.00-8,0-9,0-9-1-announced1.0-MQTT----OpenWire-----REST---STOMP---STOMPoverWebsockets---XMPPOverGateway----ClientInterfaces對比ActiveMQRabbitMQRocketMQHornetQQpidZeroQC--C++--Erlang----Haskell----JavaJMS--ActiveMQRabbitMQRocketMQHornetQQpidZeroQJavaproprietary--.NET---Objective-C-----Perl----PHP----Python---Ruby---mqbenchmark[文獻1]結論在傳統(tǒng)的MQ中rabbitmq的性能表現最好。kafka的性能優(yōu)于rabbitmq；傳統(tǒng)MQ對比場景設計：ScenarioA:Enqueuing20,000messagesof1024byteseach,thendequeuingthemafterwards.ScenarioB:Enqueuinganddequeuingsimultaneously20,000messagesof1024byteseach.ScenarioC:Enqueuinganddequeuingsimultaneously200,000messagesof32byteseach.ScenarioD:Enqueuinganddequeuingsimultaneously200messagesof32768byteseach.ScenarioAScenarioBScenarioCScenarioDKafkavsrabbitmq結論在kafka和rabbitmq的對比中，kafka的性能表現較好。rabbitmqkafkarocketmq設計初衷更多工作是保證消息遞交；認為consumer是一直處于alive狀態(tài)去消費消息的；broker設計的比較重，需要記錄很多狀態(tài)；充分考慮消息堆積因素，認為consumer不一定處于alive狀態(tài)；考慮各個角色的分布式；為追求吞吐量設計；broker設計較輕，不保存consumer的消費狀態(tài)同kafka；且考慮了事務特性；路由特性提供了豐富的routing，實現了amqp的exchange、binding、queuing等model，queue、topic等routing都支持；topicroutingonlytopicroutingonly集群特性對應用透明的集群式實現；雖然是集群式的實現，但是集群對producer并不是完全透明，producer需要確認消息發(fā)送到哪一個同kafkapartition；同時也利用這一點實現了消息的順序遞交特性（partition內的消息是順序的）；HA通過mirrorqueue來支持（master/slave）master提供服務，slave僅備份通過replica來支持queue的master/slave的可以自動切換（當master宕掉后）支持，但需要人工切換master和slave的角色；消息處理量消息量~=20k/sec;消息量>100k/sec消息量>100k/sec隊列數目<1000支持上萬隊列支持上萬隊列順序投遞不支持支持支持事務性不支持不支持支持簡介rocketmq的前身為metaq，metaq的第一個版本是可以看成是linkedin的kafka（scala）的java版本，并對其增加了事務的支持。rocketmq為metaq3.0，相比于原始kafka，其擅長點出了原始的logcollecting之外，還增加諸如HA、事務等特性，使得從功能點上講，可以替代傳統(tǒng)大部分MQ。目前已經在阿里開始大規(guī)模應用，并且預計未來會逐漸在阿里取代notify，meta2.x以前的所有隊列。由于rocketmq還比較新，且官方沒有給出相應的benchmark。而rocketmq社區(qū)主要活躍者均是中國國內人員，因此不便于直接對比rocketmq和其他隊列。在這里用kafka和其他隊列進行對比，rocketmq的性能比kafka還要好一些。kafkavsrabbitmqindetailKafkaisageneralpurposemessagebroker,likeRabbItMQ,withsimilardistributeddeploymentgoals,butwithverydifferentassumptionsonmessagemodelsemantics.Iwouldbeskepticalofthe"AMQPismoremature"argumentandlookatthefactsofhoweithersolutionsolvesyourproblem.UseKafkaifyouhaveafirehoseofevents(100k+/sec)youneeddeliveredinpartitionedorder'atleastonce'withamixofonlineandbatchconsumers,youwanttobeabletore-readmessages,youcandealwithcurrentlimitationsaroundnode-levelHA(orcanusetrunkcode),and/oryoudon'tmindsupportingincubator-levelsoftwareyourselfviaforums/IRC.UseRabbitifyouhavemessages(20k+/sec)thatneedtoberoutedincomplexwaystoconsumers,youwantper-messagedeliveryguarantees,youdon'tcareaboutordereddelivery,youneedHAatthecluster-nodelevelnow,and/oryouneed24x7paidsupportinadditiontoforums/IRC.Neitheroffersgreat"filter/query"capabilities-ifyouneedthat,considerusingStormontopofoneofthesesolutionstoaddcomputation,filtering,querying,onyourstreams.OrusesomethinglikeCassandraasyourqueryablecache.Kafkaisalsodefinitelynot"mature"eventhoughitis"productionready".Details(caveat-myopinion,I'venotusedeitheringreatanger,andIhavemoreexposuretoRabbitMQ)Firstly,onRabbitMQvs.Kafka.Theyarebothexcellentsolutions,RabbitMQbeingmoremature,butbothhaveverydifferentdesignphilosophies.Fundamentally,I'dsayRabbitMQisbroker-centric,focusedarounddeliveryguaranteesbetweenproducersandconsumers,withtransientpreferredoverdurablemessages.WhereasKafkaisproducer-centric,basedaroundpartitioningafirehoseofeventdataintodurablemessagebrokerswithcursors,supportingbatchconsumersthatmaybeoffline,oronlineconsumersthatwantmessagesatlowlatency.RabbitMQusesthebrokeritselftomaintainstateofwhat'sconsumed(viamessageacknowledgements)-itusesErlang'sMnesiatomaintaindeliverystatearoundthebrokercluster.Kafkadoesn'thavemessageacknowledgements,itassumestheconsumertracksofwhat'sbeenconsumedsofar.BothKafkabrokers&consumersuseZookeepertoreliablymaintaintheirstateacrossacluster.RabbitMQpresumesthatconsumersaremostlyonline,andanymessages"inwait"(persistentornot)areheldopaquely(i.e.nocursor).RabbitMQpre-2.0(2010)wouldfalloverifyourconsumersweretooslow,butnowit'srobustforonlineandbatchconsumers-butclearlylargeamountsofpersistentmessagessittinginthebrokerwasnotthemaindesigncaseforAMQPingeneral.Kafkawasbasedfromthebeginningaroundbothonlineandbatchconsumers,andalsohasproducermessagebatching-it'sdesignedforholdinganddistributinglargevolumesofmessages.RabbitMQprovidesrichroutingcapabilitieswithAMQP0.9.1'sexchange,bindingandqueuingmodel.Kafkahasaverysimpleroutingapproach-inAMQPparlanceitusestopicexchangesonly.Bothsolutionsrunasdistributedclusters,butRabbitMQ'sphilosophyistomaketheclustertransparent,asifitwereavirtualbroker.Kafkamakesitexplicit,byforcingtheproducertoknowitispartitioningatopic'smessagesacrossseveralnodes,thishasthebenefitofpreservingordereddeliverywithinapartition,whichisricherthanwhatRabbitMQexposes,whichisalmostalwaysunordereddelivery(theAMQP0.9.1modelsays"oneproducerchannel,oneexchange,onequeue,oneconsumerchannel"isrequiredforin-orderdelivery).Putanotherway,Kafkapresumesthatproducersgenerateamassivestreamofeventsontheirowntimetable-there'snoroomforthrottlingproducersbecauseconsumersareslow,sincethedataistoomassive.ThewholejobofKafkaistoprovidethe"shockabsorber"betweenthefloodofeventsandthosewhowanttoconsumethemintheirownway--someonline,othersoffline-onlybatchconsumingonanhourlyorevendailybasis.Kafkacandeliver"atleastonce"semanticsperpartition(sincemaintainsdeliveryorder),justlikeRabbitMQ,butitdoesitinaverydifferentway.Performance-wise,ifyourequireordereddurablemessagedelivery,currentlyitlookslikethere'snocomparison:KafkacurrentlyblowsawayRabbitMQintermsofperformanceonsyntheticbenchmarks.Thispaperindicates500,000messagespublishedpersecondand22,000messagesconsumedpersecondona2-nodeclusterwith6-diskRAID10./en...OfcoursethiswaswrittenbytheLinkedInguyswithoutnecessarilyexpertRabbitMQinput,soYMMV.Finally,areminder:KafkaisanearlyApacheincubatorproject.Itdoesn'tnecessarilyhaveallthehard-learnedaspectsinRabbitMQ.Now,awordonAMQP.Frankly,itseemsthestandardisamess.Officiallythereisa1.0proposedspecificationthatisgoingthroughtheOASISstandardsprocess.Inpracticeitisaforkedstandard,one(0.9.1)supportedbyvendors,theother(1.0)supportedbytheworkinggroup.Asetofgenerallyavailable,widely-adopted,production-qualityAMQP1.0implementationsacrossthemajorreleases(QpidfromRedhat,RabbitMQ,etc.)won'texistuntil2013,ifever.Asanexternalobserverwithnoinsideknowledge,hereiswhatitlookslike:theworkinggroupspent5yearsonaspec,from2003to2008,culminatinginawidelyadoptedrelease(0.9.1).Thenasubsetofmorepowerfulworkinggroupmembersrewrotethespecbylate2011,completelyshiftingthefocusofthespecfromamessagingmodeltoatransportprotocol(sortoflikeTCP++),anddeclaredit1.0.So,wehavethestrangecasewherethe"mature"AMQPisthenon-standard0.9.1specificationandthe"immature"AMQPistheactual1.0standard.Thisisn'ttosuggest1.0isn'tgoodtechnology,itlikelyis,