超大規(guī)模信息系統(tǒng)核心技術(shù) 丁軼群 博士@還是看書課件

1、超大規(guī)模信息系統(tǒng)核心技術(shù)丁軼群 博士還是看書超大規(guī)模信息系統(tǒng)定義超大規(guī)模信息系統(tǒng)總體框架超大規(guī)模信息系統(tǒng)的一些細節(jié)性能建模存儲安全消息Agenda超大規(guī)模信息系統(tǒng)指的是規(guī)模龐大、結(jié)構(gòu)復(fù)雜、目標(biāo)多樣、影響因素眾多，具有高可靠性、高并發(fā)性和高可擴展性的信息系統(tǒng)。其中：規(guī)模龐大：主要反映在以下維度1 4 5原代碼多：可達到上百萬行代碼開發(fā)人員多：幾百上千人，全球多個團隊協(xié)作開發(fā)數(shù)據(jù)量大2 ：處理的數(shù)據(jù)達到PB級并發(fā)量大：可能會全球上億人同時訪問結(jié)構(gòu)復(fù)雜：包括以下幾個方面1 4 5組件關(guān)系復(fù)雜：可能會有遺留系統(tǒng)的組件、第三方組件和自己開發(fā)的組件持久化機制復(fù)雜：可能有文件、數(shù)據(jù)庫等硬件平臺復(fù)雜：可能包括

2、嵌入式系統(tǒng)、PC、大型機、手機、傳感器等等超大規(guī)模信息系統(tǒng)在工業(yè)界、政府、軍隊和科研機構(gòu)都有廣泛的應(yīng)用3，如一些大型互聯(lián)網(wǎng)內(nèi)容提供商、電子商務(wù)、文本、視頻系統(tǒng)、在線游戲、金融系統(tǒng)、醫(yī)療系統(tǒng)、海量科研數(shù)據(jù)處理，等等。超大規(guī)模信息系統(tǒng)定義 1/2超大規(guī)模信息系統(tǒng)定義 2/2NameExampleConcurrencyServerLines of codeData sizeSmall system個人網(wǎng)站、公司、政府的門戶網(wǎng)站每天幾百次訪問一臺PC機即可幾千行MBMedium systemFreecity, MIS, etc幾千用戶同時訪問專門的服務(wù)器幾萬行GBLarge system19樓、天涯、

3、貓撲上萬用戶同時訪問專門的服務(wù)器集群幾十萬行TBVery large systemGoogle, Taobao, Facebook, Youtube, etc. 上億用戶同時訪問大型數(shù)據(jù)中心上百萬行PBUltra large systemFutureTBD云計算、互聯(lián)網(wǎng)上億行代碼EB?超大規(guī)模信息系統(tǒng)總體框架 2/2目標(biāo)：速度 可擴展性 可靠性=自行開發(fā)或較大改動=利用現(xiàn)有工具=暫時不做性能專家面對的問題20% 的應(yīng)用程序存在過性能問題50% 的主管對性能有信心14% 機構(gòu)有正式的性能管理流程71% 機構(gòu)性能問題被用戶投訴性能建模-問題 1/2A Survey from 150 euro se

4、nior IT Managers ，2006性能建模: 為計算機軟件、計算機硬件、系統(tǒng)用戶行為建立數(shù)學(xué)模型,以數(shù)學(xué)推理方法預(yù)測系統(tǒng)在不同負(fù)載下的性能性能=f(系統(tǒng)負(fù)載，系統(tǒng)資源)性能建模-是什么性能建模-流程性能建模-實例Predict the performance of a Post trade systemTechnology:Struts ServletHibernateJDBC Topology:性能建模-實例性能建模-實例過載控制容量規(guī)劃定量評估系統(tǒng)組件、系統(tǒng)架構(gòu)評估新功能、新用戶、新平臺的性能影響性能建模-應(yīng)用存儲&事務(wù)-總體思路 2/2關(guān)系型數(shù)據(jù)庫DataFault Toler

5、antDecentralizedScalableHigh PerformanceWeak TransactionProvenStrong transaction supportSometimes an overkillHard to ScaleNOSQLSQL16關(guān)系型數(shù)據(jù)庫方面 1/2SSD Base storage optimizationDecrease the small writeDecrease the in-place updateMulti-core based optimizationDecrease the granularity of lockDecrease the c

6、ost of thread switch(membrain)Memory optimizationBetter buffer flush algorithmTotal Order based sync replication云存儲-NOSQL方面NOSQL vs. SQL 讀寫速度19NOSQL成功案例簡單的數(shù)據(jù)模型和操作無需強事務(wù)支持高可擴展性高性能海量數(shù)據(jù)寫操作靈活的數(shù)據(jù)schema高可用性簡單管理與維護Twitter tweetsYes, Query by ID and AuthorYesYes, TB YesYesYesFacebook Inbox SearchYesYesYes, T

7、B YesYesYesYesDigg Green BadgeYesYesYesYesYesTaobao Product DetailYesYesYesYesYes20NOSQL的市場需求Market Research Media: NoSQL Market Forecast 2011-201521代表性NOSQL系統(tǒng)工業(yè)界Dynamo, AmazonBigtable, GoogleCassandra, ApacheHbase, Apache學(xué)術(shù)界Scalaris, Zuse Institute BerlinecStore, National University of SingaporeG-S

8、tore, UCSB 22NOSQL存儲工作計劃Cassandra BasedBigtable style data model can support complex operation while still has high scalabilityIs easy to hacked-in compared with HbaseOptimized for read latencyAtomicity guarantees across multiple keysOptimized for some sub-component, e.g. failure detectorOptimized f

9、or multi-core and flash-storageSecurity, Granular ACL controlMore complex data access model23Cassandra- 橫向可擴展性Cassandra-高可用性011/2FEDCBAN=3h(key1)并發(fā)讀Cassandra-高讀寫性能 1/2QueryClosest replicaCassandra ClusterReplica AResultReplica BReplica CDigest QueryDigest ResponseDigest ResponseResultClientCassandra

10、- 可動態(tài)調(diào)整一致性/持久性與延時WriteReadLevelDescriptionLevelDescriptionZEROCross fingersOne1st ResponseOne1st ResponseQUORUMN/2 + 1 ReplicasQUORUMN/2 + 1 ReplicasALLAll ReplicasALL All Replicas一致性模型取決于副本（Replicas）的數(shù)量（N）,一般為3 在Cassandra中一般選擇Quorum數(shù)量的讀節(jié)點數(shù)（R,一般為2）以及Quorum數(shù)量的寫節(jié)點數(shù)(W,一般為2)Cassandra-實例 1/3Cassandra-實例

11、2/3Cassandra-實例 3/3惡意代碼檢測-目標(biāo)Status update Evaluate existing security products (AppScan, Fortify, Veracode) and find the security issues that they dont coverSurvey on the innovative approaches in research area related to malicious code detectionGoalFind out a security solution for the uncovered secur

12、ity issuesIBM AppScanAn application security testing suite designed to automate vulnerability assessments for web applications, with a black-box approachAppScan EngineSDKUIExtensionsbrowser-side malicious code detection Hybrid JavaScript Analysis34HP FortityAn application security testing suite desi

13、gned to automate vulnerability assessments for software, with both static & dynamic approachesSoftware LifecycleSource Code Analysis (SCA)Program Trace Analysis (PTA)Runtime Time Analysis (RTA)18 Development Languages Java, .NET, C/C+ JavaTomcatWebLogicWebSphereJBossIISLinuxSolarisAIXWindows ServerH

14、P-UX.NETJavaTomcatWeblogicWebsphereJBossIISSolarisMac OS XWindows 2K/2003/XPLinuxWindows Server.NET35Fortify - Static ApproachSource CodeAnalysis EngineControl FlowData FlowNST18 Source codeReport36Fortify - Dynamic ApproachPTA/RTA RulesInterceptCovered Vulnerabilities:SQL InjectionCross-Site Script

15、ing: ReflectiveCross-Site Scripting: PersistentCommand InjectionArbitrary URL RedirectionPath ManipulationHTTP Response SplittingUnhandled ExceptionPrivacy Violation: Social Security NumberPrivacy Violation: Credit Card NumberLog ForgingXpath InjectionLDAP Injection-Buffer overflowCommand injectionX

16、-Site scriptingDirectory listingHTTP response splittingPrivacy violationsProbingSQL injectionInformation leakageUnhandled exceptions37VeracodeA suit of application security testing service designed to automate vulnerability assessments for software, with both static & dynamic & Manual approachesBina

17、ry and/or URLStatic Binary AnalysisDynamic Penetration TestingManual Penetration TestingReportCode security analysis38How Veracode Static binary and Dynamic approach worksProgram BinaryLoaderdisassemble the binaryApplication ModelerCreate a model of the application including data flow, control flow,

18、 and etc.Model AnalyzerPerform analysis by scanning the modelReporterGenerate security metrics & correlate security flaws with original source codeSecurity MetricsDetailed Flaw ListURLNTOSpider Engineenumerates the web siteAttack phasesubmits specially crafted requests to trigger potentially malicio

19、us application behaviorReportStatic binary analysisDynamic analysis39Survey results of Research malware detection40How Do We Differ?FVFVAVAVOurTeamDynamicStaticPenetration TestSoftware WeaknessDetection ApproachF=FortifyV=VeracodeA=AppScan41Dynamic ComponentMalicious Code Detection Platform42QEMUTEM

20、UMalware AnalysisVulnerabilitydetection, diagnosis and defenseProtocol Reverse EngineeringPanoramaRevonoHookFinderBitScopePolyglotDispatcherstingAPEGMineSweeperAnubisGFI SandBoxNorman SandBoxJoe SandBoxStatic Component42消息中間件-現(xiàn)狀與目標(biāo)43FunctionalityNon-FunctionalHRZARich FunctionalityScalable, Reliable

21、 Based on Erlang OTP AMQPFast PersistenceJournal solutions AIOHigh PerformanceBrokerlessAsync, Non-blockingZero copy, Buffer 2004(ASL)2008(MPL)2009(ASL&LGPL)2010(LGPL)為什么要開發(fā)自己的消息中間件1. VLIS Messaging TargetHigh performance, high scalability, high reliability for financial transaction.Light-weight, lo

22、w-cost, easy-to-use.Commercialization.2. Why not choose an open-source product?No products can satisfy all of the above requirements.Cant follow technology innovation, optimization necessarily.License barrier.3. Our Strategy:Following design philosophy.Light-weight, plugable.44目標(biāo)High PerformanceI/O

23、& Concurrency Model: Asynchronous, Non-BlockingMemory Management: Different sizes processing, Zero-copy, Mem_pool Cache Locality: Time & SpaceHigh Scalability Load balancing: Static & Dynamic IPC via messages: Take advantage of multi-core Lock-free: CAS(Compare And Swap)3. High Reliability High avai

24、lability: Master & Slave, Failure detection(Heart-beating), Recovery strategy(backoff, Idempotency) Persistence: High Performance Journal solution45Monitoring-基于現(xiàn)有工具的平臺層監(jiān)控Schedule(周曉震+李偉+王耿亮+1RA)9月:選型10月:安裝使用，總結(jié)經(jīng)驗，嘗試開發(fā)插件11-12月:開發(fā)用于中間件的插件，并與我們的平臺集成1月:測試46緩存-基于現(xiàn)有工具的分布式緩存開源軟件名稱功能簡介Memcached分布式緩存方案，共享緩存

25、方式Ehcache支持分布式緩存設(shè)計，支持RMI, JGroup, JMS進行通訊，支持replication和invalidation同步信息JBossCache支持分布式緩存配置，支持嚴(yán)格事務(wù)處理SwarmCache輕量級分布式緩存軟件JCS支持對象存儲的分布式緩存軟件dbcached基于memcached協(xié)議的一款分布式緩存軟件，支持事務(wù)功能Hbase基于Big Table的Key-Value文件存儲系統(tǒng)，支持分布式存儲OSCache大而全的緩存軟件，通過不同的配置可以實現(xiàn)集群中的分布式緩存Scalaris比較新的開源分布式緩存軟件，Key-Value類型，支持事務(wù)處理MongoDB分布

26、式文件存儲系統(tǒng)軟件47Micro Container-微容器的職責(zé)業(yè)務(wù)層服務(wù)對象的生命周期管理向業(yè)務(wù)層對象提供由其他中間件提供的企業(yè)服務(wù)業(yè)務(wù)層應(yīng)用部署和管理暴露系統(tǒng)服務(wù)、業(yè)務(wù)層服務(wù)管理員接口微容器自身運行所需結(jié)構(gòu)：類加載器層次結(jié)構(gòu)、異常處理Application LevelProgramming TechniquesSystem LevelProgramming TechniquesArchitecture DesignDomain KnowledgeStrutsSpringHibernateSocket programmingEclipseSVN/CVSConcurrency modelCl

27、ass loader hierarchyMemory managementMulti-thread programmingCachingConsensus AlgorithmsQueuing network modelPetri NetBayesian Network ModelWavelet TransformationEXP合作機會合作計劃成立興趣小組學(xué)習(xí)反饋加入團隊姓名、學(xué)號、電話、參與小組尹可挺老師報名流程：消息中間件興趣小組學(xué)習(xí)計劃和方法:學(xué)習(xí)網(wǎng)絡(luò)協(xié)議: TCP/IP協(xié)議詳解卷1（了解基本的網(wǎng)絡(luò)協(xié)議方面的內(nèi)容）學(xué)習(xí)基本的socket網(wǎng)絡(luò)編程 UNIX網(wǎng)絡(luò)編程第三版卷1 (重點章節(jié):

28、2，3，4，5，6，8，16，30)通過閱讀開源庫來掌握網(wǎng)絡(luò)編程的基本IO模型 對于懂java的同學(xué): 了解java IO和NIO中socket的一些編程方法，并且能夠通過閱讀源代碼了解網(wǎng)絡(luò)開源庫mina, netty，grizzly之一的基本工作原理。 對于懂C/C+的同學(xué): 通過閱讀源代碼了解libevent的基本工作原理。閱讀hornetQ的快速指南和用戶手冊。/hornetq/chinesedocs.html閱讀我的博客上對zeroMQ的分析并且結(jié)合zeromq源代碼理清思路。/kaka11Output（以下任務(wù)均不必掌握細節(jié)，目前這個階段就是要掌握基礎(chǔ)的原理）:UNIX網(wǎng)絡(luò)編程重點章節(jié)的讀書筆記。對應(yīng)任務(wù)A、B，2周寫下libevent，mina, netty，grizzly其中一個的源代碼分析來介紹基本工作原理。對應(yīng)任務(wù)C，2周hornetQ用戶手冊的讀書筆記。對應(yīng)任務(wù)D，2周學(xué)習(xí)計劃和方法:了解NoSQL相關(guān)概念:/wiki/NoSQL閱讀相關(guān)經(jīng)典論文前