騰訊TXSQL賦能云分布式數(shù)據(jù)庫CynosDB

1、騰訊T X S Q L 賦能云分布式數(shù)據(jù)庫C y n o s D BPART01架構(gòu)介紹PART02機(jī)遇&挑戰(zhàn)PART03實(shí)現(xiàn)&優(yōu)化PART04產(chǎn)品演進(jìn)01架構(gòu)介紹TXSQLRedundancyI/OMigrationUpgradeBackup&RecoveryRTO/RPOScalabilityMasterSlaveDiDsikskDiDsikskDiskDiskCOSDataBinlogRedoFileCynosDBMasterReplicaDiskDiskCOSDataBinlogRedoFileCellCellCellTXStoreCBSFewer I/OLog is database

2、ReliabilityAvailabilityScalabilityThroughputLog TrafficStorage ClientHot LogData BlockCellSyncAsyncInnoDB1、Send log record2、Persist log record3、Ack for engine4、Apply log records into block5、Advance base block6、Garbage collect old versions02機(jī)遇& 挑戰(zhàn)Feature Shared storage I/O offload Stateless Computabl

3、e storage Parallel & AsynchronousBenefit StorageSharded Log/DataMultiple CopiesRecoveryBackupStorage Pool Light EngineOptional BinlogFile SystemFlush PageDouble WriteCheckpointChange BufferChallenge I/O GC Consistency DDL03實(shí)現(xiàn)& 優(yōu)化Asynchronous Group CommitWait for log flushedRace conditionContext swit

4、chStartOperationsCommitFlush LogFinishStartOperationsCommitWait LogFinishStartOperationsCommitRe turnStartOperationsCommitRe turnBGThreadsFinishFinishFinishThread poolBackground threadsLock-free queueContext holdingLogging Parallelly Background logging threads Lock-free buffer Easily serializeLSNUse

5、r ThreadUser ThreadUser ThreadWriterFlusherDoneIn progressDoneLog FlowTYPESpace IDPage noContentBlockHeaderTailMTR1MTR2MTR3 Log flow Compact format Remove unnecessary log Transfer / Archive （Reduce 20%-30%）MTR4RIO Lock-freeSync R/WRIO Handle ThreadcellRIO QueueAsync R/WRIO Queuecellcellcellcell Asyn

6、chronous ParallelReplicaScan ThreadHash tableHash tableHash tableHash tableApply ThreadApply ThreadApply ThreadApply ThreadBuffer PoolLog flow Transaction DelayBatch OperationParallelMillisecond Level Transaction ConsistencyDelayedRC/SI Structure ConsistencyBtreeSegmentUndo&DataGCMasterDataLogPurgeC

7、heckpointCellCellCellTXStoreGC state Oldest TRX Purge Checkpoint Cluster oldest TRX Combined-Purge Advance page version Combined-CheckpointReplicaReplicaMasterReplicaReplicaMetaDataServer MetadataFRMOPTTRNTRGInnoDB MetadataSYS_TABLESSYS_INDEXESSYS_COLUMNSServer Metadata TablesTablesLogsTriggersCheck

8、point & RecoveryCheckpointed DataRedo logCheckpointDL(Durable LSN)Checkpointed DataCheckpoint & DLBase pageDL(cell)Incremental checkpointFlush listSingle thread parse(2-3 times)A large number of disk accessSlow(5 mins)No checkpointParallelAsynchronousReplay on demandFast(5s)Buffer & Space No flush o

9、rder mutex Temporary space only Fast eviction No space reservation 0 extending cost Short latching Few file system latchingSurvivable Cache Keep warm Fast startup High availabilityCachesInnoDBCellCellCellTXStoreR/WLoadVertical optimization Log fragmented Space limited Extent recycling Direct IOA New Transaction System Not lock-free hash Remove transaction system mutex Fusion transaction table(Phys