HEVC編碼技術(shù)概覽

HEVC編碼技術(shù)概覽■

ISO-IEC/MPEG和ITU-T/VCEG成立了一個研究視頻編碼的聯(lián)合協(xié)作小組JCT-VC(thejointcollaborativeteamonvideocoding)，其宗旨是建立新一代的視頻編碼標準。

■

2010年4月在德國德雷斯頓召開了JCT-VC(JointCollaborativeTeamonVideoCoding)第一次會議，確定了新一代視頻編碼標準的名稱：HEVC(HighEfficiencyVideoCoding)。并且建立了測試模型TMuC(TestModelunderConsideration)，分領(lǐng)域搜集和審閱技術(shù)提案。初步定于2012年7月完成標準的最終稿。

■

2010年10月在廣州召開了JCT-VC第三次會議，會上確立了HEVC的第一個試驗?zāi)Ｐ虷M1。

■

新一代視頻壓縮標準的核心目標，是在H.264的基礎(chǔ)上將壓縮效率提高一倍。即在保證相同視頻圖像質(zhì)量的前提下，視頻流的碼率減少50%。在提高壓縮效率的同時，可以允許編碼端適當提高復(fù)雜度。

HMcontainsonlyaminimumsetofwell-testedtoolsthattogetherfromacoherentdesignthatisconfirmedtoshowgoodcapability.

Twosettingshavebeendefined：1)highefficiency;2)low-complexity.HMHMConfigurationHighEfficiencyConfigurationLowcomplexityConfigurationCodingUnittreestructure(8×8upto64×64lumasamples)PredictionUnitsTransformunittreestructure(3levelmax.)Transformunittreestructure(2levelmax.)Transformblocksizeof4x4to32x32samples(alwayssquare)AngularIntraPrediction(34directionsmax.)DCT-basedinterpolationfilterforlumasamples(1/4-sample,12-tap)Directionalinterpolationfilterforlumasamples(1/4-sample,6-tap)Bi-linearinterpolationfilterforchromasamples(1/8-sample)AdvancedmotionvectorpredictionContextadaptivebinaryarithmeticentropycodingLowcomplexityentropycodingphase2Internalbit-depthincrease(4bits)XXTransformprecisionextension(4bits)DeblockingfilterAdaptiveloopfilterX1、PicturePartitioning2、IntraPrediction3、InterPrediction4、TransformandQuantization5、EntropyCoding6、LoopFiltering7、CostFunction1、PicturePartitioning1.1.Treeblock(TB)PartitioningPicturesaredividedintoslicesandslicesaredividedintoasequenceoftreeblocks.AtreeblockisanNxNblock

oflumasamplestogetherwiththetwocorrespondingblocksofchromasamplesforapicturethathasthreesamplearrays.ThetreeblockconceptisbroadlyanalogoustothatofthemacroblockinpreviousstandardssuchasH.264/AVC[6].Themaximumallowedsizeofthetreeblockis64x64lumasamples.

TheCodingUnit(CU)isthebasicunitofregionsplittingusedforinter/intraprediction.

Codingunit(CU)thebasiccodingunitlikethemacroblock.CUscanbefurthersplitintopredictionunits(PUs).transformunit(TU)isdefinedfortransformandquantization.Itisalwayssquareanditmaytakeasizefrom8x8lumasamplesuptothesizeofthetreeblock.(8x8,16x16,32x32,64x64)TheCUconceptallowsrecursivesplittingintofourequallysizedblocks,startingfromthetreeblock.Thisprocessgivesacontent-adaptivecodingtreestructurecomprisedofCUblocks.1.2.CodingUnit(CU)structureGiventhattheregionofinterestishomogeneousandthelargesizesaCUcanhave,arelativelylargeCUcanrepresentthisregioninasmallernumberofsymbols.ThewiderangeofsizesforLCUqualitiesthecodectobeefficientandcarefullyoptimizedaccordingtousageorapplication

ThePredictionUnit(PU)isthebasicunitusedforcarryingtheinformationrelatedtothepredictionprocesses.PUsaredefinedafterthelastlevelofCUsplittingIngeneral,itisnotrestrictedtobeingsquareinshape,inordertofacilitatepartitioningwhichmatchestheboundariesofrealobjectsinthepicture.EachCUmaycontainoneormorePUs.1.3.PredictionUnit(PU)structureFigure1.3.FourtypesofPredictionUnitstructurePredictiontypecanhavethefollowingvalues:SkipIntra-prediction

Inter-prediction

illustratesallpossiblePUsplittingaccordingtodifferent

prediction

types.從上圖中可以看到一種新的不對稱運動分割預(yù)測（AsymmetricMotionPartition,AMP）,這也是HEVC與H.264在分塊預(yù)測技術(shù)中最為不同之處。所謂AMP即將編碼單元分割成兩個大小不一致的預(yù)測塊其中一個PU單元的寬/長為CU單元的3/4,這種預(yù)測考慮了大尺寸可能的紋理分布可以有效提高大尺寸塊的預(yù)測效率

TheTransformUnit(TU)isthebasicunitusedforthetransformandquantizationprocesses.Itisalwayssquareanditmaytakeasizefrom4x4upto32x32lumasamples.EachCUmaycontainoneormoreTUs,wheremultipleTUsmaybearrangedinaquadtreestructure,asillustratedinFigure1.4below.3-levelquadtreeforhighefficiencyconfiguration.Samemaximumquadtreedepthforlumaandchroma.1.4.TransformUnit(TU)structureFigure1.4.ExampleofTransformUnitstructure配合不對稱預(yù)測單元以及矩形預(yù)測單元，新的HM4.0模型還采納了相應(yīng)的矩形四叉數(shù)TU結(jié)構(gòu)突破了方塊變換的限制，下圖展示了3級矩形四叉樹變換水平TU結(jié)構(gòu)，同理可以有垂直分割結(jié)構(gòu)SummeryofCU,PU,TUthreeindependentblockconcepts:CU(Codingunit):thebasicunitofregionsplitting;PU(Predictionunit):thebasicunitofinter/intraprediction;TU(Transformunit):thebasicunitoftransform.CUisanalogoustotheconceptofmacroblock,butitdoesnotrestrictthemaximumsizeanditallowsrecursivesplittingintofourequalsizeCUstoimprovethecontentsadaptivity.EachCUmaycontainoneormorePUs.PUmaycontainmultiplearbitraryshapepartitionsinasinglePUtoeffectivelycodeirregularimagepatterns.TUcanbedefinedindependentlyfromthePU;however,itssizeislimitedtotheCUwhichtheTUbelongsto.Thisseparationoftheblockstructureintothreedifferentconceptsallowseachtobeoptimizedaccordingtoitsrole,whichresultsintheimprovedcodingefficiency.Codingunittreestructureislimitedfrom8×8to64×64forluma,thatistosay,nosplittingisallowedforCU3.IftheLCUsize=16andhierarchicaldepth=2,thenthisisasimilarcodingstructuretomacro-blockandsub-macro-blockinH.264/AVC.TospecifythepredictionmethodusePUtypeandPUsplitting.PUtypescanbeskip,intraandinter.ItshouldbenotedthatthesizeofTUmaybelargerthanthatofthePUbutnotexceedthatoftheCU.TUhavedifferentsplittingforlowcomplexity(LC)andhighefficiency(HE)configurations.2、IntraPredictionTheunifiedintrapredictioncodingtoolprovidesupto33directionalpredictionmodesandoneDCpredictionmodeforeachPU.Figure2.1.The33intrapredictiondirectionsFigure2.2.MappingbetweenintrapredictiondirectionandintrapredictionmodeTable2.1.NumberofsupportedintramodesaccordingtoPUsizePUsizeNumberofintramodes41783416343234645ThetotalnumberofavailablepredictionmodesisdependentonthesizeofthecorrespondingPU,asshowninTable2.1below.Thepixelisreconstructedusingthelinearinterpolationofthereferencetoporleftsamplesat1/32thpixelaccuracyforallblocksizes,insteadofdifferentaccuraciesfordifferentsizes.Twoarraysofreferencesamplesareused,correspondingtotherowofsampleslyingabovethecurrentPUtobepredicted,andthecolumnofsampleslyingtotheleftofthesamePU.Givenadominantpredictiondirection(horizontalorvertical),oneofthereferencearraysisdefinedtobethemainarrayandtheotherarraythesidearray.ThereferencerowabovethePUiscalledthemainarrayandthereferencecolumntotheleftofthesamePUiscalledthesidearray,incaseofverticalprediction.WhilethereferencecolumntotheleftofthePUiscalledthemainarrayandthereferencerowabovethePUiscalledthesidearray,incaseofhorizontalprediction.Onlythemainarraysamplesareusedforpredictionwhentheintrapredictionangleispositive,Whentheintrapredictionangleisnegative,aper-sampletestisperformedtodeterminewhethersamplesfromthemainorthesidearrayshouldbeusedforprediction,asshowninfigure.Whenthesidearrayisused,thecomputationoftheindexofthesidearrayrequiresadivisionoperation.Thedivisionoperationisremovedbyusingalook-uptable(LUT)technique.Thistechniqueisusedforthenegativeangularpredictionprocessinthecalculationofthey-interceptincaseofverticalpredictionorthex-interceptin

caseofhorizontalprediction.Theintegerandfractionalpartsoftheinterceptarecalculatedusingthefollowingequations:deltaIntSide=(256*32*(l+1)/absAng)>>8deltaFractSide=(256*32*(l+1)/absAng)%256absAngistheabsolutevalueofintrapredictionangle(=[2,5,9,13,17,21,26,32])andlisx/ypixellocationforvertical/horizontalprediction.UsingtheLUTtechniques,theaboveequationscanbereplacedwiththefollowingequations:deltaIntSide=(invAbsAngTable[absAngIndex]*(l+1))>>8deltaFractSide=(invAbsAngTable[absAngIndex]*(l+1))%256whereinvAbsAngTable=[4096,1638,910,630,482,390,315,256].ByusingLUT,thereisnodivisionduringthecomputationoftheindexofthesidearray.However,therestillexiststheper-sampletesttodeterminethemainorthesidearrayforprediction.Tosimplifythisprocess,themainarrayisextendedbyprojectingsamplesfromthesidearrayontoitaccordingtothepredictiondirection,asshowninFigure.Duringtheprojection,thefractionalpartoftheinterceptisomittedandtheinterceptisroundedtothenearestinteger:deltaInteger=(invAbsAngTable[absAngIndex]*(l+1)+128)>>8.Atlast,thepredictionprocessonlyusestheextendedmainarrayandthesamesimplelinearinterpolationformulatopredictallsamplesinthePU.

如果經(jīng)過DC預(yù)測后，當前塊的左側(cè)和上側(cè)邊緣跟周圍的重構(gòu)像素可能會不大一致，HEVC對經(jīng)過DC模式預(yù)測后的變換單元的邊緣像素進行濾波以使變換塊的邊緣跟參考像素之間更加的平滑一些。濾波策略如圖從上圖中我們可以看到，左側(cè)和上側(cè)的灰綠色區(qū)域是已經(jīng)解碼的重構(gòu)像素，當前塊中被紅色標注的像素是當前塊與已重構(gòu)像素相鄰接的邊緣像素，需要對這些像素進行濾波處理。根據(jù)這些邊緣像素與已重構(gòu)的像素相鄰接的像素個數(shù)來決定是使用2-tap濾波器還是3-tap濾波器。其中左上角那個桃紅色像素因為和周圍兩個像素相鄰，所以使用3-tap濾波器；別的邊緣像素只跟周圍一個像素相鄰，所以只使用2-tap濾波器。HEVC對不同大小的變換單元使用不同強度的濾波器。對于4x4大小的變換單元使用FilterA所指的濾波器，對于8x8大小的變換單元使用FilterB所指的濾波器，對于16x16大小的變換單元使用FilterC所指的濾波器。3、InterPrediction3.1.MotionVectorPridictionForeachCUorPU,thebestmotionvectorpredictoriscomputedwiththeprocessspecifiedasfollows.Firstly,asetofmotionvectorpredictorcandidatesshall

be

normativeprocess,byreferringtomotionparametersofneighboringpartitions.Then,thebestonefromthecandidatesetisdeterminedbyacriterionthatselectsamotionvectorpredictorcandidatethatminimizesthecostJmotion.3.1.1.CodingModeIntheHMencoder,aninterCUcanbesegmentedintomultipleinterPUs.AninterCUcanbeencodedwithoneofthefollowingcodingmodes:MODE_SKIP,MODE_DIRECT,MODE_INTER,andMODE_MERGE.ForMODE_SKIP,MODE_DIRECTandMODE_MERGEcases,anysub-partitioningisnotallowed.Onthecontrary,fourtypesoffurtherpartitioningintheCUcanbeallowedforMODE_INTERcase.

TheHMencoderfirstevaluatesCUcodingcostswithskipanddirectmode.

Forthesecases,motionvectorpredictorforthecurrentCUisdirectlyusedascurrentmotionvector.ThemotionvectortobeusedforMODE_SKIPorMODE_DIRECTaredeterminedbycheckingallpossiblemotionvectorcandidatesandselectingthebestonethatminimizesthecostJmode.TheHMencodercomparesthecodingcostofMODE_SKIPandMODE_DIRECTfirst,takesthebettermode,andthenproceedtocheckingotherremainingvariations.3.1.2.Skip/DirectmodeIfthecodingcostJmodeissmallerthantheoneobtainedinSkip/Directmode,pred_modeissetto“MODE_MERGE”.Inthiscase,motionvectoroftheselectedleftorupperpartitionaredirectlyusedascurrentmotionparameters.Aftertheprocessspecifiedintheprevioussection,theHMencoderevaluateswhethermotionmergingshouldbeenabledornotforthecurrentCU.DecisionontheuseofmotionmergingisperformedonlyatCUlevelwhosePUsizeisSIZE_2Nx2N,whichisidenticaltoCUsize.Thedecisionontheuseofmotionmergingstartswithcheckingavailabilitiesofleftandupperneighboringpartitions.Ifthereisnoavailablepartitionformotionmerging,theHMencoderdoesnotchooseMODE_MERGEforthecurrentCU.Otherwise,thecodingcostJmodeiscomputedforavailablepartitions.3.1.3.MergemodeIftheprocesssuggestsfurtherevaluationofinterpredictionmodes,theHMencodercomputescodingcostsofCUpartitioningmodesSIZE_2Nx2NandSIZE_NxNofMODE_INTERcase.Then,theHMencoderfurtherevaluatesSIZE_2NxNandSIZE_Nx2Npartitions.Inthesecases,motionvectorsisderivedbymotionestimationprocessspecifiedinsection3.1.IfthecostJmodeofthebestPUpartitioningamongallevaluatedmodesissmallerthantheoneobtainedinMergemode,pred_modeissetto“MODE_INTER”.3.1.4.IntermodeLumainterpolationfilter1/4-sample,12-tapDCT-basedinterpolationfilter(DCT-IF)forhighefficiencyconfiguration(HE)1/4-sample,6-tapdirectionalinterpolationfilter(DIF)forlowcomplexityconfiguration(LC)Chromainterpolationfilter1/8-sample,bi-linearinterpolationfilterforbothHEandLC3.2.InterpolationfilterMotioncompensationisthekeyfactorforhighefficientvideocompressing,wherefractionalpelaccuracyrequiresinterpolationofthereferenceframe.InH.264/AVC,a6-tapfixedWienerfilterisfirstusedforhalfpelaccuracyinterpolationandthenabilinearcombinationofintegerandhalfpelvaluesisusedtoprovide?pelaccuracyinterpolation.Insteadofadaptiveinterpolationfilter.HMstilladoptsthefixed12-tapDCT-basedinterpolationfiltertoprovidefractionalpelaccuracyinterpolationbyreplacingthecombinationofWienerandbilinearfilterswithasetofinterpolationfiltersatthedesiredfractionalaccuracy.Morespecifically,onlyonefilteringprocedureisneededtoprovidetheinterpolationpixeltoanypixelaccuracy,insteadofacombinationof6-tapandbilinearfilteringproceduresinH.264/AVC.Thus,themotioncompensationprocesscanbesimplifiedintheimplementationpointofviewandthecomplexitycanalsobereducedforquarter-pelaccuracy.TheDCT-basedinterpolationfilteringprocessusesthehorizontalneighboringintegerpixelsI(x,0),wherex=-5…6,tointerpolatethehorizontalfractionalpixels,a,bandc,byusingthefollowingequations,a=\sum_{x=-5}^{6}I(x,y)*f(x,1/4)(1)b=\sum_{x=-5}^{6}I(x,y)*f(x,2/4)(2)c=\sum_{x=-5}^{6}I(x,y)*f(x,3/4)(3)

wherey=0.Intheaboveequantions,f(x,1/4),f(x,2/4)andf(x,3/4)denotethe12-tapDCT-basedinterpolationfiltercoefficientsinfractionalpositions1/4,2/4,and3/4,respectively,aslistedinTable

AsetofinterpolationfilterforlowcomplexityconfigurationisreferredtoastheDirectionalInterpolationFilter(DIF).Foreachofthethreehorizontalfractionalpositions,a,bandc,andthethreeverticalfractionalpositions,d,handn,whicharealignedwithfullpixelpositions,asingle6-tapfilterisused.ThefiltercoefficientsofDIFare{3,-15,111,37,-10,2}/128for?position(andmirroredfor?position),{3,-17,78,78,-17,3}/128for?position.

a=(3H-15I+111J+37K-10L+2M+64)>>7b=(3H-17I+78J+78K-17L+3M+64)>>7c=(2H-10I+37J+111K-15L+3M+64)>>7

d=(3B-15E+111J+37O-10S+2W+64)>>7h=(3B-17E+78J+78O-17S+3W+64)>>7

n=(2B-10E+37J+111O-15S+3W+64)>>7在高效預(yù)測模式下，HEVC仍然采用H.264中的等級B預(yù)測方式同時還增加了廣義B（GeneralizedPandBpicture,GPB）取代低時延場景下中的P的預(yù)測方式G。GPB預(yù)測結(jié)構(gòu)是指對傳統(tǒng)P幀采用類似于B幀的雙向預(yù)測方式。在這種預(yù)測模式下，前向和后向參考列表中的參考圖像都必須為當前圖像之前的圖像。對P幀采用B幀的預(yù)測方式增加了運動估計得準確度，提高編碼效率，同時也有利于編碼流程統(tǒng)一。 4、TransformandQuantization4.1.TransformTransformsofsizesfrom4x4to32x32aresupported.Fortransformsofsize4x4and8x8,thesamemethodasthatspecifiedintheH.264/AVC[6]standardisused.Fortransformsof16x16and32x32,fastintegertransformsusingabutterflystructureareused.4.2.ScalingandQuantizationThesamequantizationmethodasinH.264/AVCisused..經(jīng)過不同的預(yù)測之后，殘差存在不同的方向性，經(jīng)過量化變換后，不同預(yù)測方向產(chǎn)生的差異性仍然存在，因此如果使用一致的zig-zag掃描，并不是最佳的掃描方法，為了克服這個缺點，HEVC采用水平，垂直和zig-zag三種掃描方式。為了獲得最佳效果，HEVC對不同大小的變換單元和幀內(nèi)預(yù)測模式采用查表的方式獲得其掃描方式，并且因為亮度像素和色度像素的模式有很大的不同，所以對亮度像素和色度像素，采用不同的表格

TwoalternativeentropycodingschemesaresupportedintheHM:ContextAdaptiveBinaryArithmeticCoding(CABAC)andLowComplexityEntropyCoding(LCEC).Contextadaptivebinaryarithmeticcoding(CABAC)forhighefficiencyconfigurationLowcomplexityentropycoding(LCEC)phase2forlowcomplexityconfiguration5、EntropyCoding6、LoopFilteringThedeblockingmethodisbasedonthatusedinH.264/AVCRecallthatforblocksizesgreaterthan16×16,only16×16transformisused.Thus,for32×32and64×64blocks,de-blockingfilterisappliedonlyalongthe16×16blockedges.Thisreducesthecomputationalcomplexityforthede-blockingoperationifthebiggerblocksarechosenfrequently.4x4blockboundariesarenotdeblockedinordertoreducethecomplexity.6.1.Deblockingfilter自適應(yīng)樣點補償技術(shù)是一個自適應(yīng)選擇過程，在去塊濾波后進行。目前自適應(yīng)樣點補償方式分為帶狀補償（BandOffset,BO）和邊緣補償（EdgeOffset,EO）兩大類帶狀補償將像素值強度等級分為若干個條帶，每個條帶內(nèi)的像素擁有相同的補償值。進行補償時，根據(jù)重構(gòu)像素點所處的帶選擇相應(yīng)的帶狀補償值進行補償?，F(xiàn)在的HM模型將像素值強度從0到最大值劃分為32個等級。同時這32個等級條帶還分為兩類，第一類是位于中間的16個等級條帶，剩余的為第二類條帶。編碼時只將一類條帶的補償信息寫入片頭；另一類條帶信息則不傳送。這樣方式