ffmpeg-tutorial中文版page電影文件有很多基本組成部分首先本身被稱為容器Contai

指導(dǎo)1：制作屏概中的數(shù)據(jù)元素被稱為幀F(xiàn)rame。每個(gè)流是由不同的編來編碼生成的。編解做CODEC。Divx和MP3就是編器的例子。接著從流中被讀出來的叫做包中操作的原始幀的數(shù)據(jù)。根據(jù)我們的目的，每個(gè)含了完整的幀或者對(duì)于音1010流204050打開文#include<avcodec.h>#include<avcodec.h>intmain(intargc,charg{av_register_all()在主函數(shù)main()中來調(diào)用它。如果你喜歡，也可以只特定的格式和編AVFormatContextAVFormatContext//if(av_open_input_file(&pFormatCtx,argv[1],NULL,0,return-1;//Couldn'topen//Retrievestreaminformation//Retrievestreaminformationreturn-1;//Couldn'tfindstream////Dumpinformationaboutfileontostandarddump_format(pFormatCtx,0,argv[1],intintAVCodecContext//Findthefor(i=0;i<pFormatCtx->nb_streams;if(pFormatCtx->streams[i]->codec-){}Stream==-return-1;//Didn'tfind//Getapointertothecodeccontextfor流中關(guān)于編器的信息就是被我們叫做"codeccontext"（編器上下文）個(gè)指向他的指針。但是需要找到真正的編器并且打開它：AVCodecAVCodec//Findthedecoderfor//Findthedecoderforif(pCodec==NULL)fprintf(stderr,"Unsupportedcodec!\n");return-1;//Codecnotfound}//Opencodecreturn-1;//CouldnotopenCODEC_FLAG_TRUNCATEDpCodecCtx->flagshack我們移除了那些代碼后還有一個(gè)需要的不同點(diǎn)：pCodecCtx->time_base現(xiàn)在已經(jīng)保存了幀率的信息。time_base幀率（例如NTSC使用29.97fps）。保存數(shù)AVFrameAVFrame//////AllocateanAVFramestructurereturn-intnumBytes;//DeterminerequiredbuffersizeandallocatebufferpCodecCtx-buffer=(uint8_tav_mallocffmpegmallocmalloc證內(nèi)存地址是對(duì)齊的（42）。它并不能保護(hù)你不被內(nèi)存泄漏，重復(fù)釋放或者其它malloc的問題所困擾?，F(xiàn)在我們使用avpicture_fill來把幀和我們新申請(qǐng)的內(nèi)存來結(jié)合。關(guān)于構(gòu)體的開始部分與AVPicture結(jié)構(gòu)體是一樣的。////Assignappropriatepartsofbuffertoimageplanesin//NotethatpFrameRGBisanAVFrame,butAVFrameisa//ofavpicture_fill((AVPicture*)pFrameRGB,buffer,PIX_FMT_RGB24,pCodecCtx->width,pCodecCtx->height); 數(shù)intintAVPacketwhile(av_read_frame(pFormatCtx,&packet)>=0)//IsthisapacketfromStream)//(pCodecCtx,pFrame,packet.data,//Didwegetif(frameFinished)//ConverttheimagefromitsnativeformattoRGBimg_convert((AVPicture*)pFrameRGB,PIX_FMT_RGB24,(AVPicture*)pFrame,pCodecCtx-pCodecCtx->width,pCodecCtx-//SavetheframetopCodecCtx->height,i);}}//Freethepacketthatwasallocatedbyav_read_frame}當(dāng)我們得到下一幀的時(shí)候，avcodec_decode_()為我們?cè)O(shè)置了幀結(jié)束標(biāo)志frameFinished。最后，我們使用img_convert()函數(shù)來把幀從原始格式傳遞給我們的SaveFrame函數(shù)。關(guān)關(guān)于包Packets的注我們得到的包Packets包含的要么是完整的要么是多種完整的幀。 生成一個(gè)簡(jiǎn)單的PPM格式文件，請(qǐng)相信，它是可以工作的。voidvoidSaveFrame(AVFrame*pFrame,intwidth,intheight,int{FILEinty;////Opensprintf(szFilename,"frame%d.ppm",iFrame);pFile=fopen(szFilename,"wb");//Writefprintf(pFile,"P6\n%d%d\n255\n",width,//Writepixeldatafor(y=0;y<height;y++)fwrite(pFrame->data[0]+y*pFrame->linesize[0],1,width*3,//Closefile}#ff0000#ff0000....就表示了了個(gè)紅色的屏幕。（高度以及最大的RGB值的大小。//FreetheRGB//FreetheRGB//FreetheYUV//Closethe//Closereturn你會(huì)注意到我們使用av_freeavcode_alloc_framav_malloc-gcc-otutorial01tutorial01.c-lavutil-lavformat-lavcodec-lz-gcc-otutorial01tutorial01.c-lavutil-lavformat-lavcodec-lz- 指導(dǎo)2SDL為了在屏幕上顯示，使用SDL.SDL是SimpleDirectLayer的縮寫。它是方的上來得到這個(gè)庫的源代碼或者如果有可能的話你可以直接開發(fā)包到你的操作系統(tǒng)中。按照這個(gè)指導(dǎo)，你將需要編丟棄了，而且你可以每2Y1U1V）。SDLYUV一樣的，除了UV4204：2：01/4多流的格式已經(jīng)是YUV420P的了或者可以被很容易的轉(zhuǎn)換成YUV420P格式1SaveFrame()函數(shù)替換掉，讓它直接輸出我們的幀到屏幕上去。但一開始需要先看一下如何使用SDL庫。首先需先包含SDL庫的頭文件并且初始化它。#include#include#include|SDL_INIT_AUDIO|SDL_INIT_TIMER))fprintf(stderr,"CouldnotinitializeSDL-%s\n",}}創(chuàng)建一個(gè)叫做面surface。SDL_SurfaceSDL_Surfacescreen=Mode(pCodecCtx->width,pCodecCtx->height,0,if(!screen)fprintf(stderr,"SDL:couldnotmode-}示使用和當(dāng)前一樣的深度。（OSXbmp=SDL_CreateYUVOverlay(pCodecCtx->width,pCodecCtx-SDL_YV12_OVERLAY,顯示圖設(shè)置其數(shù)據(jù)指針和行尺寸來為我們的YUV覆蓋服務(wù)：{AVPicturepict.data[0]=bmp->pixels[0];pict.data[1]=bmp->pixels[2];pict.data[2]=bmp->pixels[1];pict.linesize[0]=bmp-pict.linesize[1]=bmp->pitches[2];pict.linesize[2]=bmp-//ConverttheimageintoYUVformatthatSDLimg_convert(&pict,(AVPicture*)pFrame,pCodecCtx->pix_fmt,pCodecCtx->width,pCodecCtx->height);}前面一樣我們使用img_convert來把格式轉(zhuǎn)換成PIX_FMT_YUV420P。繪制圖但我們?nèi)匀恍枰嬖VSDLSDLSDL為我們做縮SDL_RectSDL_Rectif(frameFinished)//ConverttheimageintoYUVformatthatSDLusesimg_convert(&pict,PIX_FMT_YUV420P,(AVPicture*)pFrame,pCodecCtx->pix_fmt,pCodecCtx->width,pCodecCtx->height);rect.x=0;rect.y=rect.w=pCodecCtx-rect.h=pCodecCtx->height;rect.h=pCodecCtx->height;}SDL{caseSDL_QUIT:}gccgcc-otutorial02tutorial02.c-lavutil-lavformat-lavcodec-lz-lm`sdl-config--cflags--這里的sdl-config命令會(huì)打印出用于gcc編譯的包含正確SDL庫的適當(dāng)參數(shù)。SDL文檔中來計(jì)算出我們什么時(shí)候需要顯示的幀。最后（在指導(dǎo)5），花足夠情要處理：音頻指導(dǎo)3：聲現(xiàn)在我們要來聲音。SDL也為我們準(zhǔn)備了輸出聲音的方法。函數(shù)特定的采樣率來進(jìn)行錄制，采樣率表示以多快的速度來這段樣本流，它的表示方式為每秒多少次采樣。例如22050和44100的采樣率就是電臺(tái)和CD常用果采樣是立體聲，那么每次的采樣數(shù)就為2個(gè)。當(dāng)我們從一個(gè)文件中等到數(shù)據(jù)的時(shí)候，我們不知道得到多少個(gè)樣本，但是ffmpeg將不會(huì)給我們部分的斷地調(diào)用這個(gè)回調(diào)函數(shù)并且要求它來向聲音緩沖填入一個(gè)特定的數(shù)量的字節(jié)。當(dāng)我們把這些信息放到SDL_AudioSpec結(jié)構(gòu)體中后，我們調(diào)用函數(shù)SDL_OpenAudio()就會(huì)打開聲音設(shè)備并且給我們送回另外一個(gè)AudioSpec結(jié)構(gòu)設(shè)置音////FindtheaudioStream=-for(i=0;i<pFormatCtx->nb_streams;i++)Stream<{}audioStream<0){}}Stream==-return-1;//Didn'tfindreturn- AVCodecContextAVCodecContextaCodecCtx=pFormatCtx->streams[audioStream]-wanted_spec.freq=aCodecCtx->sample_rate;wanted_spec.format=AUDIO_S16SYS;wanted_spec.channels=aCodecCtx->channels;wanted_spec.silence=0;wanted_spec.freq=aCodecCtx->sample_rate;wanted_spec.format=AUDIO_S16SYS;wanted_spec.channels=aCodecCtx->channels;wanted_spec.silence=0;wanted_spec.samples=SDL_AUDIO_BUFFER_SIZE;wanted_spec.callback=audio_callback;wanted_spec.userdata=aCodecCtx;if(SDL_OpenAudio(&wanted_spec,&spec)<{fprintf(stderr,"SDL_OpenAudio:%s\n",SDL_GetError());return-1;}·freq·format告訴SDL要給的格式。在“S16SYS”中的S表示有符號(hào)的·channels區(qū)的尺寸。一個(gè)比較合適的值在5128192之間；ffplay1024。aCodec=avcodec_find_decoder(aCodecCtx-if(!aCodec)fprintf(stderr,"Unsupportedcodec!\n");return-1;}avcodec_open(aCodecCtx,隊(duì)方法為創(chuàng)建一個(gè)全局的結(jié)構(gòu)體變量以便于我們從文件中得到包有地方存據(jù)。所以我們要做的是創(chuàng)建一個(gè)包的隊(duì)列queue。在ffmpeg中有一個(gè)叫typedefstructtypedefstruct{AVPacketList*last_pkt;intnb_packets;intsize;SDL_cond*cond;}首先，我們應(yīng)當(dāng)nb_packets是與size不一樣的－－size表示我們從理的。如果我們沒有正確的鎖定這個(gè)隊(duì)列，我們有可能把數(shù)據(jù)。來列，但是把這部分也來討論從而可以學(xué)習(xí)到SDL的函數(shù)。voidvoidpacket_queue_init(PacketQueue{memset(q,0,sizeof(PacketQueue));q->mutex=SDL_CreateMutex();q->cond=}intpacket_queue_put(PacketQueue*q,AVPacket*pkt)intpacket_queue_put(PacketQueue*q,AVPacket*pkt)AVPacketList*pkt1;if(av_dup_packet(pkt)<0){return-}pkt1=av_malloc(sizeof(AVPacketList));if(!pkt1)return-pkt1->pkt=*pkt;pkt1->next=SDL_LockMutex(q-if(!q-q->first_pkt=pkt1;q->last_pkt->next=pkt1;q->last_pkt=pkt1;q-q->size+=pkt1->pkt.size;return0;}SDL_CondSignal()通過我們的條件變量為一個(gè)接收函數(shù)（如果它在等待）數(shù)阻塞block的（例如一直等到隊(duì)列中有數(shù)據(jù)）。intquit=intquit=staticintpacket_queue_get(PacketQueue*q,AVPacket*pkt,int{AVPacketList*pkt1;intret;SDL_LockMutex(q-for(;;){ret=-1;}pkt1=q->first_pkt;if(pkt1){q->first_pkt=pkt1->next;if(!q->first_pkt)q->last_pkt=q->nb_packets--q->nb_packets--q->size-=pkt1-*pkt=pkt1->pkt;ret=1;}elseif{ret=0;}elseSDL_CondWait(q->cond,q-}}returnret;}基本上，所有的CondWait只等待從SDL_CondSignal()函數(shù)（或者，SDL_CondWait()函數(shù)也為我們做了意外情intint{return}main(){caseSDL_QUIT:quit=1;必需要設(shè)置quit標(biāo)志為1。為隊(duì)列提PacketQueuePacketQueuemain()main()avcodec_open(aCodecCtx,據(jù)，它會(huì)靜音。while(av_read_frame(pFormatCtx,&packet)>=0)while(av_read_frame(pFormatCtx,&packet)>=0)//IsthisapacketfromStream)//}}else{packet_queue_put(&audioq,}{} 在后來釋放它取出audio_callbackvoidcallback(void*userdata,Uint8*stream,intlen)，這里的userdata就是我們給到SDL的指針，stream是我們要把聲音數(shù)據(jù)寫入的緩沖區(qū)指針，len是緩沖區(qū)的大小。下面就是代碼：voidaudio_callback(void*userdata,Uint8*stream,intlen)voidaudio_callback(void*userdata,Uint8*stream,intlen)AVCodecContext*aCodecCtx=(AVCodecContext*)userdata;intlen1,audio_size;staticuint8_taudio_buf[(AVCODEC_MAX_AUDIO_FRAME_SIZE*3)/2];staticunsignedintaudio_buf_size=0;staticunsignedintaudio_buf_index=while(len>0)if(audio_buf_index>=audio_buf_size)audio_size=audio_decode_frame(aCodecCtx,if(audio_size<0)audio_buf_size=memset(audio_buf,0,}else}elseaudio_buf_size=}audio_buf_index=}len1=audio_buf_size-audio_buf_index;if(len1>len)len1=memcpy(stream,(uint8_t*)audio_buf+audio_buf_index,len1);len-=len1;stream+=len1;audio_buf_index+=len1;}}這基本上是一個(gè)簡(jiǎn)單的從另外一個(gè)要寫的audio_decode_frame()函數(shù)中l(wèi)en字節(jié)并且在我們沒有足夠的數(shù)據(jù)的時(shí)候會(huì)獲取的數(shù)據(jù)或者當(dāng)我們有多余數(shù)小以便于有一個(gè)比較好的緩沖，這個(gè)聲音幀的大小是ffmpeg給出的。最后音讓我們看一下器的真正部分intintaudio_decode_frame(AVCodecContext*aCodecCtx,uint8_tintbuf_size)staticAVPacketstaticuint8_t*audio_pkt_data=NULL;staticintaudio_pkt_size=0;intlen1,{while(audio_pkt_size>{data_size=len1=avcodec_decode_audio2(aCodecCtx,(int16_t*)audio_buf,if(len1<0){audio_pkt_size=0;}audio_pkt_data+=len1;audio_pkt_size-=len1;if(data_size<=0){}}return}{return-}if(packet_queue_get(&audioq,&pkt,1)<0)return-}audio_pkt_data=pkt.data;audio_pkt_size=}整個(gè)過程實(shí)際上從函數(shù)的尾部開始，在這里我們調(diào)用了packet_queue_get()函數(shù)。我們從隊(duì)列中取出包，并且保存它的信息。然后，一旦我們有了可以使用幀，所以你可能要調(diào)用很多次來出包中所有的數(shù)據(jù)。同時(shí)也要記住進(jìn)行指 audio_callback函數(shù)從隊(duì)列中 gccgcc-otutorial03tutorial03.c-lavutil-lavformat-lavcodec-lz-lm`sdl-config--cflags--中，但是聲音設(shè)備卻會(huì)按照原來指定的采樣率來進(jìn)行。得程序更加更加易于控制和模塊化。在我們開始同步音之前，我們需要讓ffmpegtutorial4：創(chuàng)建SpawningLasttimeweaddedaudiosupportbytakingadvantageofSDL'saudiofunctions.SDLstartedathreadthatmadecallbackstoafunctionwedefinedeverytimeitneededaudio.Nowwe'regoingtodothesamesortofthingwiththedisplay.Thismakesthecodemoremodularandeasiertoworkwith-especiallywhenwewanttoaddsyncing.Sowheredowestart?Firstwenoticethatourmainfunctionishandlinganawfullot:it'sSowhatwe'regoingtodoissplitallthoseapart:we'regoingtohavewillthenbeaddedtothequeueandreadbythecorrespondingaudioandthreads.Theaudiothreadwehavealreadysetupthewaywewantit;thethreadwillbealittlemorecomplicatedsincewehavetodisplaytheourselves.Wewilladdtheactualdisplaycodetothemainloop.Butinsteadofjustdisplayingeverytimeweloop,willintegratethedisplayintotheeventloop.Theideaistodecodethe,savetheresultingframeinanotherqueue,thencreateacustomevent(FF_REFRESH_EVENT)thatweaddtotheeventsystem,thenwhenoureventloopseesthisevent,itwilldisplaythenextframeinthequeue.Here'sahandyASCIIartillustrationofwhatisgoingon: |pkts |to|DECODE|----->|AUDIO|--->|SDL|-- | +---------->| |EVENT +------>||to|LOOP|----------------->|DISP.|-- ThemainpurposeofmovingcontrollingthedisplayviatheeventloopisthatusinganSDLDelaythread,wecancontrolexactlywhenthenextframeshowsuponthescreen.Whenwefinallysynctheinthenexttutorial,itwillbeasimplemattertoaddthecodethatwillschedulethenextrefreshsotherightpictureisbeingshownonthescreenattherighttime.SimplifyingWe'realsogoingtocleanupthecodeabit.Wehaveallthisaudioandcodecinformation,andwe'regoingtobeaddingqueuesandbuffersandwhoknowswhatelse.Allthisstuffisforonelogicalunit,viz.themovie.Sowe'regoingtomakealargestructthatwillholdallthatinformationcalledtheState.typedefstruct{AVFormatContext Stream,audioStream; audio_buf[(AVCODEC_MAX_AUDIO_FRAME_SIZE*3)/2];unsignedint unsignedint q; pictq_size,pictq_rindex,SDL SDL SDL * }Hereweseeaglimpseofwhatwe'regoingtogetto.Firstweseethebasicinformation-theformatcontextandtheindicesoftheaudioandstream,andthecorrespondingAVStreamobjects.Thenwecanseethatwe'vemovedsomeofthoseaudiobuffersintothisstructure.These(audio_buf,audio_buf_size,etc.)wereallforinformationaboutaudiothatwasstilllyingaround(orthelackthereof).We'veaddedanotherqueueforthe,andabuffer(whichwillbeusedasaqueue;wedon'tneedanyfancyqueueingstuffforthis)forthedecodedframes(savedasanoverlay).ThePicturestructisofourowncreations(we'llseewhat'sinitwhenwecometoit).Wealsonoticethatwe'veallocatedpointersforthetwoextrathreadswewillcreate,andthequitflagandthefilenameofthemovie.Sonowwetakeitallthewaybacktothemainfunctiontoseehowthischangesourprogram.Let'ssetupourStatestruct:intmain(intargc,char{SDLEvent is=avmallocz(sizeof(avmallocz()isanicefunctionthatwillallocatememoryforusandzeroitout.Thenwe'llinitializeourlocksforthedisplaybuffer(pictq),becausesincetheeventloopcallsourdisplayfunction-thedisplayfunction,remember,willbepullingpre-decodedframesfrompictq.Atthesametime,willgettherefirst.Hopefullyyourecognizethatthisisaclassicracecondition.Soweallocateitnowbeforewestartanythreads.Let'salsocopythefilenameofourmovieintoourState.pstrcpy(is->filename,sizeof(is->filename),argv[1]);is->pictq_mutex=SDLCreateMutex();is->pictq_cond=SDLpstrcpyisafunctionfromffmpegthatdoessomeextraboundscheckingbeyondstrncpy.OurFirstNowlet'sfinallylaunchourthreadsandgettherealworkdone:schedule_refresh(is,40);is->parse_tid=SDL_CreateThread(decode_thread,is);if(!is->parse_tid){return-1;}schedule_refreshisafunctionwewilldefinelater.WhatitbasicallydoesislthesystemtopushaFF_REFRESH_EVENTafterthespecifiedwhenweseeitintheeventqueue.Butfornow,let'slookatSDLSDL_CreateThread()doesjustthat-itspawnsanewthreadthathascompleteaccesstoallthememoryoftheoriginalprocess,andstartsthethreadrunningonthefunctionwegiveit.Itwillalsopassthatfunctionuser-defineddata.Inthiscase,we'recallingdecode_thread()andwithourStatestructattached.Thefirsthalfofthefunctionhasnothingnew;itsimplydoestheworkofopeningthefileandfindingtheindexoftheaudioandstreams.Theonlythingwedodifferentissavewecallanotherfunctionthatwewilldefine, Thisisaprettynaturalwaytosplitthingsup,andsincewedoalotofsimilarthingstosetuptheandaudiocodec,wereusesomecodebymakingthisafunction. ponent_open()functioniswherewewillfindourcodecstruct,andlaunchouraudioandthreads.Thisiswherewealsoinsertotheroptions,suchasforcingthecodecinsteadofautodetectingitandsoforth.Hereitis: ponent_open(State*is,intstream_index)AVFormatContext*pFormatCtx=is->pFormatCtx;AVCodecContext*codecCtx;AVCodecSDLAudioSpecwanted_spec,if(stream_index<0||stream_index>=pFormatCtx-{return-}//GetapointertothecodeccontextforthestreamcodecCtx=pFormatCtx->streams[stream_index]->codec;if(codecCtx->codec_type==CODEC_TYPE_AUDIO)//Setaudiosettingsfromcodecinfowanted_spec.freq=codecCtx->sample_rate;/*....wanted_spec.callback=audio_callback;wanted_spec.userdata=is;if(SDLOpenAudio(&wanted_spec,&spec)<{fprintf(stderr,"SDL_OpenAudio:%s\n",SDL_GetError());return-1;}}codec=avcodecfinddecoder(codecCtx->codec_id);if(!codec||(avcodec_open(codecCtx,codec)<0)){fprintf(stderr,"Unsupportedcodec!\n");return-1;}switch(codecCtx-{caseis->audioStream=is->audio_st=pFormatCtx->streams[stream_index];is->audio_buf_size=0;is->audio_buf_index=memset(&is->audio_pkt,0,sizeof(is->audio_pkt));SDLcaseCODEC_TYPE_is->Stream=is->_st=pFormatCtx-packet_queue_init(&is->is->_tid=SDLCreateThread(_thread,is);}}Thisisprettymuchthesameasthecodewehadbefore,exceptnowit'sgeneralizedforaudioand.NoticethatinsteadofaCodecCtx,we'vesetupourbigstructastheuserdataforouraudiocallback.We'vealsosavedthestreamsthemselvesasaudio_stand_st.Wealsohaveaddedourqueueandsetitupinthesamewaywesetupouraudioqueue.Mostofthepointistolaunchtheandaudiothreads.Thesebitsdoit:/*is->_tid=SDL_CreateThread(_thread,WerememberSDLPauseAudio()fromlasttime,andSDLCreateThread()isBeforethat,let'sgobacktothesecondhalfofourdecode_thread()function.It'sbasicallyjustaforloopthatwillreadinapacketandputitontherightqueue:for(;;)if(is-{}//seekstuffgoesif(is->audioq.size>MAX_AUDIOQ_SIZE||is-> MAX_{SDL}if(avreadframe(is->pFormatCtx,packet)<0)if(urlferror(&pFormatCtx->pb)==0)SDLDelay(100);/*noerror;waitforuserinput*/}{}}//Isthisapacketfromthestream?if(packet->stream_index==is->Stream){packet_queue_put(&is->q,}elseif(packet->stream_index==is-{packet_queue_put(&is->audioq,}elseavfree}}一個(gè)檢測(cè)讀錯(cuò)誤的函數(shù)。格式上下文里面有一個(gè)叫做pb的ByteIOContext類型結(jié)構(gòu)體。這個(gè)結(jié)構(gòu)體是用來保存一些低級(jí)的文件信息。函數(shù)url_ferror用來檢測(cè)結(jié)構(gòu)體并發(fā)現(xiàn)是否有些文件錯(cuò)誤。代碼是有益的，因?yàn)樗甘境隽巳绾悟?qū)動(dòng)事件－－后面顯示影像。while(!is-while(!is-{}SDL_Eventevent;event.type=FF_QUIT_EVENT;event.user.data1=is;}returnreturnSDL_QUIT_EVENT部分一樣。在自己的事件隊(duì)列中詳細(xì)討論，現(xiàn)在只是確的退出標(biāo)志quit。得到幀： 它成幀，然后調(diào)用queue_picture函數(shù)把處理好的幀放入到 _thread(void{State*is=StateAVPacketpkt1,*packet=intlen1,frameFinished;AVFrame*pFrame;pFrame=for(;;)if(packet_queue_get(&is-q,packet,1)<0)//meanswequitgetting}//len1=(is-_st->codec,packet->data,packet-//Didweget//Didweget{if(queue_picture(is,pFrame)<{}}}return0;}在這里的很多函數(shù)應(yīng)該很熟悉吧。我們把a(bǔ)vcodec_decode_函數(shù)移到了這把幀隊(duì)列讓我們看一下保存后的幀pFrame到圖像隊(duì)列中去的函數(shù)。因?yàn)槲覀兊膱D像隊(duì)列是SDL的覆蓋的集合（基本上不用讓顯示函數(shù)再做計(jì)算了），我們需要typedeftypedef{SDL_Overlayintwidth,height;intallocated;}置來保存我們的Picture。然后我們檢查看我們是否已經(jīng)申請(qǐng)到了一個(gè)可還不太清楚原因；我相信是為了避免在其它線程中調(diào)用SDL）。intintState*is,AVFrame{PictureintAVPicturepict;while(is->pictq_size_PICTURE_QUEUE_SIZE!is->quit)SDL_CondWait(is->pictq_cond,is-}return-//windexissetto0vp=&is->pictq[is->pictq_windex];if(!vp->bmp||vp->width!=is-_st->codec->widthvp->height!=is-_st->codec->height)SDL_Eventevent;vp->allocatedSDL_Eventevent;vp->allocated=0;event.type=FF_ALLOC_EVENT;event.user.data1=is;while(!vp->allocated&&!is->quit){SDL_CondWait(is->pictq_cond,is-}if(is->quit){return-}}{);{casevoidalloc_picture(voidvoidalloc_picture(void{State*is=StatePicturevp=&is->pictq[is->pictq_windex];if(vp->bmp){//wealreadyhaveonemakeanother,bigger/smaller}//AllocateaplacetoputourYUVimageonthatvp->bmp=SDL_CreateYUVOverlay(is-_st->codec-vp->width=is-_st->codec-vp->height=is-_st->codec-vp->allocated=1;}你可以看到我們把SDL_CreateYUVOverlay函數(shù)從主循環(huán)中移到了這里。這段代碼應(yīng)該完全可以自我注釋。記住我們把高度和寬度保存到Picture結(jié)構(gòu)體中因?yàn)槲覀冃枰４嫖覀兊牡拇笮]有因?yàn)槟承┰蚨淖?。{if(vp->bmp)dst_pix_fmt=PIX_FMT_YUV420P;pict.data[0]=vp->bmp->pixels[0];pict.data[1]=vp->bmp->pixels[2];pict.data[2]=vp->bmp->pixels[1];pict.linesize[0]={if(vp->bmp)dst_pix_fmt=PIX_FMT_YUV420P;pict.data[0]=vp->bmp->pixels[0];pict.data[1]=vp->bmp->pixels[2];pict.data[2]=vp->bmp->pixels[1];pict.linesize[0]=vp->bmp->pitches[0];pict.linesize[1]=vp->bmp->pitches[2];pict.linesize[2]=vp->bmp-//ConverttheimageintoYUVformatthatSDLimg_convert(&pict,(AVPicture*)pFrame,is-_st->codec-is-_st->codec->width,is-_st->codec-if(++is->pictq_windex {is->pictq_windex=}int State*is,AVFrame}}}}return}一點(diǎn)只是簡(jiǎn)單的給隊(duì)列加1。這個(gè)隊(duì)列在寫的時(shí)候會(huì)一直寫入到滿為止，在讀顯用schedule_refresh()函數(shù)嗎？讓我們看一下實(shí)際中是如何做的：staticstaticvoidState*is,int{SDL_AddTimer(delay,sdl_refresh_timer_cb,}把一幀從圖像隊(duì)列中顯示到屏幕上。staticstaticUint32sdl_refresh_timer_cb(Uint32interval,void{SDL_Eventevent.type=FF_REFRESH_EVENT;event.user.data1=opaque;return_refresh_timer(void{State*is=State_refresh_timer(void{State*is=StatePictureif(is-_st)if(is->pictq_size=={schedule_refresh(is,}elsevp=&is->pictq[is->pictq_rindex];schedule_refresh(is,80);if(++is->pictq_rindex{is->pictq_rindex={);{case}}{schedule_refresh(is,}下一幀設(shè)置定時(shí)器，調(diào)用_display函數(shù)來真正顯示圖像到屏幕上，然后中我們并沒有真正對(duì)vp做一些實(shí)際的動(dòng)作，原因是這樣的：在后面處理。注釋信息“timinghere”。那里討論什么時(shí)候顯示下一幀視頻，然后把相應(yīng)的值寫入到schedule_refresh()函數(shù)中?，F(xiàn)在我們只是隨便寫入一個(gè)值80。從技術(shù)上來講，你可以猜測(cè)并驗(yàn)證這個(gè)值，并且為每個(gè)重新編譯State{SDL_RectPicture*vp;AVPicturepict;intw,h,x,y;intvp=&is->pictq[is->pictq_rindex];if(vp->bmp){if(is- _st->codec->sample_aspect_ratio.num=={aspect_ratio=}elseaspect_ratio=av_q2d(is-> _st->codec->sample_aspect_ratio)* _st->codec->width/is-> }if(aspect_ratio<=0.0)aspect_ratio=(float)is-> _st->codec->width/ }h=screen-w=((int)rint(h*aspect_ratio))&-3;if(w>screen->w){w=screen-h=((int)rint(w/aspect_ratio))&-}x=(screen->w-w)/2;y=(screen->h-h)/2;rect.x=x;rect.y=y;rect.w=rect.h=rect.h=SDL_DisplayYUVOverlay(vp->bmp,}} 比aspectratio，表示方式為寬度除以高度。某些編器 例。因?yàn)閷挾群透叨仍谖覀兊木幤髦惺怯孟袼貫閱挝坏模詫?shí)際的 比為1x1。然后我們把縮放到適合屏幕的盡可能大的尺寸。這到中心位置，接著調(diào)用SDL_DisplayYUVOverlay()函數(shù)。StateStateintdecode_interrupt_cb(void)return_state&&_state-}gccgcc-otutorial04tutorial04.c-lavutil-lavformat-lavcodec-lz-lm`sdl-config--cflags--指導(dǎo)5：同如何同頻，也能音頻，但是它還不能被稱為一部。那么我們還要做什么呢？PTS幀和乘以幀率的方式來同步，那么就很有可能會(huì)失去同步。于是作為一種了這兩個(gè)參數(shù)，你需要了解存放的方式。像MPEG等格式，使用被叫做B幀（Bbidrectional）IP（I關(guān)鍵幀，P）。IPavcodec_decode_以后會(huì)得不到一幀圖像。所以對(duì)于一個(gè)，幀是這樣來顯示的：IBBP?，F(xiàn)在我們需要在顯示B幀之前知道P幀中的信息。因此，幀可能會(huì)按照這樣的方式來：IPBB。這就是為什么我們會(huì)有一個(gè)時(shí)間戳和一個(gè)顯示時(shí)間戳的原因。時(shí)間戳告訴我們什PTS:142PTS:142DTS:123Stream:IPB包中。但是我們真正想要的PTS是我們剛剛出來的原始幀的PTS，這樣我們才能知道什么時(shí)候來顯示它。然而，我們從avcodec_decode_()函數(shù)中得到的幀只是一個(gè)AVFrame，其中并沒有包含有用的PTS值（注意：AVFrameffmpeg重新排序包以便于被avcodec_decode_()函數(shù)處理的包的DTS可以PTS相同。但是，另外的一個(gè)警告是：我們也并不是總能得到這個(gè)以通過函數(shù)avcodec_decode_()來計(jì)算出哪個(gè)包是一幀的第一個(gè)包。怎樣實(shí)現(xiàn)呢？任何時(shí)候當(dāng)一個(gè)包開始一幀的時(shí)候，avcodec_decode_()將調(diào)用一個(gè)函數(shù)來為一幀申請(qǐng)一個(gè)緩沖。當(dāng)然，ffmpeg存的函數(shù)。所以我們制作了一個(gè)新的函數(shù)來保存一個(gè)包的時(shí)間戳。同PTS首先，要知道下一個(gè)PTS是什么?，F(xiàn)在我們能添加速率到我們的PTS中－－文件也不是完好的。所以，我們有三種選擇：同步音頻到，同步步到音頻。寫代碼：獲得幀的時(shí)現(xiàn)在讓我們到代碼中來做這些事情。需要為我們的大結(jié)構(gòu)體添加一些成們得到了 次處理的包中得到DTS，這是很容易的：doubledoublefor(;;)if(packet_queue_get(&is-q,packet,1)<0)//meanswequitgetting}pts=//len1=(is-_st-pFrame,packet->data,packet->size);if(packet->dts!=AV_NOPTS_VALUE){pts=packet-}{pts=}intget_buffer(structAVCodecContext*c,AVFrameintget_buffer(structAVCodecContext*c,AVFramevoidrelease_buffer(structAVCodecContext*c,AVFramePTS保存到一個(gè)全局變量中去。我們自己以讀到它。然后，我們把值保存到AVFrame結(jié)構(gòu)體難理解的變量中去。所以一開始，這就是我們的函數(shù)：uint64_tuint64_t_pkt_pts=intour_get_buffer(structAVCodecContext*c,AVFrame{intret=avcodec_default_get_buffer(c,pic);uint64_t*pts=av_malloc(sizeof(uint64_t));*pts=pic->opaque=return}voidour_release_buffer(structAVCodecContext*c,AVFrame{if(pic)av_freep(&pic->opaque);avcodec_default_release_buffer(c,pic);}avcodec_default_get_bufferavcodec_default_release_buffer把內(nèi)存釋放而且把指針設(shè)置為NULL?，F(xiàn)在到了我們流打開的函數(shù)（ponent_open），我們添加這幾行來告訴ffmpeg如何去做：codecCtx->get_buffer=codecCtx->get_buffer=codecCtx->release_buffer=for(;;)if(packet_queue_get(&is- q,packet,1)<0)//meanswequitgettingpackets}pts=//SaveglobalptstobestoredinpFrameinfirstcall _pkt_pts=packet->pts;// len1=avcodec_decode_ _st->codec,pFrame,if(packet->dts==AV_NOPTS_VALUE&&pFrame->opaque&&*(uint64_t*)pFrame->opaque!=AV_NOPTS_VALUE){pts=*(uint64_t*)pFrame-}elseif(packet->dts!={pts=packet-}{pts=}型來保存的。這個(gè)值是一個(gè)時(shí)間戳相當(dāng)于時(shí)間的度量，用來以流的time_base4224（）。我們可以通過除以幀率來把這個(gè)值轉(zhuǎn)化為秒。流中的time_base值表示乘以time_base。寫代碼：使用PTS來同PTS率。我們可以使用內(nèi)部的反映當(dāng)前已經(jīng)時(shí)間的時(shí)鐘_clock來完typedeftypedefState_clock;{(doubleif(pts!=0)is-_clock=}elsepts=is-pts=is-}frame_delay=av_q2d(is-_st->codec-frame_delay+=src_frame->repeat_pict*(frame_delay*is-_clock+=return}時(shí)間戳參數(shù)pts：////Didwegetif(frameFinished)pts=(is,pFrame,if(queue_picture(is,pFrame,pts)<0)}}typedeftypedef Picturedouble}intState*is,AVFrame*pFrame,doublepts)...stuffif(vp->bmp)...convertpicture...vp->pts=pts;...alertqueue}戳的時(shí)間。同時(shí)，我們需要同步到音頻。設(shè)置一個(gè)音頻時(shí)間audioget_audio_clock。一旦我們有了這個(gè)值，我們?cè)谝纛l和視頻失去同步的時(shí)候應(yīng)該做些什么呢？簡(jiǎn)單而有點(diǎn)笨的辦法是試著用跳過正確幀或者其它的快的刷新。既然我們有了調(diào)整過的時(shí)間和延遲，把它和我們通過影中所有的延時(shí)。換句話說，這個(gè)frame_timer就是指我們什么時(shí)候來顯后使用那個(gè)值來調(diào)度下一次刷 _refresh_timer(void*userdata)State*is=( State*)userdata;Picture*vp;doubleactual_delay,delay,sync_threshold,ref_clock,if(is- _st)if(is->pictq_size=={schedule_refresh(is,}elsevp=&is->pictq[is-delay=vp->pts-is->frame_last_pts;if(delay<=0||delay>=1.0){delay=is-}is->frame_last_delay=delay;is->frame_last_pts=vp-ref_clock=get_audio_clock(is);diff=vp->pts-ref_clock;sync_threshold=(delay>AV_SYNC_THRESHOLD)?delay:if(fabs(diff)<{if(diff<=-sync_threshold){delay=0;}elseif(diff>={delay=2*}}is->frame_timer+=actual_delay=is->frame_timer-(av_gettime()/ if(actual_delay<0.010){actual_delay=}schedule_refresh(is,(int)(actual_delay*1000+if(++is->pictq_rindexif(++is->pictq_rindex{is->pictq_rindex=}}}{schedule_refresh(is,}ffplay最小的刷新值設(shè)置為10毫秒。 ponent_open中初始化幀時(shí)間frame_timer和前面的幀延遲framedelay：is->frame_timer=(double)av_gettime()is->frame_timer=(double)av_gettime()is->frame_last_delay=40e-同步：聲現(xiàn)在讓我們看一下怎樣來得到聲音時(shí)鐘。我們可以在聲音函數(shù)audio_decode_frameif(pkt->pts!=AV_NOPTS_VALUE)if(pkt->pts!=AV_NOPTS_VALUE)is->audio_clock=av_q2d(is->audio_st->time_base)*pkt-}pts=is-pts=is-*pts_ptr=n=2*is->audio_st->codec->channels;is->audio_clock+=(double)data_size(double)(n*is->audio_st->codec-is->audio_clock戳，但是如果你看了audio_callback函數(shù)，它花費(fèi)了時(shí)間來把數(shù)據(jù)從聲音包中移到我們的輸出緩沖區(qū)中。這意味著我們聲音時(shí)記錄的時(shí)間比實(shí)際的要早太多。所以須要檢查一下我們還有多少?zèng)]有寫入。下面是完整的代碼：doubledoubleState{doubleinthw_buf_size,bytes_per_sec,pts=is-pts=is-hw_buf_size=is->audio_buf_size-is->audio_buf_index;bytes_per_sec=0;n=is->audio_st->codec->channels*2;if(is->audio_st){bytes_per_sec=is->audio_st->codec->sample_rate*}if(bytes_per_sec)pts-=(double)hw_buf_size/}return}gcc-otutorial05tutorial05.c-lavutil-lavformat-lavcodec-gcc-otutorial05tutorial05.c-lavutil-lavformat-lavcodec--lm`sdl-config--cflags--6：同步音同步音的同步方式。采用和一樣的方式：做一個(gè)內(nèi)部時(shí)鐘來記錄音頻和都同步到外部時(shí)鐘。生成一個(gè)時(shí)樣簡(jiǎn)單。但是，了幀之間的時(shí)間間隔是很長(zhǎng)的，以毫秒為計(jì)量的。前時(shí)間值就是PTS_of_last_frame+(current_time-get_audio_clock中的方式很類似。和一個(gè)64位寬整型變量_current_pts_time。時(shí)鐘更新將被放在_refresh_timer函數(shù)中。_refresh_timer(void*userdata)if(is-_st)if(is->pictq_size=={schedule_refresh(is,}elsevp=&is->pictq[is-is-is- _current_pts_time= is-is- _current_pts_time=doubledoubleState{doubledelta=(av_gettime()-is-_current_pts_time)returnis-_current_pts+}提取時(shí)但是為什么要強(qiáng)制使用時(shí)鐘呢？我們更改同步代碼以致于音頻和get_external_clock：enumenum#defineDEFAULT_AV_SYNC_TYPE#defineDEFAULT_AV_SYNC_TYPEdoubleState*is)if(is->av_sync_type=={return}elseif(is->av_sync_type=={return}elsereturn}}main()is->av_sync_type=}同步音通過丟棄樣本的方式來加速還是需要通過插值樣本的方式來放慢？synchronize_audio步，因?yàn)檫@樣會(huì)使同步音頻多于包。所以我們?yōu)楹瘮?shù)synchronize_audio所以使用一個(gè)分?jǐn)?shù)系數(shù)，叫c，所以現(xiàn)在可以說我們得到了N個(gè)失去同步的長(zhǎng)度的均值。例如，第一次調(diào)用的時(shí)候，顯示出來我們失去同步的長(zhǎng)度為40ms，50ms最近的值比靠前的值要重要的多。所以使用一個(gè)分?jǐn)?shù)系統(tǒng)，叫c，然后用這樣的公式來計(jì)算差異：diff_sumnew_diffdiff_sum*c。當(dāng)我們準(zhǔn)備好去找平均差異的時(shí)候，我們用簡(jiǎn)單的計(jì)算方式：avg_diffdiff_sum*(1-c)intintState*is,shortintsamples_size,doublepts)intdoublen=2*is->audio_st->codec-if(is->av_sync_type!={doublediff,intwanted_size,min_size,max_size,ref_clock=diff=get_audio_clock(is)-if(diff<AV_NOSYNC_THRESHOLD)//accumulatetheis->audio_diff_cum=diff+is-*is-if(is->audio_diff_avg_count<{is-}elseavg_diff=is->audio_diff_cum*(1.0-is-}}elseis->audio_diff_avg_count=is->audio_diff_cum=}}return}“Shrinking/expandingbuffercode”部分來寫上代碼：if(fabs(avg_diff)if(fabs(avg_diff)>=is-{wanted_size=samples_size((int)(diff*is->audio_st->codec->sample_rate)*min_size=samples_size*((100-/max_size=samples_size*((100+/if(wanted_size<{wanted_size=}elseif(wanted_size>{wanted_size=}記住audio_length*(sample_rate*#ofchannels*2)就是audio_length 因?yàn)槿绻覀兏淖兊奶?，用戶?huì)聽到刺耳。修正樣本但是如果我們想讓它變大，我們不能只是讓樣本大小變大，因?yàn)樵诰彌_區(qū)中沒if(wanted_size<samples_size)if(wanted_size<samples_size)samples_size=}elseif(wanted_size>{uint8_t*samples_end,*q;intnb;nb=(samples_size-samples_end=(uint8_t*)samples+samples_size-n;q=samples_end+n;while(nb>0)memcpy(q,samples_end,n);q+=n;nb-=}samples_size=}voidaudio_callback(void*userdata,Uint8*stream,intlen)voidaudio_callback(void*userdata,Uint8*stream,intlen)State*is=Stateintlen1,doubledoublewhile(len>0)if(is->audio_buf_index>=is->audio_buf_size)audio_size=audio_decode_frame(is,is->audio_buf,sizeof(is->audio_buf),&pts);if(audio_size<0)is->audio_buf_size=memset(is->audio_buf,0,is-}elseaudio_size=synchronize_audio(is,(int16_t*)is->audio_buf,audio_size,pts);is->audio_buf_size= if(is->av_sync_typeif(is->av_sync_type!={ref_clock=diff=vp->pts-sync_thresholdsync_threshold=(delay>AV_SYNC_THRESHOLD)?delay:if(fabs(diff)<{if(diff<=-sync_threshold){delay=0;}elseif(diff>={delay=2*}}}gcc-otutorial0