用DirectShow實現(xiàn)QQ的音視頻聊天功能

1、.用DirectShow實現(xiàn)QQ的音視頻聊天功能 當(dāng)下比較流行的即時通信工具，比如MSN，QQ等都實現(xiàn)了視音頻的功能，通過視頻，音頻，我們可以更好的和朋友通過網(wǎng)絡(luò)進行溝通，本文通過DirectShow技術(shù)模擬QQ實現(xiàn)了視頻和音頻的采集，傳輸，基本實現(xiàn)了QQ的視音頻聊天的功能。        網(wǎng)絡(luò)視音頻系統(tǒng)主要功能就在于視音頻的采集，網(wǎng)絡(luò)傳輸兩個方面，通過Video Capture系列API函數(shù)，你就可以輕松的搞定視頻捕捉，但是對于視頻的網(wǎng)絡(luò)傳輸，則要費一番功夫了。 對于視音頻數(shù)據(jù)的傳輸，只簡單地使用數(shù)據(jù)報套接字傳輸音視頻數(shù)據(jù)

2、是不可行的，還必須在UDP層上采用RTP（實時傳輸協(xié)議）和RTCP（實時傳輸控制協(xié)議）來改善服務(wù)質(zhì)量。實時傳輸協(xié)議提供具有實時特征的、端到端的數(shù)據(jù)傳輸服務(wù)。我們在音視頻數(shù)據(jù)前插入包含有載荷標(biāo)識、序號、時間戳和同步源標(biāo)識符的RTP包頭，然后利用數(shù)據(jù)報套接字在IP網(wǎng)絡(luò)上傳輸RTP包，以此改善連續(xù)重放效果和音視頻同步。實時傳輸控制協(xié)議RTCP用于RTP的控制，它最基本的功能是利用發(fā)送者報告和接收者報告來推斷網(wǎng)絡(luò)的服務(wù)質(zhì)量，若擁塞狀況嚴(yán)重，則改用低速率編碼標(biāo)準(zhǔn)或降低數(shù)據(jù)傳輸比特率，以減少網(wǎng)絡(luò)負(fù)荷，提供較好的Q.S保證。Directshow對于音視頻的采集提供了很好的接口，利用ICaptureGraph

3、Builder2接口可以很輕松的建立起視頻捕捉的graph圖，通過枚舉音頻設(shè)備Filter，也可以很輕松的實現(xiàn)音頻的捕捉，有點麻煩的是音視頻數(shù)據(jù)的傳輸，我們可以自己封裝RTP和RTCP的協(xié)議，來自己實現(xiàn)一個filter，用來發(fā)送和接收音視頻數(shù)據(jù)，當(dāng)然了Directshow也提供了一組支持使用RTP協(xié)議的網(wǎng)絡(luò)傳輸多媒體流的Filters。你也完全可以用Directshow提供的RTP系列的filter實現(xiàn)數(shù)據(jù)的傳輸。下面分析一下這些RTP Filters。新定義的Filter包括 RTP Source Filter ，RTP Render Filter，RTP Demux Filter，RTP

4、Receive Playload Handler (RPH) filter，RTP Send Payload (SPH) filter，使用這5個filter構(gòu)建一個通過RTP協(xié)議傳輸音視頻數(shù)據(jù)的Graph是沒有問題的。RTP Source filter被用來從一個單獨的RTP會話中接收RTP和RTCP包。這個filter提供一個指定發(fā)送給其它主機RTCP接收器報告和指定網(wǎng)絡(luò)地址和端口接口來接收RTP會話的接口。RTP Rend filter是用來將數(shù)據(jù)發(fā)到網(wǎng)絡(luò)上的一個filter，這個filter也提供了和RTP source Filter 類似的接口。RTP Demux filter用來多

5、路分離來自 RTP Source filter的RTP 包，這個filter有一個或者多個輸出的pin。這個Filter提供了如何控制多路分離和如何分配到特定輸出pin的接口。RTP RPH Filter 是用來網(wǎng)絡(luò)過來的RTP包還原成原來的數(shù)據(jù)格式，主要支持H.261，H.263，Indeo，G.711，G.723和G.729和常見的多種音視頻負(fù)載類型。RTP SPH filter則和RPH filter的功能相對，它的任務(wù)是將音視頻 壓縮filter輸出的 數(shù)據(jù)分解為RTP包，它提供的接口有指定最大生成包大小和pt值。下面我們看看如何用這些filter來搭建我們采集和傳輸?shù)膅raph圖。圖

6、1和圖2展示了DirectShow RTP中定義的filters如何運用。圖1是一個采集本地多媒體數(shù)據(jù)并使用RTP協(xié)議通過網(wǎng)絡(luò)發(fā)送的filter graph。它包含一個輸出原始視頻幀的視頻采集filter，緊跟一個壓縮幀的編碼filter。一旦壓縮，這些幀就會被發(fā)送到RTP SPH filter，分片打包，生成RTP包，對應(yīng)的發(fā)送到 RTP Render filter，通過網(wǎng)絡(luò)傳輸這些包。圖2展現(xiàn)了一個filter graph，用來接收包含視頻流RTP包，播放視頻。這個graph由一個用來接收包的RTP Source filter，一個根據(jù)源和負(fù)載類型進行分類的RTP Demux filter

7、，一個把RTP包轉(zhuǎn)為壓縮視頻幀的RTP RPH filter組成。這些filter隨后的是用來解壓幀的解碼filter，一個顯示未壓縮幀的渲染filter。有了RTP filter的幫助我們就可以完成類似qq的功能了，可以實現(xiàn)在網(wǎng)絡(luò)上進行視頻和音頻的交互了，下面我給出在網(wǎng)絡(luò)上兩個客戶端A和B進行音頻和視頻交互的Graph圖。這里我對圖1和圖2中的RTP filter進行了自己封裝，將編解碼filter直接封裝到了RTP Source filter 和RTP Render filter中，這樣Graph圖就顯得很簡潔，RTP Source filter只是用來接收網(wǎng)絡(luò)過來的音視頻數(shù)據(jù)，然后將數(shù)據(jù)

8、傳遞給客戶程序，RTP Render filter則是將采集到的音視頻數(shù)據(jù)發(fā)送到網(wǎng)絡(luò)上的另一個客戶端，編解碼則的工作則封裝到這兩個filter之中。圖3 網(wǎng)絡(luò)視頻和音頻交互的Graph圖如果你也想自己封裝自己的Source 和Render filter，首先你要選擇自己的編解碼，視頻編解碼是選擇H261，H263，還是 MEPG4，音頻是選擇G729還是G711，要首先確定好。選好編解碼，封裝的工作就簡單了。不多說了，下面看看我給出的代碼吧。首先要定義一下用到的四個RTP filter的CLSID。static const GUID CLSID_FG729Render = 0x3556f7d8

9、, 0x5b5, 0x4015, 0xb9, 0x40, 0x65, 0xb8, 0x8, 0x94, 0xc8, 0xf9 ; /音頻發(fā)送 static const GUID CLSID_FG729Source = 0x290bf11a, 0x93b4, 0x4662, 0xb1, 0xa3, 0xa, 0x53, 0x51, 0xeb, 0xe5, 0x8e ;/音頻接收static const GUID CLSID_FH263Source = 0xa0431ccf, 0x75db, 0x463e, 0xb1, 0xcd, 0xe, 0x9d, 0xb6, 0x67, 0xba, 0x72

10、 ;/視頻接收static const GUID CLSID_FH263Render = 0x787969cf, 0xc1b6, 0x41c5, 0xba, 0xa8, 0x4e, 0xff, 0xa3, 0xdb, 0xe4, 0x1f ;/視頻發(fā)送/發(fā)送和接收音視頻數(shù)據(jù)的filterCComPtr< IBaseFilter > m_pAudioRtpRender ;CComPtr< IBaseFilter > m_pAudioRtpSource ;CComPtr< IBaseFilter > m_pVideoRtpRender ;CComPtr<

11、IBaseFilter > m_pVideoRtpSource ;char szClientA100;int iVideoPort = 9937;int iAudioPort = 9938;/構(gòu)建視頻的graph圖,并發(fā)送數(shù)據(jù)CComPtr< IGraphBuilder > m_pVideoGraphBuilder; /視頻圖形管理器 CComPtr< ICaptureGraphBuilder2 > m_pVideoCapGraphBuilder; CComPtr< IBaseFilter > m_pFilterVideoCap;CComPtr<

12、 IVideoWindow > m_pVideoWindow;CComPtr< IMediaControl > m_pVideoMediaCtrl ;CComPtr< IBaseFilter > m_pVideoRenderFilter;HRESULT CMyDialog:VideoGraphInitAndSend()HRESULT hr;hr =m_pVideoGraphBuilder.CoCreateInstance( CLSID_FilterGraph );if(FAILED(hr)return hr;hr =m_pVideoCapGraphBuilder.

13、CoCreateInstance( CLSID_CaptureGraphBuilder2);if(FAILED (hr)return hr;m_pVideoCapGraphBuilder->SetFiltergraph(m_pVideoGraphBuilder);m_pVideoGraphBuilder->QueryInterface(IID_IMediaControl, (void *)&m_pVideoMediaCtrl);m_pVideoGraphBuilder->QueryInterface(IID_IVideoWindow,(void*)&m_pVi

14、deoWindow)FindDeviceFilter(&m_pFilterVideoCap,CLSID_VideoInputDeviceCategory);if(m_pFilterVideoCap)m_pVideoGraphBuilder->AddFilter( m_pFilterVideoCap,T2W("VideoCap") ) ;/創(chuàng)建預(yù)覽的filterhr = m_pRenderFilterVideo.CoCreateInstance(CLSID_VideoRenderer);if(FAILED(hr)return hr;m_pVideoGraphBu

15、ilder->AddFilter( m_pRenderFilterVideo, L"VideoRenderFilter" );Connect(m_pFilterVideoCap ,m_pRenderFilterVideo) ;/設(shè)置預(yù)覽的窗口CRect rc ; GetClientRect(m_hOwnerWnd, &rc );int iWidth = rc.right - rc.left ;int iHeight = rc.bottom - rc.top ;int iLeft, iTop;if(iHeight*1.0)/(iWidth*1.0) >=

16、0.75)/按寬度算int tmpiHeight = iWidth*3/4;iTop = (iHeight - tmpiHeight)/2;iHeight = tmpiHeight;iLeft = 0;else/按高度算int tmpiWidth = iHeight*4/3;iLeft = (iWidth - tmpiWidth)/2;iWidth = tmpiWidth;iTop = 0; m_pVideoWindow->put_Owner( (OAHWND) m_hPreviewWnd ) ;m_pVideoWindow->put_Visible( OATRUE );m_pVi

17、deoWindow->put_WindowStyle( WS_CHILD | WS_CLIPSIBLINGS ) ;/連接到網(wǎng)絡(luò)并發(fā)送CComPtr< IRtpOption > pRenderOption; CComPtr< IVideoOption > pVideoOption;tagVideoInfo vif(160,120,24);int t=(int)(m_iFrameRate/5)*5)+5;vif.nBitCount=24;vif.nWidth=160;vif.nHeight=120;hr = :CoCreateInstance(CLSID_FH263

18、Render, NULL, CLSCTX_INPROC, IID_IBaseFilter, (void *)&m_pVideoRtpRender);if(FAILED(hr)return hr;m_pVideoRtpRender->QueryInterface(IID_IJRTPOption, (void*)&pRenderOption);m_pVideoRtpRender->QueryInterface(IID_IVideoOption,(void*)&pVideoOption); pVideoOption->SetProperty(&vif

19、);pVideoOption->SetSendFrameRate(m_iFrameRate,1);/1 不發(fā)送數(shù)據(jù)，0 實際發(fā)送數(shù)據(jù)Connect(m_pFilterVideoCap ,m_pVideoRtpRender) ;/連接對方hr= pRenderOption->Connect(szClientA,iVideoPort,1024);if(FAILED(hr)return hr;m_pVideoMediaCtrl->Run();/視頻的接收CComPtr< IGraphBuilder > m_pVideoGraphBuilder; /視頻圖形管理器 CCo

20、mPtr< IBaseFilter > m_pFilterVideoCap;CComPtr< IVideoWindow > m_pVideoWindow;CComPtr< IMediaControl > m_pVideoMediaCtrl ;CComPtr< IBaseFilter > m_pVideoRenderFilter;HWND m_hRenderWnd ;HRESULT VideoRecive()HRESULT hr;hr=CoCreateInstance(CLSID_FilterGraph,NULL,CLSCTX_INPROC,IID

21、_IFilterGraph,(void*)&m_pVideoGraphBuilder);m_pVideoGraphBuilder->QueryInterface(IID_IMediaControl, (void *)&m_pVideoMediaCtrl);m_pVideoGraphBuilder->QueryInterface(IID_IVideoWindow,(void*)&m_pVideoWindow)hr = :CoCreateInstance(CLSID_FH263Source, NULL, CLSCTX_INPROC, IID_IBaseFilte

22、r, (void *)&m_pVideoRtpSource);if(FAILED(hr)return hr;m_pVideoGraphBuilder->AddFilter(m_pVideoRtpSource, L"My Custom Source");CComPtr< IRtpOption > m_pRtpOption; CComPtr< IVideoOption > m_pVideoOption;m_pVideoRtpSource->QueryInterface(IID_IJRTPOption, (void *)&m_p

23、RtpOption);m_pVideoRtpSource->QueryInterface(IID_IVideoOption, (void *)&m_pVideoOption);tagVideoInfo vif(160, 120 ,24);m_pVideoOption->SetProperty(&vif);hr= pRenderOption->Connect(szClientA,iVideoPort +1,1024);if(FAILED(hr)return hr;/創(chuàng)建預(yù)覽的filterhr = m_pRenderFilterVideo.CoCreateInst

24、ance(CLSID_VideoRenderer);if(FAILED(hr)return hr;m_pVideoGraphBuilder->AddFilter( m_pRenderFilterVideo, L"VideoRenderFilter" );Connect(m_pVideoRtpSource ,m_pRenderFilterVideo) ; CRect rc ; GetClientRect(m_hOwnerWnd, &rc );int iWidth = rc.right - rc.left ;int iHeight = rc.bottom - rc

25、.top ;int iLeft, iTop;if(iHeight*1.0)/(iWidth*1.0) >= 0.75)/按寬度算int tmpiHeight = iWidth*3/4;iTop = (iHeight - tmpiHeight)/2;iHeight = tmpiHeight;iLeft = 0;else/按高度算int tmpiWidth = iHeight*4/3;iLeft = (iWidth - tmpiWidth)/2;iWidth = tmpiWidth;iTop = 0; m_pVideoWindow->put_Owner( (OAHWND) m_hRen

26、derWnd ) ;m_pVideoWindow->put_Visible( OATRUE );m_pVideoWindow->put_WindowStyle( WS_CHILD | WS_CLIPSIBLINGS ) ; m_pVideoMediaCtrl->Run();return S_OK;/HRESULT FindDeviceFilter(IBaseFilter * ppSrcFilter,GUID deviceGUID)HRESULT hr;IBaseFilter * pSrc = NULL;CComPtr <IMoniker> pMoniker =NU

27、LL;ULONG cFetched;if (!ppSrcFilter)return E_POINTER;/ Create the system device enumeratorCComPtr <ICreateDevEnum> pDevEnum =NULL;hr = CoCreateInstance (CLSID_SystemDeviceEnum, NULL, CLSCTX_INPROC,IID_ICreateDevEnum, (void *) &pDevEnum);if (FAILED(hr)return hr;/ Create an enumerator for the

28、 video capture devicesCComPtr <IEnumMoniker> pClassEnum = NULL;hr = pDevEnum->CreateClassEnumerator (deviceGUID, &pClassEnum, 0);if (FAILED(hr)return hr;if (pClassEnum = NULL)return E_FAIL;if (S_OK = (pClassEnum->Next (1, &pMoniker, &cFetched)hr = pMoniker->BindToObject(0,

29、0,IID_IBaseFilter, (void*)&pSrc);if (FAILED(hr)return hr;elsereturn E_FAIL;*ppSrcFilter = pSrc;return S_OK;/構(gòu)建音頻Graph圖，并發(fā)送CComPtr< IGraphBuilder > m_pAudioGraphBuilder; /音頻圖形管理器 CComPtr< ICaptureGraphBuilder2 > m_pCapAudioGraphBuilder; CComPtr< IBaseFilter > m_pFilterAudioCap;C

30、ComPtr< IMediaControl > m_pAudioMediaCtrl ;HRESULT AudioGraphInit()HRESULT hr;hr =m_pAudioGraphBuilder.CoCreateInstance( CLSID_FilterGraph );if(FAILED(hr)return hr;hr =m_pCapAudioGraphBuilder.CoCreateInstance( CLSID_CaptureGraphBuilder2);if(FAILED (hr)return hr;m_pAudioGraphBuilder->SetFilt

31、ergraph(m_pCapAudioGraphBuilder);m_pAudioGraphBuilder->QueryInterface(IID_IMediaControl, (void *)&m_pAudioMediaCtrl);FindDeviceFilter(&m_pFilterVideoCap,CLSID_AudioInputDeviceCategory);if(m_pFilterAudioCap)m_pAudioGraphBuilder->AddFilter( m_pFilterAudioCap,T2W("AudioCap") ) ;

32、/發(fā)送到網(wǎng)絡(luò)hr =:CoCreateInstance(CLSID_FG729Render,NULL,CLSCTX_INPROC,IID_IBaseFilter,(void*)&m_pFilterRtpSendAudio)if(FAILED(hr)return hr;m_pAudioGraphBuilder->AddFilter(m_pAudioRtpRender, L"FilterRtpSendAudio");Connect(m_pFilterAudioCap,m_pAudioRtpRender);CComPtr< IRtpOption > pOption ;m_pAudioRtpRender->QueryInterface(IID_IJRTPOption,(void*)&pOption)hr =pOption->Connect(szClientA,iAudioPort,1024);if(FAILED(hr)return hr;m_pAudioMediaCtrl->Run();return