貪心科技cv計算機視覺訓(xùn)練營project1-image captioning

(ImageAcatlyingona aacatlyingonaCNN+LSTM+aacatlyingonaCNN+LSTM+應(yīng)用領(lǐng)域文字嵌入

遷移學(xué)習(xí)(transferLSTM(遞歸神經(jīng)網(wǎng)絡(luò)文字嵌入

遷移學(xué)習(xí)(transferLSTM(遞歸神經(jīng)網(wǎng)絡(luò)文字嵌入

遷移學(xué)習(xí)(transferLSTM(遞歸神經(jīng)網(wǎng)絡(luò)文字嵌入

遷移學(xué)習(xí)(transferLSTM(遞歸神經(jīng)網(wǎng)絡(luò)文字嵌入

遷移學(xué)習(xí)(transferLSTM(遞歸神經(jīng)網(wǎng)絡(luò)絡(luò)adogisrunningthroughthe絡(luò)文字嵌入

遷移學(xué)習(xí)(transferLSTM(遞歸神經(jīng)網(wǎng)絡(luò)PythonAnaconda安裝包64windows:https://

condainstall-cconda-forgeKerascondainstall-cconda-forgenltkcondainstall-canacondaPillowcondainstall-canacondaOpenCVcondainstall-cconda-forgecondainstallpytorch=0.4.1-c數(shù)據(jù)集(FramingImageDescriptionasaRankingTaskDataModelsandEvaluationMetrics2013.)8000圖像每幅圖5個標(biāo)題分為3個集合6000個訓(xùn)練圖像1000個開發(fā)圖像1000個測試圖Achildinapinkdressisclimbingupasetofstairsinanentryway.AgirlgoingintoawoodenbuildingAlittlegirlclimbingintoawoodenyhouse.Alittlegirlclimbingthestairstoheryhouse.Alittlegirlinapinkdressgoingintoawoodencabin

AchildinapinkdressisclimbingupasetofstairsinanentrywayAgirlgoingintoawoodenbuilding.Alittlegirlclimbingintoa yhouse.Alittlegirlclimbingthestairsto yhouse.Alittlegirlinapinkdressgoingintoawoodencabin衡量兩個句子的相似度一個句子與其他幾個句子的相似度(CorpusBLEU:BilingualEvaluationUnderstudy(雙語評估替換BLEU，全稱為BilingualEvaluationUnderstudy（雙語評估替具體內(nèi)容可以參考: 或者sequencefeature)圖像sequencefeature)圖像單詞sequencesequencefeature)單詞從圖像(image)到圖像特征(image[0.235,0.457,0.5685,...,VGG16(VeryDeepConvolutionalNetworksforLarge-ScaleVisualRecognition)Pre-trainedmodel:OxfordVisualGeometryGroup贏得2014用于圖像分類將輸入圖像分為1000遷移學(xué)習(xí)(transferVGG16CNN原本的目標(biāo)是分類基于ImageNet數(shù)據(jù)集進(jìn)行訓(xùn)練,VGG16的最后一層是將倒數(shù)第二層 單詞嵌入(WordLSTM的輸入是數(shù)值,單詞需要轉(zhuǎn)換為數(shù)值才能使用LSTM,最簡單的方式是將單詞轉(zhuǎn)化為整數(shù),每個單詞都對應(yīng)于一個整數(shù)但是這樣的方式無法有效的表達(dá)單詞直接的相關(guān)性,例如:國王,王后,男,女如果整數(shù)10,11,200,300表示的話,無法體現(xiàn)出:國王–男+女≈王一個整數(shù)來表達(dá)一個單詞.向量提供了更大的信息量,里面可以嵌入單詞之間的關(guān)系,更好的表達(dá)一個單詞.步驟初始化圖像標(biāo)題為循環(huán)如下步驟將圖像特征和當(dāng)前的圖像標(biāo)題作為輸入,預(yù)測標(biāo)題的下一個單詞,假設(shè)單詞為將word1如果word1的值為”endseq”,或者當(dāng)前標(biāo)題的長度達(dá)到了標(biāo)題最大長度,退出循任務(wù)一:使 創(chuàng) 定義 N網(wǎng)絡(luò)結(jié)任務(wù)二將flicker8k的圖像文件轉(zhuǎn)為圖像特征保存從給定的VGG16網(wǎng)絡(luò)結(jié)構(gòu)文件和網(wǎng)絡(luò)權(quán)值文件創(chuàng)建VGG16 ,key為文件名(不帶.jpg后綴),value為一個網(wǎng)絡(luò)的輸出單詞任務(wù)三完成create_tokenizer單詞sequencesequencefeature)單詞任務(wù)三完成create_tokenizer單詞sequencesequencefeature)任務(wù)三完成create_input_data_for_one_image函為了訓(xùn)練LSTM訓(xùn)練數(shù)據(jù)中的每一個圖像的每一個標(biāo)題都需要被重新拆分為輸入和輸出部分如果標(biāo)題為”acatsitsonthetable”,需要添加起始和結(jié)束標(biāo)志,變?yōu)椤畇tartseqacatsitsonthetableendseq’,再從它astartseqstartseqastartseqacatstartseqacatsitsstartseqacatsitsonstartseqacatsitsonthe另外需要預(yù)處理單詞,‘s和一些不需要的標(biāo)點符號sequencefeature)任務(wù)四:構(gòu)建自動產(chǎn)生圖像標(biāo)題的網(wǎng)絡(luò)模型sequencefeature)單詞單詞使用交叉驗證cross_validation來衡量不同結(jié)構(gòu)的優(yōu)astartseqstartseqastartseqacat[0.234,0.1124,…,startseqacatsitsstartseqacatsitsonstartseqacatsitsonthesequencefeature)sequencefeature)單