神經(jīng)網(wǎng)絡(luò)機器翻譯技術(shù)及應用

1、神經(jīng)網(wǎng)絡(luò)機器翻譯技術(shù)及應用機器翻譯基本原理神經(jīng)網(wǎng)絡(luò)機器翻譯技術(shù)挑戰(zhàn)典型應用未來發(fā)展機器翻譯做什么？我在周日看了一本書Source:IreadabookonSundayTarget:機器翻譯挑戰(zhàn) 譯文選擇IreadabookonSundayTarget:我在周日看了一本書Source:Imeon in atlook watch read see機器翻譯挑戰(zhàn) 譯文調(diào)序我在周日看了一本書Source:IreadabookonSundayTarget:機器翻譯挑戰(zhàn) 數(shù)據(jù)稀疏91%9%中英 中日 中韓 中葡 中俄 中法 中德 中意 中阿 中西 中泰人類語言超過5000種中文相關(guān)主要語種雙語資源分布機器翻

2、譯發(fā)展歷程基于規(guī)則的翻譯我在周日看了一本書Source:翻譯知識：人工撰寫規(guī)則專家知識 準確率高IreadabookonSundayTarget:成本高 周期長 規(guī)則沖突統(tǒng)計機器翻譯我在周日看了一本書Source:數(shù)學模型 成本低數(shù)據(jù)稀疏問題 融合專家知識翻譯知識：從語料庫中自動學習IreadabookonSundayTarget:統(tǒng)計機器翻譯IreadabookonSundayTarget:我在周日看了一本書Source:我去了一趟圖書館。 I went to library. 會議將在周日舉行。The meeting will be held on Sunday. 他在書店看了一本書。He

3、 read a book in a book store.I went to library.The meeting will be held on Sunday.He read a book in a book store. It is a cleverly constructed story.I gave the gifts to mary平 行 語 料單 語 語 料統(tǒng)計機器翻譯我在周日看了一本書Source:我去了一趟圖書館。 I went to library. 會議將在周日舉行。The meeting will be held on Sunday. 他在書店看了一本書。He read

4、 a book in a book store.I went to library.The meeting will be held on Sunday.He read a book in a book store. It is a cleverly constructed story.I gave the gifts to mary平 行 語 料單 語 語 料我I0.5我me 0.3在周日 on Sunday 0.8看書 read book 0.9read a-book0.5 read a-TV0.001翻譯模型 p(e | f )語言模型 p(e)p(e! | f!) i1mexp(iFi

5、 ( f!, e!)exp(ii! F ( f , e )i1me! IreadabookonSundayTarget:神經(jīng)網(wǎng)絡(luò)機器翻譯我在周日看了一本書Source:編碼器IreadabookonSundayTarget:解碼器翻譯質(zhì)量評價 人工評價“譯事三難：信、達、雅。求其信已大難矣！顧 信矣不達，雖譯猶不譯也，則達尚焉。 ” - 嚴復信：忠實度，譯文是否忠實的表達了原文的內(nèi)容 達：流利度，譯文是否流暢和地道翻譯質(zhì)量評價 自動評價基于n-gram，計算機器譯文和人工譯文（參考譯文）的匹配程度Source: Reference:我在周日讀了一本書。I read a book on Sund

6、ay.System1: System2:On Sunday, I read book. I read a book on Sunday.BiLingual Evaluation Understudy(Papineni et al., 2002)機器翻譯質(zhì)量大幅躍升中英翻譯統(tǒng)計機器翻譯神經(jīng)網(wǎng)絡(luò)機器翻譯BLEU神經(jīng)網(wǎng)絡(luò)機器翻譯為了在科學技術(shù)上盡快趕超世界發(fā)達國家中國近年來大幅度增加了研究開發(fā)資金。In order to catch up with the developed countries as soon as possible in science and technologyChina h

7、as substantially increased research and development funds in recent years.神經(jīng)網(wǎng)絡(luò)機器翻譯 基本原理我在周日看了一本書Source:雙向編碼000100000.10.1-0.10.20.30.10.2-0.12.80.30.7-2.6-0.51.20.30.31.2-0.2-0.90.91.00.50.60.2-1-0.10.21.10.70.10.60.40.2-2.60.91.1詞向量表Wh1.1-0.31.2-0.2xi EWihi (hi 1, xi)神經(jīng)網(wǎng)絡(luò)機器翻譯 基本原理我在周日看了一本書Source:雙向

8、編碼向量表示Txci hjj1神經(jīng)網(wǎng)絡(luò)機器翻譯 基本原理我在周日看了一本書Source:雙向編碼注意力機制Txci ijhjj1神經(jīng)網(wǎng)絡(luò)機器翻譯 基本原理我在周日看了一本書Source:雙向編碼解碼zi (ci, ui1, z i1)注意力機制z0u0神經(jīng)網(wǎng)絡(luò)機器翻譯 基本原理我在周日看了一本書Source:I解碼Target:雙向編碼注意力機制z0u0softmax神經(jīng)網(wǎng)絡(luò)機器翻譯 基本原理我在周日看了一本書Source:Iread解碼Target:雙向編碼注意力機制z0u0神經(jīng)網(wǎng)絡(luò)機器翻譯 SOTA神經(jīng)網(wǎng)絡(luò)機器翻譯 SOTA“Attention Is All You Need”- Vasw

9、ani et al., 2017W KWQWV看書q1k1q2v1k2v2W KWQWV看書soft max(QKTd K)Vq1k1q2v1k2v2W KWQWV看書soft max(QKTd K)Vz1z2q1k1q2v1k2v2W KWQWV看書z1z2Multi-headq1k1q2v1k2v2W KWQWV看書z1z2Add&Normalizeq1k1q2v1k2v2W KWQWV看書Multi-headz1z2FeedForwardAdd&Normalizeq1k1q2v1k2v2W KWQWV看書Multi-headz1z2Self AttentionFeedForwardFee

10、dForwardAdd & NormalizeAdd&Normalizeq1k1q2v1k2v2W KWQWV看書Multi-headEncoder-1FeedForward神經(jīng)網(wǎng)絡(luò)機器翻譯面臨的挑戰(zhàn)挑戰(zhàn)一：漏譯Source:最終 真善美徹底打敗了假惡丑Eventually , true , good , and beauty thoroughly defeated the uglyTarget:挑戰(zhàn)一：漏譯Source:Eventually , true , good , and beauty thoroughly defeated the ugly“Addressing the Under

11、-translation Problem from the Entropy Perspective ”zhao et al., To Appear in AAAI-19Target:最終 真善美徹底打敗了假惡丑漏譯與詞語的熵成正相關(guān)數(shù)據(jù)方面Addressing the Under-translationProblem from the Entropy Perspective“Addressing the Under-translation Problem from the Entropy Perspective ”zhao et al., To Appear in AAAI-19Address

12、ing the Under-translationProblem from the Entropy Perspective4546474849NIST03NIST06TransformerThis Work3.00%4.00%5.00%6.00%7.00%8.00%AllTransformerThis Work“Addressing the Under-translation Problem from the Entropy Perspective ”zhao et al., To Appear in AAAI-19NIST04NIST05翻譯質(zhì)量High-Entropy漏譯比例更多工作“Im

13、proved Neural Machine Translation with SMT Features”he et al., 2016“Modeling Coverage for Neural Machine Translation”Tu et al., 2016“Modeling Past and Future for Neural Machine Translation”Zheng et al., 2018譯文長度特征優(yōu)化Attention 歷史和未來信息Coverage Penalty and Length Normalization“Googles Neural Machine Tra

14、nslation System: Bridging the Gap between Human and Machine Translation”wu et al., 2016模 型搜 索（解碼）挑戰(zhàn)二：數(shù)據(jù)稀疏多語言翻譯“Multi-task Learning for Multiple Language Translation”Dong et al., 2015多語言翻譯無監(jiān)督學習“Phrase-Based & Neural Unsupervised Machine Translation”Lampleet al., 2018MonolingualDatasetsInitializationL

15、anguage ModelingBack Translation更多工作Luong et al, Multi-task sequence to sequence learning, ICLR-2016Firat et al., Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism,ACL-2016Firat et al., Zero-resource translation with multi-lingual neural machine translation, arxiv-

16、2016Tobias Domhan and Felix Hieber, Using Target-side Monolingual Data for Neural Machine Translation through Multi-task Learning，EMNLP-2017Unsupervised Machine Translation Using Monolingual Corpora Only. Lample et al., ICLR2018.Unsupervised Neural Machine Translation with Weight Sharing. Yang et al

17、., ACL2018Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation. Fadaee and Monz., EMNLP2018Joint Training for Neural Machine Translation Models with Monolingual Data., Zhang et al. AAAI2018.Dual Transfer Learning for Neural Machine Translation with Marginal Distributi

18、on Regularization. Wang et al., AAAI2018.Zero-Resource Neural Machine Translation with Multi-Agent Communication Game. Chen et al.,AAAI2018.Triangular Architecture for Rare Language Translation. Ren et al., ACL2018.挑戰(zhàn)三：引入知識“Improved Neural Machine Translation with SMT Features”he et al., 2016挑戰(zhàn)三：引入知

19、識更多工作Tu et al., Modeling Coverage for Neural Machine Translation, ACL-2016Wang et al., Neural Machine Translation Advised by Statistical Machine Translation, AAAI-2017Li et al., Modeling Source Syntax for Neural Machine Translation, ACL-2017Zhang et al., Incorporating Word Reordering Knowledge into

20、Attention-based NMT,ACL-2017Wu et al., Improved Neural Machine Translation with Source Syntax, IJCAI-2017Feng et al., Memory-augmented Neural Machine Translation, EMNLP-2017Wang et al., Translating Phrases in Neural Machine Translation, EMNLP-2017Dahlmann et al., Neural Machine Translation Leveragin

21、g Phrase-based Models in a Hybrid Search, EMNLP-2017Incorporating Statistical Machine Translation Word Knowledge Into Neural Machine Translation. Wang et al., TASLP2018.Forest-Based Neural Machine Translation. Ma et al., ACL2018.Top-down Tree Structured Decoding with Syntactic Connections for Neural

22、 Machine Translation and Parsing. Gu et al., EMNLP2018.Dependency-to-Dependency Neural Machine Translation. Wu et al., TALSP2018.Linguistic Knowledge-Aware Neural Machine Translation. Li et al., TALSP2018.Search Engine Guided Neural Machine Translation. Gu et al.,AAAI2018.融合常識、世界知識China-PakistanChin

23、a-Palestine China-Brazil China-Bahamas中巴 經(jīng)貿(mào) 關(guān)系 在 金磚 框架 下 得到 長足 發(fā)展Economic and trade relations have made great progress under the BRIC framework.中巴 經(jīng)貿(mào) 關(guān)系 得到 長足 發(fā)展挑戰(zhàn)四：可解釋性Source:最終 真善美徹底打敗了假惡丑Eventually , true , good , and beauty thoroughly defeated the uglyTarget:Black BoxVisualizing and Understandin

24、g Neural Machine Translation“Visualizing and Understanding Neural Machine Translation ”Ding et al. ACL-2017挑戰(zhàn)五：語篇翻譯S1:我們加入霓虹，我們加入柔和的粉蠟色，我們使用新型材料。S2:人們愛死這樣的建筑了。 S3:我們不斷的建造。T1: We add neon and we add pastels and we use new materials.T2: People love architecture. T3: We keep building.挑戰(zhàn)五：語篇翻譯S1:我們加入霓虹，

25、我們加入柔和的粉蠟色，我們使用新型材料。S2:人們愛死這樣的建筑了。S3:我們不斷的建造。T1: We add neon and we add pastels and we use new materials.T2: And you love it.T3: And we cannot give you enough of it.Modeling Coherence for Discourse Neural Machine Translation“Modeling Coherence for Discourse Neural Machine Translation ”xiong et al.,

26、To appear in AAAI-20192827.52726.52625.52524.5Discourse Level BLEU機器翻譯應用與傳統(tǒng)文化結(jié)合 文言文翻譯你媽喊你回家吃 愛我你飯怕了嗎土豪我們做朋友吧汝母呼汝歸食愛吾君懼矣乎？富賈可為吾友乎與傳統(tǒng)文化結(jié)合 自動寫詩傘 中 何 處 是 儂 家一 曲 斷 橋 人 去 后不 見 西 湖 二 月 花江 南 煙 雨 柳 絲 斜吹 落 梨 花 雪 滿 天東 風 不 解 相 思 意幽 香 一 縷 為 哪 般臘 梅 開 時 獨 自 寒自動寫詩模型與傳統(tǒng)文化結(jié)合 智能春聯(lián)多模翻譯多模 翻譯視覺語言語音同聲傳譯同聲傳譯聲：語音技術(shù)譯：翻譯技術(shù)同：時間延遲小傳：信息傳遞準 要求技術(shù)挑戰(zhàn)那么大家知道這個重莊家最怕的是出現(xiàn)病蟲害一旦就是一個病蟲害出現(xiàn)防治不及時的話會造成大量的這個減產(chǎn)挑戰(zhàn)那么大家知道這個重莊家最怕的是出現(xiàn)病蟲害一旦就是一個病蟲害出現(xiàn)防治不及時的話會造成大量的這個減產(chǎn)問題對策噪聲：冗余、識別錯誤容錯翻譯模型（文本歸一、多維 度特征、抗噪訓練）挑戰(zhàn)那么大家知道這個