淺談人工智能下一個十年

1、人工智能的第三次浪潮2人工智能歷史3人工智能領(lǐng)域發(fā)展趨勢Powered byClaude ShannonShannon, Claude E. XXII. Programming a computer for playing chess. Philosophical magazine 41.314 (1950): 256-275.1950計算機象棋博弈1954圖靈測試Alan TuringTuring, Alan M. Solvable and unsolvable problems. Science News-ens. fr 39 (1954).4人工智能領(lǐng)域發(fā)展趨勢Powered by195

2、6達特茅斯會議McCarthy, J., et al. Dartmouth Conference. Dartmouth Summer Research Conference on Artificial Intelligence. 19561959一般問題解決器JohnMarvin McCarthy MinskyNathan RochesterClaudeShannonHerbertJ.C. ShawAllen NewellSimonNewell, A.; Shaw, J.C.; Simon, H.A. (1959). Report on a general problem-solving pr

3、ogram. Proceedings of the International Conference on Information Processing. pp. 256264.5人工智能領(lǐng)域發(fā)展趨勢Powered byDaniel BobrowBobrow, Daniel G.Natural language input for a computer problem solving system. (1964)1964 理解自然語言輸入1966 ELIZA人機對話Joseph Weizenbaum Weizenbaum, Joseph. ELIZAa computerprogram for

4、the study of natural language communication between man and machine. Communications of the ACM 9.1 (1966): 36-45.6人工智能領(lǐng)域發(fā)展趨勢owered by1968 世界首個專家系P 統(tǒng) DENDRALEdward FeigenbaumBuchanan, Bruce, Georgia Sutherland, and Edward A. Feigenbaum. Heuristic DENDRAL: a program for generating explanatory hypothes

5、es in organic chemistry. Defense Technical Information Center, 1968.7人工智能領(lǐng)域發(fā)展趨勢Powered byRandall DavisDrew McDermott, Jon DoyleMcDermott D, Doyle J. Non-monotonic logic IJ. Artificial intelligence, 1980, 13(1): 41-72.1976 大規(guī)模知識庫構(gòu)建與維護Applications of meta level knowledge to the construction, maintenan

6、ce and use of large knowledge basesM. Stanford University, Computer Science Department, AI Laboratory, 1976.1980 非單調(diào)邏輯8人工智能領(lǐng)域發(fā)展趨勢Powered by1980 計算機戰(zhàn)勝雙陸棋世界冠軍Hans BerlinerBerliner H J. Backgammon computer program beats world championJ. Artificial Intelligence, 1980, 14(2): 205-220.9人工智能領(lǐng)域發(fā)展趨勢owered by

7、1987 基于行為的機器P人學(xué)Rodney BrooksBrooks R. A robust layered control system for a mobile robotJ.Robotics and Automation, IEEEJournal of, 1986, 2(1): 14-2310人工智能領(lǐng)域發(fā)展趨勢owered by1987 自我學(xué)習(xí)雙陸棋P程序Gerry TesauroTesauro G. TD-Gammon, a self-teaching backgammon program, achieves master-level playJ. Neural computati

8、on, 1994, 6(2): 215-219.11人工智能領(lǐng)域發(fā)展趨勢Powered byTim Berners-LeeBerners-Lee, Tim. Semantic web road map. (1998).McGuinness, Deborah L., and Frank Van Harmelen. OWL web ontology language overview. W3C recommendation 10.2004-03 (2004): 10.1998 語義互聯(lián)網(wǎng)路線圖2004 OWL語言12人工智能領(lǐng)域發(fā)展趨勢Powered byGeoffrey HintonLe, Qu

9、oc V., et al. Building high- level features using large scale unsupervised learning. arXiv preprint arXiv:1112.6209 (2011).2006 深度學(xué)習(xí)Hinton, Geoffrey E., Simon Osindero, and Yee-Whye Teh. A fast learning algorithm fordeep belief nets. Neural computation 18.7(2006): 1527-1554.2011 高層抽象特征構(gòu)建13人工智能領(lǐng)域發(fā)展趨勢

10、Powered by2009 谷歌自動駕駛汽車Sebastian ThrunMarkoff, John. Google cars drive themselves, in traffic. The New York Times 10 (2010): A1.14人工智能領(lǐng)域發(fā)展趨勢Powered by2011 沃森獲得Jeopardy冠軍IBMs WatsonMarkoff, John. Computer program to take on Jeopardy!. The New York Times (2009).2011 自然語言問答Apples Siri Sadun, Erica, and

11、 Steve Sande. Talking to Siri: Learning the Language of Apples Intelligent Assistant. Que Publishing, 2013.15人工智能近10 年17AI趨勢：從感知到認知From perceptron to cognitionComputingPerceptionCognitionStorage & ComputingRecognize text, images, objects, voicesOrganize and generate knowledge, reasoning18Artificial

12、IntelligenceAlphaGoImage recognitionSelf-driving1920DDPG(2015)A3C(2016)Perceptron(1958)Frank Rosenblatt Cornell University psychologistBPNN/MLP(1986)Hopfield Network(1982) recurrent & feedbackGeoffery Hinton University of Toronto deep learningNeocognitron(1980) convolution & poolingLeNet/CNN(1998)Ya

13、nn LecunNew York University deep learningAlexNet(2012)Relu, dropout & biggerVGG(2014)GoogLeNet(2015)ResNet(2016)Kaiming He MSRA = FAIRcomputer visionDenseNet(2017)RBM(1986/2006)stackDeep Belief Nets(2006)VAE(2013)AutoEncoder(1989/2006) Denosing Autoencoder(2008)Variational InferenceMax WellingUniver

14、sity of Amsterdam statistical learningGAN(2014)DCGAN(2014) WGAN(2017) PGGAN(2017)Ian Goodfellow Google Braindeep adversarial learningRNN/LSTM(1997)Jrgen SchmidhuberIDSIAUniversal AISeq2Seq(2014)RNN in Speech Recognition(2013)Yoshua Bengio University of Montreal Deep learningNeural Probabilistic Lang

15、uage Model(2003)word2Vec(2013)SeqGAN(2017) LeakGAN(2018)Character CNN(2015) self-attention(2017)Deep Q- learning(2013)AlphaGo(2016) Double DQN(2015)Dueling Net(2016)David Silver DeepMindReinforcement learningAlpha Zero(2017) Capsule Nets(2017)算法BERTPre-trainFine tuneBeat all state-of-the-arts on 11

16、NLP tasks in 2018/pdf/1810.04805.pdf21XLNetAutoregressive ModelBeat BERT in 2019/pdf/1810.04805.pdf22ALBERTA Lite BERTParameter-reduction techniquesBeat XLNet and all the others/pdf/1810.04805.pdf23Video-to-Video SynthesisThe best video synthesis performance/abs/1808.0660124graph_netBy DeepMind/abs/

17、1808.0660125MoCoUnsupervised visual representation learningMomentum contrastive learningOutperform its supervised pre-training counterparts/abs/1911.057222627SimCLRSimplified contrastive learning frameworkOutperform previous self-supervised and semi- supervised methods on ImageNet/abs/2002.05709人工智能

18、未來28第三代人工智能的理論體系早在2015年，張鈸老師就提出第三代人工智能體系的雛形； 2017年DARPA發(fā)起XAI項目，從可解釋的機器學(xué)習(xí)系統(tǒng)、人機 交互技術(shù)以及可解釋的心理學(xué)理論三個方面，全面開展可解釋性 AI系統(tǒng)的研究2018年底，正式公開提出第三代人工智能的理論框架體系建立可解釋、魯棒性的人工智能理論和方法發(fā)展安全、可靠、可信及可擴展的人工智能技術(shù)推動人工智能創(chuàng)新應(yīng)用具體實施路線圖與腦科學(xué)融合，發(fā)展腦啟發(fā)的人工智能理論數(shù)據(jù)與知識融合的人工智能理論與方法第三代人工智能的理念在國內(nèi)外 獲得廣泛影響力29認知圖譜 (Cognitive Graph)知識圖譜, 認知推理, 邏輯表達30知識

19、圖譜“Knowledge graph”由Google于2012年提出知識工程，專家系統(tǒng)CYC: 世界上歷史最長的AI項目 (1985)Tim Berners Lee Father of WWW Turing AwardEdward Feigenbaum Father of KB Turing Award31認知圖譜：算法與認知的結(jié)合Quality CafThe Quality Cafe is a now-defunct diner in Los Angeles, California. The restaurant has appeared as a location featured in

20、a number of Hollywood films, including Old School, Gone in 60 Seconds, .Los AngelesLos Angeles is the most populous city in California, the second most populous city in the United States, after New York City, and the third most populous city in North America.Old SchoolOld School is a 2003 American c

21、omedy film released by DreamWorks Pictures andThe Montecito Picture Company and directed by Todd Phillips.Todd PhillipsTodd Phillips is an American director, producer, screenwriter, and actor. He is best known for writing and directing films, including Road Trip (2000), Old School (2003), Starsky &

22、Hutch (2004), and The Hangover Trilogy.Alessandro Moschitti is a professor of the CS Department of the University of Trento, Italy. He is currently a Principal Research Scientist of the Qatar Computing Research Institute (QCRI)Alessandro MoschittiTsinghua UniversityTsinghua University is a major res

23、earch university in Beijing and dedicated to academic excellence and global development.Tsinghua is perenniallyranked as one of the topacademic institutions inChina, Asia, and worldwide.32算法: BIDAF, BERT, XLNet目標：理解整個文檔，而不僅僅是局部片段但仍然缺乏在知識層面上的推理能力BiDAFBERTXLNet33挑戰(zhàn) : 可解釋性34大部分閱讀理解方法都只能看做黑盒:輸入: 問題和文檔輸出

24、: 答案文本塊 (在文檔中的起止位置)如何讓用戶可以驗證答案的對錯:推理路徑或者子圖每個推理節(jié)點上的支撐事實用于對比的其他可能答案和推理路徑36和認知科學(xué)的結(jié)合Dual Process Theory (Cognitive Science)System 1 IntuitiveSystem 2 Analytic371. From Bengios NIPS2019 KeynoteReasoning w/ Cognitive GraphSystem 1:Knowledge expansion by association in text when readingSystem 2:Decision ma

25、king w/ all the informationSystem 1IntuitiveSystem 2Analytic38CogQA: Cognitive Graph for QAAn iterative framework corresponding to dual process theorySystem 1extract entities to build the cognitive graphgenerate semantic vectors for each nodeSystem 2Do reasoning based on semantic vectors and graphFe

26、ed clues to System 1 to extract next-hop entitiesQuestionQuality cafTodd PhillipsGone in 60 secondsOld schoolLos AngelesDominic SenaSystem 2Cognitive Graphlocation featured in a number ofSystem 1inputHollywood films, icncluluedsing Old School,Gone in 60 Secondspredict39Cognitive Graph: DL + Dual Pro

27、cess Theory?System 1: implicit knowledge expansion1. M. Ding, C. Zhou, Q. Chen, H. Yang, and J. Tang. Cognitive Graph for Multi-Hop Reading Comprehension at Scale. ACL19.40System 2: explicit decisionCognitive Graph: DL + Dual Process TheorySystem 1: implicit knowledge expansion1. M. Ding, C. Zhou, Q

28、. Chen, H. Yang, and J. Tang. Cognitive Graph for Multi-Hop Reading Comprehension at Scale. ACL19.41System 2: explicit decision42Cognitive Graph: Representation,Reasoning, and Decision?43認知與推理Trillion-scale common-sense knowledge graphTim Berners Lee Turing Award WinnerEdward Feigenbaum Turing Award

29、 Winner* AI = Knowledge + Intelligence1. J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su. ArnetMiner: Extraction and Mining of Academic Social Networks. KDD08. pp.990-998.Big DataKnowledgeIntelligenceRelated Publications44Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang. Cognitive

30、 Graph for Multi-Hop Reading Comprehensionat Scale. ACL19.Jie Zhang, Yuxiao Dong, Yan Wang, Jie Tang, and Ming Ding. ProNE: Fast and Scalable Network RepresentationLearning. IJCAI19.Yukuo Cen, Xu Zou, Jianwei Zhang, Hongxia Yang, Jingren Zhou and Jie Tang. Representation Learning for AttributedMulti

31、plex Heterogeneous Network. KDD19.Fanjin Zhang, Xiao Liu, Jie Tang, Yuxiao Dong, Peiran Yao, Jie Zhang, Xiaotao Gu, Yan Wang, Bin Shao, Rui Li, and Kuansan Wang. OAG: Toward Linking Large-scale Heterogeneous Entity Graphs. KDD19.Qibin Chen, Junyang Lin, Yichang Zhang, Hongxia Yang, Jingren Zhou and Jie Tang. Towards Knowledge-BasedPersonalized Product Description Generation in E-commerce. KDD19.Yifeng Zhao, Xiangwei Wang,