中國機器學(xué)習(xí)白皮書

中國機器學(xué)習(xí)白皮書中國人工智能學(xué)會二○一五年十一月《中國人工智能系列白皮書》編委會主任：李德毅執(zhí)行主任：王國胤副主任：楊放春譚鐵牛黃河燕焦李成馬少平劉宏蔣昌俊任福繼楊強委員：陳杰董振江杜軍平桂衛(wèi)華韓力群何清黃心漢賈英民李斌劉民劉成林劉增良魯華祥馬華東馬世龍苗奪謙樸松昊喬俊飛任友群孫富春孫長銀王軒王飛躍王捍貧王萬森王衛(wèi)寧王小捷王亞杰王志良吳朝暉吳曉蓓夏桂華嚴(yán)新平楊春燕余凱余有成張學(xué)工趙春江周志華祝烈煌莊越挺《中國機器學(xué)習(xí)白皮書》編寫組組長：陳松燦高陽組員：黃圣君李武軍薛暉俞揚余志文詹德川詹志輝張利軍張敏靈 莊福振

目錄第1章引言 [229]等反饋受限問題中，主要目的是支持模糊決策，在探索和利用之間尋找最優(yōu)的平衡。在解決這些實際問題時，又會發(fā)現(xiàn)一些新的問題，產(chǎn)生新的研究方向，促進在線學(xué)習(xí)算法和理論的發(fā)展。完全信息下的在線學(xué)習(xí)研究前沿包括非凸函數(shù)在線學(xué)習(xí)、非線性函數(shù)在線學(xué)習(xí)等問題。賭博機在線學(xué)習(xí)的研究熱點主要圍繞如何將算法和理論拓展到弱反饋場景，比如基于比較的賭博機。

第5章結(jié)束語本白皮書從主流機器學(xué)習(xí)技術(shù)、新興機器學(xué)習(xí)技術(shù)以及大數(shù)據(jù)機器學(xué)習(xí)三方面對機器學(xué)習(xí)的研究和應(yīng)用現(xiàn)狀做了有選擇的簡要介紹。機器學(xué)習(xí)經(jīng)過30余年的發(fā)展，目前已成為計算機科學(xué)中研究內(nèi)涵極其豐富、新技術(shù)、新應(yīng)用層出不窮的重要研究分支。國際上關(guān)于機器學(xué)習(xí)的主要學(xué)術(shù)會議包括每年定期舉行的國際機器學(xué)習(xí)會議（ICML）、國際神經(jīng)信息處理系統(tǒng)會議（NIPS）、歐洲機器學(xué)習(xí)會議（ECML）以及亞洲機器學(xué)習(xí)會議（ACML）等，主要學(xué)術(shù)期刊包括《MachineLearning》、《JournalofMachineLearningResearch》、《IEEETransactionsonNeuralNetworksandLearningSystems》等。此外，人工智能領(lǐng)域的一些主要國際會議（如IJCAI、AAAI等）和國際期刊（如《ArtificialIntelligence》、《IEEETransactionsonPatternAnalysisandMachineIntelligence》等）也經(jīng)常發(fā)表與機器學(xué)習(xí)相關(guān)的最新研究成果。國內(nèi)機器學(xué)習(xí)的重要學(xué)術(shù)活動包括每兩年舉行一次的中國機器學(xué)習(xí)會議（ChinaConferenceonMachineLearning,CCML），該會議目前由中國人工智能學(xué)會和中國計算機學(xué)會聯(lián)合主辦，中國人工智能學(xué)會機器學(xué)習(xí)專業(yè)委員會和中國計算機學(xué)會人工智能與模式識別專業(yè)委員會協(xié)辦，目前已歷經(jīng)15屆。此外，每年舉行的中國機器學(xué)習(xí)及其應(yīng)用研討會（ChineseWorkshoponMachineLearningandApplications,MLA），該會議遵循“學(xué)術(shù)至上、其余從簡”的原則，每屆會議邀請海內(nèi)外從事機器學(xué)習(xí)及相關(guān)領(lǐng)域研究的多位專家與會進行學(xué)術(shù)交流，包括特邀報告、頂會交流、以及TopConferenceReview等部分。迄今已歷經(jīng)13屆，2015年度參會人數(shù)超過1200人。目前，大數(shù)據(jù)浪潮正對人類社會生活、科學(xué)研究的方方面面產(chǎn)生深刻影響。早期機器學(xué)習(xí)研究通常假設(shè)數(shù)據(jù)具有相對簡單的特性，如數(shù)據(jù)來源單一、概念語義明確、數(shù)據(jù)規(guī)模適中、結(jié)構(gòu)靜態(tài)穩(wěn)定等。當(dāng)數(shù)據(jù)具有以上簡單特性時，基于現(xiàn)有的機器學(xué)習(xí)理論與方法可以有效實現(xiàn)數(shù)據(jù)的智能化處理。然而，在大數(shù)據(jù)時代背景下，數(shù)據(jù)往往體現(xiàn)出多源異構(gòu)、語義復(fù)雜、規(guī)模巨大、動態(tài)多變等特殊性質(zhì)，為傳統(tǒng)機器學(xué)習(xí)技術(shù)帶來了新的挑戰(zhàn)。為應(yīng)對這一挑戰(zhàn)，國內(nèi)外科技企業(yè)巨頭如谷歌、微軟、亞馬遜、華為、百度等紛紛成立以機器學(xué)習(xí)技術(shù)為核心的研究院，以充分挖掘大數(shù)據(jù)中蘊含的巨大商業(yè)與應(yīng)用價值?？梢灶A(yù)見，在未來相當(dāng)長的一段時期內(nèi)，機器學(xué)習(xí)領(lǐng)域的研究將以更廣泛、更緊密的方式與工業(yè)界深度耦合，推動信息技術(shù)及產(chǎn)業(yè)的快速發(fā)展。

參考文獻(xiàn)周志華.機器學(xué)習(xí)與數(shù)據(jù)挖掘.中國計算機學(xué)會通訊,2007,3(12):35-44.T.Mitchell.MachineLearning,NewYork:McGraw-Hill,1997.A.N.Meltzoff,P.K.Kuhl,J.Movellan,T.J.Sejnowski.Foundationsforanewscienceoflearning.Science,2009,325(5938):284-288.X.

Wang,A.

Mueen,

H.

Ding,

G.

Trajcevski,

P.

Scheuermann,E.

Keogh.Experimentalcomparisonofrepresentationmethodsanddistancemeasuresfortimeseriesdata.DataMiningandKnowledgeDiscovery.2013,26(2):275-309,2013.E.Levina,P.Bicke.Theearthmover'sdistanceistheMallowsdistance:Someinsightsfromstatistics.InProceedingsofthe8thInternationalConferenceonComputerVision,Vancouver,Canada,2001,251–256.E.Xing,A.Ng,M.Jordan,S.Russell.Distancemetriclearning,withapplicationtoclusteringwithside-information.InAdvancesinNeuralInformationProcessingSystems15,Cambridge,MA:MITPress,2003,505-512.A.Bar-Hillel,T.Hertz,N.Shental,D.Weinshall.Learningdistancefunctionsusingequivalencerelations.InProceedingsofthe20thInternationalConferenceonMachineLearning,Washington,D.C.,2003,11-18.J.Davis,B.Kulis,P.Jain,S.Sra,I.Dhillon.Information-theoreticmetriclearning.InProceedingsofthe24thInternationalConferenceonMachineLearning,Corvallis,OR.,2007,209-216.S.Shalev-Shwartz,Y.Singer,A.Ng.Onlineandbatchlearningofpseudo-metrics.InProceedingsofthe21stInternationalConferenceonMachineLearning,Alberta,P.Jain,B.Kulis,I.Dhillon,K.Grauman.Onlinemetriclearningandfastsimilaritysearch.InAdvancesinNeuralInformationProcessingSystems21,Cambridge,MA:MITPress,2008,761-768.K.Weinberger,L.Saul.Fastsolversandefficientimplementationsfordistancemetriclearning.InProceedingsofthe25thInternationalConferenceonMachineLearning,Helsinki,Finland,2008,1160–1167.S.Paramswaran,K.Weinberger.Largemarginmulti-taskmetriclearning.InAdvancesinNeuralInformationProcessingSystems23,Cambridge,MA:MITPress,2010,1867-1875.K.Huang,R.Jin,Z.Xu,C.-L.Liu.Robustmetriclearningbysmoothoptimization.InProceedingsofthe26thConferenceonUncertaintyinArtificialIntelligence,CatalinaIsland,CA,2010,244-251.G.Checik,U.Shalit,V.Sharma,S.Bengio.Anonlinealgorithmforlargescaleimagesimilaritylearning.InAdvancesinNeuralInformationProcessingSystems22,Cambridge,MA:MITPress,2009,306-314.M.Cuturi,D.Avis.Groundmetriclearning.JournalofMachineLearningResearch,2014,15:533-564.D.-C.Zhan,Y.-F.Li,Z.-H.Zhou.Learninginstancespecificdistancesusingmetricpropagation.InProceedingsofthe26thInternationalConferenceonMachineLearning,Montreal,Canada,2009,1225–1232.J.Goldberger,S.Roweis,G.Hinton,R.Salakhutdinov.NeighbourhoodComponentsAnalysis.In:AdvancesinNeuralInformationProcessingSystems17,Cambridge,MA:MITPress,2004,513–520.A.Bellet,A.Habrard,M.Sebban.Metriclearning.In:SynthesisLecturesonArtificialIntelligenceandMachineLearning,SanFrancisco,CA:MorganandClaypoolPublishers,2015,1-151.Y.Shi,A.Bellet,F.Sha.Sparsecompositionalmetriclearning.In:Proceedingsofthe28thAAAIConferenceonArtificialIntelligence,QuébecCity,Q.Qian,R.Jin,S.Zhu,Y.Lin.Anintegratedframeworkforhighdimensionaldistancemetriclearninganditsapplicationtofine-grainedvisualcategorization.arXiv:1402.0453,2014.M.Schultz,T.Joachims.Learningadistancemetricfromrelativecomparisons.InAdvancesinNeuralInformationProcessingSystems16,Cambridge,MA:MITPress,2004,41-48.X.Gao,S.Hoi,Y.Zhang,J.Wan,J.Li.SOML:Sparseonlinemetriclearningwithapplicationtoimageretrieval.In:Proceedingsofthe28thAAAIConferenceonArtificialIntelligence,QuébecCity,Canada,2014,1206–1212.K.Liu,A.Bellet,F.Sha.Similaritylearningforhigh-dimensionalsparsedata.arXiv:1411.2374,2014.T.Mensink,J.Verbeek,F.Perronnin,G.Csurka.Metriclearningforlargescaleimageclassification:Generalizingtonewclassesatnear-zerocost.InProceedingsofthe12thEuropeanConferenceonComputerVision,Firenze,Italy,2012,488-501.N.Verma,D.Mahajan,S.Sellamanickam,V.Nair.Learninghierarchicalsimilaritymetrics.InProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,Providence,RI,2012,2280-2287.N.Jiang,W.Liu,Y.Wu.Orderdeterminationandsparsity-regularizedmetriclearningadaptivevisualtracking.InProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,Providence,RI,2012,1956-1964.G.Lebanon.Metriclearningfortextdocuments.IEEETransactionsonPatternAnalysisandMachineIntelligence,2006,28(4):497-508.D.Lim,B.McFee,G.Lanckriet.Robuststructuremetriclearning.InProceedingsofthe30thInternationalConferenceonMachineLearning.Atlanta,GA,2013,615-623.T.Kato,N.Nagano.Metriclearningforenzymeactive-sitesearch.Bioinformatics,2010,26(21):2698-2704.J.Wang,X.Gao,Q.Wang,Y.Li.ProDis-ContSHC:Learningproteindissimilaritymeasuresandhierarchicalcontextcoherentlyforprotein-proteincomparisoninproteindatabaseretrieval.BMCBioinformatics,2012,13(S-7):S2.汪洪橋,孫富春,蔡艷寧,陳寧.多核學(xué)習(xí)方法.自動化學(xué)報,2010,36(8):1037-1050.G.R.G.Lanckriet,T.D.Bie,N.Cristianini,M.I.Jordan,W.S.Noble.Astatisticalframeworkforgenomicdatafusion.Bioinformatics,2004,20:2626-2635.F.R.Bach,G.R.G.Lanckriet,andM.I.Jordan.Multiplekernellearning,conicduality,andtheSMOalgorithm.In:Proceedingsofthe21stInternationalConferenceonMachineLearning,Banff,Canada,2004,41-48.G.R.G.Lanckriet,N.Cristianini,P.Bartlett,L.E.Ghaoui,M.I.Jordan.Learningthekernelmatrixwithsemidefiniteprogramming.JournalofMachineLearningResearch,2004,5:27-72.S.Sonnenburg,G.R?tsch,C.Sch?fer,B.Sch?lkopf.Largescalemultiplekernellearning.JournalofMachineLearningResearch,2006,7:1531-1565.A.Rakotomamonjy,F.Bach,S.Canu,Y.Grandvalet.Moreefficiencyinmultiplekernellearning.In:Proceedingsofthe24thInternationalConferenceonMachineLearning,Corvallis,ORA.Rakotomamonjy,F.Bach,S.Canu,Y.Grandvalet.SimpleMKL.JournalofMachineLearningResearch,2008,9:2491-2521.Z.Xu,R.Jin,I.King,M.R.Lyu.Anextendedlevelmethodforefficientmultiplekernellearning.In:AdvancesinNeuralInformationProcessingSystems22,Cambridge,MA:MITPress,2009,1825-1832.Z.Xu,R.Jin,H.Yang,I.King,M.R.Lyu.Simpleandefficientmultiplekernellearningbygrouplasso.In:Proceedingsof27thInternationalConferenceonMachineLearning,Haifa,Israel,2010,1175--1182.S.V.N.Vishwanathan,Z.Sun,N.Ampornpunt.MultiplekernellearningandtheSMOalgorithm.In:AdvancesinNeuralInformationProcessingSystems23,Cambridge,MA:MITPress,2010,2361--2369.R.Jin,T.Yang,M.Mahdavi.Sparsemultiplekernellearningwithgeometricconvergencerate.arXiv:1302.0315v1,2013.M.Kloft,U.Brefeld,S.Sonnenburg,P.Laskov.Efficientandaccuratelp-normmultiplekernellearning.In:AdvancesinNeuralInformationProcessingSystems22,Cambridge,MA:MITPress,2009,997-1005.M.Varma,B.R.Babu.Moregeneralityinefficientmultiplekernellearning.In:Proceedingsofthe26thInternationalConferenceonMachineLearning,Montreal,Canada,2009,1065-1072.A.Jain,S.V.N.Vishwanathan,M.Varma.SPG-GMKL:Generalizedmultiplekernellearningwithamillionkernels.In:Proceedingsofthe18thACMSIGKDDInternationalConferenceonKnowledgeDiscoveryandDataMining,Beijing,China,2012,750-758.C.Hinrichs,V.Singh,J.Peng,S.C.Johnson.Q-MKL:matrix-inducedregularizationinmulti-kernellearningwithapplicationstoneuroimaging.In:AdvancesinNeuralInformationProcessingSystems25,Cambridge,MA:MITPress,2012,1421-1429.C.Cortes,M.Mohri,A.Rostamizadeh.Learningnon-linearcomibinationsofkernels.In:AdvancesinNeuralInformationProcessingSystems22,Cambridge,MA:MITPress,2009,396-404.Q.Mao,I.W.Tsang,S.Gao,L.Wang.Generalizedmultiplekernellearningwithdata-dependentpriors.IEEETransactionsonNeuralNetworksandLearningSystems,2015,26(6):1134-1148.A.Nazarpour,P.Adibi.Two-stagemultiplekernellearningforsuperviseddimensionalityreduction.PatternRecognition,2015,48(5):1854-1862.C.Xu,D.Tao,C.Xu.Asurveyonmulti-viewlearning.arXiv:1304.5434v1,2013.A.Blum,T.Mitchell.Combininglabeledandunlabeleddatawithco-training.In:Proceedingsofthe11thAnnualConferenceonComputationalLearningTheory,Madison,WI,1998,92-100.K.Nigam,R.Ghani.Analyzingtheeffectivenessandapplicabilityofco-training.In:Proceedingsofthe9thInternationalConferenceonInformationandKnowledgeManagement,McLean,VA,2000,86-93.V.Sindhwani,D.S.Rosenberg.AnRKHSformulti-viewlearningandmanifoldco-regularization.In:Proceedingsofthe25thInternationalConferenceonMachineLearning,Montreal,Canada,2009,976-983.Z.-H.Zhou,M.Li.Semi-supervisedregressionwithco-training.In:Proceedingsofthe19thInternationalJointConferencesonArtificialIntelligence,Edinburgh,UK,2005,908-916.S.Bickel,T.Scheffer.Multi-viewclustering.In:Proceedingsofthe4thIEEEInternationalConferenceonDataMining,Brighton,UK,2004,19-26.S.Yu,K.Yu,V.Tresp,H.P.Kriegel.Multi-outputregularizedfeatureprojection.IEEETransactionsonKnowledgeandDataEngineering,2006,18(12):1600-1613.A.Sharma,A.Kumar,H.Daume,D.W.Jacobs.Generalizedmultiviewanalysis:Adiscriminativelatentspace.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,Providence,RI,2012,2160-2167.Z.-H.Zhou,D.-C.Zhan,Q.Yang.Semi-supervisedlearningwithveryfewlabeledtrainingsamples.In:Proceedingsofthe22ndNationalConferenceonArtificialIntelligence,Vancouver,Canada,2007,675-680.J.He,R.Lawrence.Agraph-basedframeworkformulti-taskmulti-viewlearning.In:Proceedingsofthe28thInternationalConferenceonMachineLearning,Bellevue,Washington,2011,25-32.J.Zhang,J.Huan.Inductivemulti-tasklearningwithmultipleviewdata.In:Proceedingsofthe18thACMSIGKDDInternationalConferenceonKnowledgeDiscoveryandDataMining,Beijing,China,2012,543-551.X.Jin,F.Zhuang,S.Wang,Q.He,Z.Shi.Sharedstructurelearningformultipletaskswithmultipleviews.In:LectureNotesinArtificialIntelligence8189,Berlin:Springer,2013,353-368.M.Hodosh,P.Young,J.Hockenmaier.Framingimagedescriptionasarankingtask:Data,modelsandevaluationmetrics.JournalofArtificialIntelligenceResearch,2013,47(1):853-899.L.Ma,Z.Lu,L.Shang,H.Li.Multimodalconvolutionalneuralnetworksformatchingimageandsentences.arXiv:1504.06063v1,2015.M.Hall,E.Frank,G.Holmes,B.Pfahringer,P.Reutemann,I.H.Witten.TheWEKAdataminingsoftware:Anupdate.SIGKDDExplorations,2009,11(1):10-18.J.Alcala-Fdez,A.Fernandez,J.Luengo,J.Derrac,S.Garcaa,L.Sanchez,F.Herrera.KEELdata-miningsoftwaretool:datasetrepository,integrationofalgorithmsandexperimentalanalysisframework.JournalofMultiple-ValuedLogicandSoftComputing,2011,17(2-3):255-287.M.Kearns,L.G.Valiant.Crytographiclimitationonlearningbooleanformulaeandfiniteautomata.In:Proceedingsofthe21stAnnualACMSymposiumonTheoryofComputing,Seattle,Washington,1989,433-444.L.Breiman.Baggingpredictors.MachineLearning,1996,24(2):123-140.Y.Freund,R.E.Schapire.Adecision-theoreticgeneralizationofonlinelearningandanapplicationtoboosting.JournalofComputerandSystemSciences,1997,55(1):119-139.L.Breiman.Randomforests.MachineLearning,2011,45(1):5-32.T.K.Ho.Therandomsubspacemethodforconstructingdecisionforests.IEEETransactionsPatternAnalysisandMachineIntelligence,1998,20(8):832-844.J.J.Rodriguez,L.I.Kuncheva,C.J.Alonso.Rotationforest:Anewclassifierensemblemethod.IEEETransactionsonPatternAnalysisandMachineIntelligence,2006,28(10):1619-1630.L.I.Kuncheva,J.J.Rodriguez.Classifierensembleswitharandomlinearoracle.IEEETransactionsonKnowledgeandDataEngineering,2007,19(4):500-508.Z.-H.Zhou,J.Wu,W.Tang.Ensemblingneuralnetworks:Manycouldbebetterthanall.ArtificialIntelligence,2002,137(1-2):239-263.Z.Yu,L.Li,J.Liu,G.Han.Hybridadaptiveclassifierensemble.IEEETransactionsonCybernetics,2015,42(2):177-190.Z.-H.Zhou.EnsembleMethods:FoundationsandAlgorithms,BocaRaton,FL:Chapman&Hall/CRC,2012.Z.Yu,Z.Deng,H.-S.Wong,L.Tan.Identifyingproteinkinase-specificphosphorylationsitesbasedonthebagging-adaboostensembleapproach.IEEETransactionsonNanoBioScience,2010,9(2):132-143.X.Zhu,P.Zhang,X.Lin,Y.Shi.Activelearningfromstreamdatausingoptimalweightclassifierensemble.IEEETransactionsonSystems,Man,andCybernetics-PartB:Cybernetics,2010,40(6):1607-1621.Y.Xu,X.Cao,H.Qiao.Anefficienttreeclassifierensemble-basedapproachforpedestriandetection.IEEETransactionsonSystems,Man,andCybernetics-PartB:Cybernetics,2011,41(1):107-117.X.Zhu.Semi-supervisedlearningwithgraphs.PhDthesis,CarnegieMellonB.Settles.Activelearningliteraturesurvey.ComputerSciencesTechnicalReport1648,UniversityofWisconsin–Madison,2009.S.Tong,D.Koller.Supportvectormachineactivelearningwithapplicationstotextclassification.In:Proceedingsofthe17thInternationalConferenceonMachineLearning,Stanford,CA,2000,999–1006.N.Roy,A.McCallum.Towardoptimalactivelearningthroughsamplingestimationoferrorreduction.In:Proceedingsofthe18thInternationalConferenceonMachineLearning,Williamstown,MA,2001,441–448.Y.Freund,H.S.Seung,E.Shamir,N.Tishby.Selectivesamplingusingthequerybycommitteealgorithm.MachineLearning,1997.28(2-3):133–168.S.Dasgupta,D.Hsu.Hierarchicalsamplingforactivelearning.In:Proceedingsofthe25thInternationalConferenceonMachineLearning,Helsinki,Finland,2008,208–215.B.Settles,M.Craven.Ananalysisofactivelearningstrategiesforsequencelabelingtasks.In:ProceedingsoftheConferenceonEmpiricalMethodsinNaturalLanguageProcessing,Honolulu,HI,2008,1069–1078.S.-J.Huang,R.Jin,Z.-H.Zhou.Activelearningbyqueryinginformativeandrepresentativeexamples.IEEETransactionsonPatternAnalysisandMachineIntelligence,2014.36(10):1936-1949.R.Chattopadhyay,Z.Wang,W.Fan,I.Davidson,S.Panchanathan,J.Ye.Batchmodeactivesamplingbasedonmarginalprobabilitydistributionmatching.In:Proceedingsofthe18thACMSIGKDDInternationalConferenceonKnowledgeDiscoveryandDataMining,Beijing,China,2012,741-749.S.-J.Huang,S.Chen,Z.-H.Zhou.Multi-labelactivelearning:Querytypematters.In:Proceedingsofthe24thInternationalJointConferenceonArtificialIntelligence,BuenosAires,ArgentinaP.Donmez,J.Carbonell,J.Schneider.Efficientlylearningtheaccuracyoflabelingsourcesforselectivesampling.In:Proceedingsofthe15thACMSIGKDDInternationalConferenceonKnowledgeDiscoveryandDataMining,Paris,France,2009,259–268.D.Margineantu.Activecost-sensitivelearning.In:Proceedingsofthe19thInternationalJointConferenceonArtificialIntelligence,Edinburgh,UK,2005,1622–1623.R.S.Sutton,A.G.Barto.ReinforcementLearning:AnIntroduction.Cambridge,MA:MITPress,1998.P.Abbeel,A.Coates,M.Quigley,A.Y.Ng.Anapplicationofreinforcementlearningtoaerobatichelicopterflight.In:AdvancesinNeuralInformationProcessingSystems19,Cambridge,MA:MITPress,2007,1-8.Y.C.Wang,J.M.Usher.Applicationofreinforcementlearningforagent-basedproductionscheduling.EngineeringApplicationsofArtificialIntelligence,2005,18(1):73-82.J.J.Choi,D.Laibson,B.C.Madrian,A.Metrick.Reinforcementlearningandsavingsbehavior.TheJournalofFinance,2009,64(6):2515-2534.J.A.Boyan,M.L.Littman.Packetroutingindynamicallychangingnetworks:Areinforcementlearningapproach.In:AdvancesinNeuralInformationProcessingSystems6,Burlington,MA:MorganKaufmann,1994,671-671.J.Frank,L.C.Seeberger,R.C.O'Reilly.Bycarrotorbystick:CognitivereinforcementlearninginParkinsonism.Science,2004,306(5703):1940-1943.K.Samejima,Y.Ueda,K.Doya,M.Kimura.Representationofaction-specificrewardvaluesinthestriatum.Science,2005,310(5752):1337-1340.T.G.Dietterich.Machinelearningresearch:Fourcurrentdirections.AIMagazine,1997,18(4),97-136.C.H.Watkins.Learningfromdelayedrewards.Ph.D.Thesis,KingsCollege,UniversityofCambridge,1989.P.L.Bartlett,J.Baxter.Infinite-horizonpolicy-gradientestimation.JournalofArtificialIntelligenceResearch,2001,15:319-350.G.Rummery,M.Niranjan.On-lineQ-learningusingconnectionistsystems.TechnicalReport,UniversityofCambridge,1994.R.J.Williams.Simplestatisticalgradient-followingalgorithmsforconnectionistreinforcementlearning.MachineLearning,1992,8(3):229–256.G.Konidaris,S.Osentoski,P.Thomas.ValuefunctionapproximationinreinforcementlearningusingtheFourierbasis.In:Proceedingsofthe25thAAAIConferenceonArtificialIntelligence,SanFrancisco,CA,2011,380-385.M.Bellemare,J.Veness,M.Bowling.Sketch-basedlinearvaluefunctionapproximation.In:AdvancesinNeuralInformationProcessingSystems25,Cambridge,MA:MITPress,2012,2222-2230.X.Xu,D.Hu,X.Lu.Kernel-basedleastsquarespolicyiterationforreinforcementlearning.IEEETransactionsonNeuralNetworks,2007,18(4):973-992.V.Mnih,K.Kavukcuoglu,D.Silver,A.A.Rusu,J.Veness,M.G.Bellemare,A.Graves,M.Riedmiller,A.K.Fidjeland,G.Ostrovski,S.Petersen,C.Beattie,A.Sadik,I.Antonoglou,H.King,D.Kumaran,D.Wierstra,S.Legg,D.Hassabis.Human-levelcontrolthroughdeepreinforcementlearning.Nature,2015,518:529–533.S.Mannor,R.Y.Rubinstein,Y.Gat.Thecrossentropymethodforfastpolicysearch.In:Proceedingsofthe30thInternationalConferenceonMachineLearning,Atlanta,GA,2013,512-519.I.Szita,A.L?rincz.Learningtetrisusingthenoisycross-entropymethod.NeuralComputation,2006,18(12):2936-2941.S.Schaal.Isimitationlearningtheroutetohumanoidrobots.TrendsinCognitive?Sciences.1999,3(6):233-242.C.Atkeson,S.Schaal.Robotlearningfromdemonstration.In:Proceedingsofthe14thInternationalConferenceonMachineLearning,SanFrancisco,CA,1997,12-20.P.Abbeel,A.Y.Ng.Apprenticeshiplearningviainversereinforcementlearning.In:Proceedingsofthe21stInternationalConferenceonMachineLearning,Banff,Canada,2004,1-8.B.Ziebart,A.Maas,J.Bagnell,A.Dey.Maximumentropyinversereinforcementlearning.In:Proceedingsofthe23thAAAIConferenceonArtificialIntelligence,Chicago,IL,2008,1433-1438.A.Y.Ng,S.J.Russell.Algorithmsforinversereinforcementlearning.In:Proceedingsofthe17thInternationalConferenceonMachineLearning,Stanford,CA,2000,663–670.P.Abbeel,D.Dolgo,A.Y.Ng,S.Thrun.Apprenticeshiplearningformotionplanningwithapplicationtoparkinglotnavigation.In:ProceedingsoftheIEEE/RSJInternationalConferenceonIntelligentRobotsandSystems,Nice,France,2008,1083–1090.M.E.Taylor,P.Stone.Transferlearningforreinforcementlearningdomains:Asurvey.JournalofMachineLearningResearch,2009,10:1633–1685.M.E.Taylor,G.Kuhlmann,P.Stone.Autonomoustransferforreinforcementlearning.In:Proceedingsofthe7thInternationalConferenceonAutonomousAgentsandMultiagentSystems,Estoril,Portugal,2008,283–290.B.DaSilva,G.Konidaris,A.Barto.Learningparameterizedskills.In:Proceedingsofthe29thInternationalConferenceonMachineLearning,Edinburgh,UK,2012,1679-1686.W.B.Knox,P.Stone.Framingreinforcementlearningfromhumanreward:Rewardpositivity,temporaldiscounting,episodicity,andperformance.ArtificialIntelligence,2015,225:24-50.S.J.Pan,Q.Yang.Asurveyontransferlearning.IEEETransactiononDataEngineering,2010.22(10):1345-1359.J.Jiang,C.X.Zhai.Atwo-stageapproachtodomainadaptationforstatisticalclassifiers.In:Proceedingsofthe16thACMConferenceonInformationandKnowledgeManagement,Lisbon,Portugal,2007,401-410.W.Y.Dai,G.R.Xue,Q.Yang,Y.Yu.Co-clusteringbasedclassificationforout-of-domaindocuments.In:Proceedingsofthe13thACMSIGKDDInternationalConferenceonKnowledgeDiscoveryandDataMining,SanJose,CA,2007,210-219.M.Fang,J.Yin,X.Q.Zhu.Transferlearningacrossnetworksforcollectiveclassification.In:Proceedingsofthe13thIEEEInternationalConferenceonDataMining,Dallas,TX,2013,161-170.S.J.Pan,J.T.Kwok,Q.Yang.Transferlearningviadimensionalityreduction.In:Proceedingsofthe23rdNationalConferenceonArtificialIntelligence,Chicago,IL,2008,677-682.J.Blitzer,R.McDonald,F.Pereira.Domainadaptationwithstructuralcorrespondencelearning.In:ProceedingsoftheInternationalConferenceonEmpiricalMethodsinNaturalLanguageProcessing,Sydney,Australia,2006,120-128.Y.Yeh,C.Huang,Y.Wang.Heterogeneousdomainadaptationandclassificationbyexploitingthecorrelationsubspace.IEEETransactionsonImageProcessing,2013,23(5):2009-2018.J.Jiang,C.X.Zhai.InstanceweightingfordomainadaptationinNLP.In:Proceedingsofthe45thAnnualMeetingoftheAssociationforComputationalLinguistics,Prague,CzechRepublic,2007,264-271.W.Y.Dai,Q.Yang,G.R.Xue,Y.Yu.Boostingfortransferlearning.In:Proceedingsofthe24thInternationalConferenceonMachineLearning,Corvallis,ORJ.Gao,W.Fan,Y.Z.Sun,J.Han.Heterogeneoussourceconsensuslearningviadecisionpropagationandnegotiation.In:Proceedingsofthe13thACMSIGKDDInternationalConferenceonKnowledgeDiscoveryandDataMining,Paris,France,2009,339-348.F.Z.Zhuang,P.Luo,H.Xiong,Y.Xiong,Q.He,Z.Shi.Cross-domainlearningfrommultiplesources:Aconsensusregularizationperspective.IEEETransactionsonKnowledgeandDataEngineering,2010,22(12):1664-1678.F.Z.Zhuang,

X.Cheng,

P.Luo,

S.J.Pan,

Q.He.Supervisedrepresentationlearning:Transferlearningwithdeepautoencoders.

In:Proceedingsofthe24thInternationalJointConferenceonArtificialIntelligence,BuenosAires,Argentina,2015,4119-4125.F.Z.Zhuang,

X.Cheng,

S.J.Pan,

W.Yu,

Q.He,

Z.Shi.Transferlearningwithmultiplesourcesviaconsensusregularizedautoencoders.

In:LectureNotesinComputerScience8726,Berlin:Springer,2014,417-431.Q.Q.Gu,J.Zhou.Learningthesharedsubspaceformulti-taskclusteringandtransductivetransferclassification.In:Proceedingsofthe9thIEEEInternationalConferenceonDataMining,Miami,FL,2009,159-168.M.Kan,J.Wu,S.Shan,X.Chen.Domainadaptationforfacerecognition:Targetizesourcedomainbridgedbycommonsubspace.InternationalJournalofComputerVision,2014,109(1):94-109.W.Pan,E.W.Xiang,Q.Yang.Transferlearningincollaborativefilteringwithuncertainratings.In:Proceedingsofthe26thAAAIConferenceonArtificialIntelligence,Toronto,Canada,2012,662-668.G.E.Hinton,R.R.Salakhutdinov.Reducingthedimensionalityofdatawithneuralnetwork.Science,2006,313(5786):504-507.G.Dahl,D.Yu,L.Deng,A.Acero.Context-dependentpre-traineddeepneuralnetworksforlargevocabularyspeechrecognition.IEEETransactionsonAudio,Speech,andLanguageProcessing,2012,20(1):30-42.A.Hannun,C.Case,J.Casper,B.Catanzaro,G.Diamos,E.Elsen,R.Prenger,S.Satheesh,S.Sengupta,A.CoatesandA.Y.Ng.DeepSpeech:Scalingupend-to-endspeechrecognition.arXiv:1412.5567,2014.D.C.Ciresan,U.Meier,L.M.Gambardella,J.Schmidhuber.Deepbigsimpleneuralnetsexcelonhandwrittendigitrecognition.arXiv:1003.0358,2010.A.Krizhevsky,I.Sutskever,G.E.Hinton.Imagenetclassificationwithdeepconvolutionalneuralnetworks.In:AdvancesinNeuralInformationProcessingSystems25,Cambridge,MA:MITPress,2012,1097-1105.C.Szegedy,W.Liu,Y.Jia,P.Sermanet,S.Reed,D.Anguelov,D.Erhan,V.Vanhocke,A.Rabinovich.Goingdeeperwithconvolutions.arXiv:1409.4842,2014.R.Collobert,J.Weston.Aunifiedarchitecturefornaturallanguageprocessing:Deepneuralnetworkswithmultitasklearning.In:Proceedingsofthe25thInternationalConferenceonMachineLearning,Helsinki,Finland,2008,160-167.A.Mnih,G.Hinton.Threenewgraphicalmodelsforstatisticallanguagemodeling.In:Proceedingsofthe24thInternationalConferenceonMachineLearning,Corvallis,OR,2007,641-648.A.Mnih,G.Hinton.Ascalablehierarchicaldistributedlanguagemodel.In:AdvancesinNeuralInformationProcessingSystems21,Cambridge,MA:MITPress,2009,1081-1088.M.K.Leung,H.Y.Xiong,L.J.Lee,B.J.Frey.Deeplearningofthetissue-regulatedsplicin