2022貝葉斯網(wǎng)絡(luò)技術(shù)

BayesianFrequentistvs.vs.Frequentist（頻率主義者）:概率是長期的預(yù)期出現(xiàn)頻率P(A)=n/N,wherenisthenumberoftimeseventAoccursinNopportunities.“某事發(fā)生的概率是0.10.1是在無窮多樣本的極限發(fā)生第三次世界大戰(zhàn)的概率是多少Bayesian:degreeofbelief.Itisameasureoftheplausibility（似然性）ofaneventgivenincompleteProbabilityProbabilityisarigorousformalismforuncertainJointprobabilitydistributionspecifiesprobabilityofeveryatomicQueriescanbeansweredbysummingoveratomicFornontrivialdomains,wemustfindawaytoreducethejointsizeIndependenceandconditionalindependenceprovidethetools/ConditionalAandBareindependentP(A|B)= orP(B|A)= orP(A,B)P(A)AisconditionallyindependentofBgivenP(A|B,C)=P(A|ConditionalindependenceisourmostbasicandrobustformofknowledgeaboutuncertainProbabilityProbabilitytheorycanbeexpressedintermsoftwosimpleequations概率理論可使用兩個簡單線性方程來SumRule（加法規(guī)則ProductRule（乘法規(guī)則Graphicalmodels（概率圖模型BayesianInference（推導(dǎo)）inBayesianAlsocalledTheyaugmentanalysisinsteadofusingWhatisaConsistsofnodes(alsocalledvertices)andlinks(alsocallededgesorarcs)每個節(jié)點(diǎn)表示一個隨機(jī)變量(or一組隨機(jī)變GraphicalModelsinacomplexsystemisbuiltbycombiningsimplerparts.WhyareGraphicalModels使每個部分連接起來確保系統(tǒng)作為一個整體是一提供模型到數(shù)據(jù)的連接方法圖理論方面提供bywhichhumanscanmodelhighly-interactingsetsofvariablesthatlendsitselfnaturallytodesigningefficientgeneral-purpose（通用的）algorithmsGraphicalmodels:mixturemodels（混合模型）factoranalysis（因子分析）,hiddenMarkovmodels,Kalmanfilters（卡爾曼濾波器）,etc.優(yōu)勢ProvidesnaturalframeworkfordesigningInsightsintopropertiesofmodelConditionalindependencepropertiesbyinspectinggraph

MorepopularinAI

Markovrandom（馬爾科夫隨機(jī)場MorepopularinVisionandphysicsBayesianasetofnodes,oneperadirected（有向）acyclic（無環(huán)）graph(link"directaconditionaldistributionforeachnodegivenitsP(Xi|Parents(Xi))—Inthesimplestcaseconditionaldistributionrepresentedasaconditionalprobabilitytable條件概率表(CPT)givingthedistributionoverXiforeachcombinationofparentvaluesTopology（拓?fù)浣Y(jié)構(gòu)）ofnetworkencodesconditionalindependenceassertions:Weather獨(dú)立于其他變量ToothacheandCatchareconditionallyindependentgivenCavityVariablesBurglary（入室行竊）Alarm,JohnCalls,網(wǎng)絡(luò)拓?fù)浣Y(jié)構(gòu)反映出因果關(guān)系A(chǔ)burglarcansetthealarmAnearthquakecansetthealarmThealarmcancauseMarytoThealarmcancauseJohntoExampleCompactness（緊致性ACPTforBooleanXiwithkBooleanparentshas2krowsforthecombinationsofparentvaluesEachrowrequiresonenumberpforXi=(thenumberforXi=falseisjust1-Ifeachvariablehasnomorethankparents,thecompletenetworkrequiresO(n·2k)numbersI.e.,growslinearlywithn,vs.O(2n)forthefulljointdistributionForburglarynet,1+1+4+2+2=10numbers(vs.25-1=31)GlobalGlobalsemantics（全局語義Thefulljointdistributionisdefinedastheproductofthelocalconditionaldistributions:Thefulljointdistributionisdefinedastheproductofthelocalconditionaldistributions:LocalLocalsemantics:eachnodeisconditionallyindependentofitsnondescendants（非后代）givenitsparentsTheorem:Local globalCausalChainsIsXindependentofZgivenEvidencealongthechain“blocks”theCommonCause另一個基礎(chǔ)的形態(tài):twoeffectsofthesamecauseAreXandZAreXandZindependentgivenObservingthecauseblocksinfluencebetweeneffects.CommonEffect最后一種配置形態(tài)twocausesofoneeffect(v-structures)AreXandZYes:remembertheballgameandtheraincausingtraffic,nocorrelation?AreXandZindependentgivenNo:rememberthatseeingtrafficputtherainandtheballgameincompetition?ThisisbackwardsfromtheotherObservingtheeffectenablesinfluence NeedamethodsuchthataseriesoflocallytestableassertionsofconditionalindependenceguaranteestherequiredglobalChooseanorderingofvariablesX1,…Fori=1toaddXitotheselectparentsfromX1,…,Xi-1suchP(Xi|Parents(Xi))=P(Xi|X1,...Xi-該父親選擇保證了全局語義ExampleExample(CausalmodelsandconditionalindependenceseemhardwiredforNetworkislesscompact:1+2+4+2+4=13numbers因果關(guān)系當(dāng)貝葉斯網(wǎng)絡(luò)反映真正的因果模式時Oftensimpler(nodeshavefewerOfteneasiertothinkOfteneasiertoelicitfromexperts（專家BNs有時無因果關(guān)系的網(wǎng)絡(luò)是存在的(especiallyifvariablesaremissing)箭頭的真正含義是什么TopologymayhappentoencodecausalTopologyreallyencodesconditionalInferenceinBayesian簡單查詢計算后驗(yàn)概率e.g.P(NoGas|Gauge油表=empty聯(lián)合查詢P(Xi,Xj|E=e)=P(Xi|E=e)P(Xj|最優(yōu)決策:decisionnetworksincludeutilityinformationprobabilisticinferencerequiredforP(outcome|action,evidence)EvaluationVariableelimination（變量消元）:carryoutsummationsright-to-leftstoringintermediateresults(factors：因子)toavoidSinglyconnectednetworks單聯(lián)通網(wǎng)絡(luò)(orpolytrees多樹anytwonodesareconnectedbyatmostone(undirected)timeandspacecostofvariableeliminationareMultiplyconnectednetworks多聯(lián)通網(wǎng)絡(luò)canreduce3SATtoexactinference?NP-equivalenttocounting3SATmodels?#P-Example:Na?veBayesNa?veBayesTotalnumberofparameters（參數(shù)）islinearinExample:Example:一個簡單些簡單的特征來嘗試識別垃圾郵件.我們先考慮兩CapsFreee.g.:amessagewiththesubjectheader“NEWMORTGAGERATE“islikelytobespam.Similarly,for“MoneyforFree”,“FREElunch”,etc.模型的構(gòu)建基于以下三個隨機(jī)變量Caps,FreeandSpam,eachofwhichtakeonthevaluesY(forYes)orN(forNo)Caps=YifandonlyifthesubjectofthemessagedoescontainlowercaseFree=Yifandonlyiftheword`free'appearsinthe(lettercaseisSpam=YifandonlyifthemessageisP(Free,Caps,Spam)=P(Spam)P(Caps|Spam)P(Free,Caps,Spam)=P(Spam)P(Caps|P(Free|Example:Example:Learningtoclassify模型包含先驗(yàn)概率P(Category)P(wordP(Category=c)isestimatedasthefractionofalldocumentsthatareofcategorycP(wordi=true|Category=c)isestimatedasthefractionofdocumentsofcategorycthatcontainwordiTwentyGiven1000trainingdocumentsfromeachgroup.LearntoclassifynewdocumentsaccordingtowhichnewsgroupitcamefromNa?veBayes:89%classificationLearningCurvefor20Example:ADigitNa?veBayesfor簡單版本一種特征ijforeachgridposition可能的特征值是onoff基于圖像中像素的亮度是否大于或小每一個輸入映射到一個特征向量Here:lotsoffeatures,eachisbinaryNa?veBayes