Neural-Network-1-(神經(jīng)網(wǎng)絡(luò)自學(xué)的英文材料)

IntroductionCourseObjectives

Thiscoursegivesanintroductiontobasicneuralnetworkarchitecturesandlearningrules.Emphasisisplacedonthemathematicalanalysisofthesenetworks,onmethodsoftrainingthemandontheirapplicationtopracticalengineeringproblemsinsuchareasaspatternrecognition,signalprocessingandcontrolsystems.WhatWillNotBeCoveredReviewofallarchitecturesandlearningrulesImplementationVLSIOpticalParallelComputersBiologyPsychologyHistoricalSketchPre-1940:vonHemholtz,Mach,Pavlov,etc.Generaltheoriesoflearning,vision,conditioningNospecificmathematicalmodelsofneuronoperation1940s:Hebb,McCullochandPittsMechanismforlearninginbiologicalneuronsNeural-likenetworkscancomputeanyarithmeticfunction1950s:Rosenblatt,WidrowandHoffFirstpracticalnetworksandlearningrules1960s:MinskyandPapertDemonstratedlimitationsofexistingneuralnetworks,newlearningalgorithmsarenotforthcoming,someresearchsuspended1970s:Amari,Anderson,Fukushima,Grossberg,KohonenProgresscontinues,althoughataslowerpace1980s:Grossberg,Hopfield,Kohonen,Rumelhart,etc.ImportantnewdevelopmentscausearesurgenceinthefieldApplicationsAerospaceHighperformanceaircraftautopilots,flightpathsimulations,aircraftcontrolsystems,autopilotenhancements,aircraftcomponentsimulations,aircraftcomponentfaultdetectorsAutomotiveAutomobileautomaticguidancesystems,warrantyactivityanalyzersBankingCheckandotherdocumentreaders,creditapplicationevaluatorsDefenseWeaponsteering,targettracking,objectdiscrimination,facialrecognition,newkindsofsensors,sonar,radarandimagesignalprocessingincludingdatacompression,featureextractionandnoisesuppression,signal/imageidentificationElectronicsCodesequenceprediction,integratedcircuitchiplayout,processcontrol,chipfailureanalysis,machinevision,voicesynthesis,nonlinearmodelingApplicationsFinancialRealestateappraisal,loanadvisor,mortgagescreening,corporatebondrating,creditlineuseanalysis,portfoliotradingprogram,corporatefinancialanalysis,currencypricepredictionManufacturingManufacturingprocesscontrol,productdesignandanalysis,processandmachinediagnosis,real-timeparticleidentification,visualqualityinspectionsystems,beertesting,weldingqualityanalysis,paperqualityprediction,computerchipqualityanalysis,analysisofgrindingoperations,chemicalproductdesignanalysis,machinemaintenanceanalysis,projectbidding,planningandmanagement,dynamicmodelingofchemicalprocesssystemsMedicalBreastcancercellanalysis,EEGandECGanalysis,prosthesisdesign,optimizationoftransplanttimes,hospitalexpensereduction,hospitalqualityimprovement,emergencyroomtestadvisementApplicationsRoboticsTrajectorycontrol,forkliftrobot,manipulatorcontrollers,visionsystemsSpeechSpeechrecognition,speechcompression,vowelclassification,texttospeechsynthesisSecuritiesMarketanalysis,automaticbondrating,stocktradingadvisorysystemsTelecommunicationsImageanddatacompression,automatedinformationservices,real-timetranslationofspokenlanguage,customerpaymentprocessingsystemsTransportationTruckbrakediagnosissystems,vehiclescheduling,routingsystemsBiology?Neuronsrespondslowly –10-3scomparedto10-9sforelectricalcircuits?Thebrainusesmassivelyparallelcomputation

–?1011neuronsinthebrain

–?104connectionsperneuronNeuronModelandNetworkArchitecturesSingle-InputNeuronTransferFunctionsTransferFunctionsMultiple-InputNeuronAbreviatedNotationLayerofNeuronsAbbreviatedNotationWw11,w12,?w1R,w21,w22,?w2R,wS1,wS2,?wSR,=b12S=bbbpp1p2pR=aa1a2aS=MultilayerNetworkAbreviatedNotationHiddenLayersOutputLayerDelaysandIntegratorsRecurrentNetworkAnIllustrativeExampleApple/BananaSorterPrototypeVectorsPrototypeBananaPrototypeAppleShape:{1:round;-1:eliptical}Texture:{1:smooth;-1:rough}Weight:{1:>1lb.;-1:<1lb.}MeasurementVectorPerceptronTwo-InputCaseDecisionBoundaryApple/BananaExampleThedecisionboundaryshouldseparatetheprototypevectors.Theweightvectorshouldbeorthogonaltothedecisionboundary,andshouldpointinthedirectionofthevectorwhichshouldproduceanoutputof1.ThebiasdeterminesthepositionoftheboundaryTestingtheNetworkBanana:Apple:“Rough”Banana:HammingNetworkFeedforwardLayerForBanana/AppleRecognitionRecurrentLayerHammingOperationInput(RoughBanana)FirstLayerHammingOperationSecondLayerHopfieldNetworkApple/BananaProblemTest:“Rough”Banana(Banana)SummaryPerceptronFeedforwardNetworkLinearDecisionBoundaryOneNeuronforEachDecisionHammingNetworkCompetitiveNetworkFirstLayer–PatternMatching(InnerProduct)SecondLayer–Competition(Winner-Take-All)#Neurons=#PrototypePatternsHopfieldNetworkDynamicAssociativeMemoryNetworkNetworkOutputConvergestoaPrototypePattern#Neurons=#ElementsineachPrototypePatternPerceptronLearningRuleLearningRules?SupervisedLearning

Networkisprovidedwithasetofexamples ofpropernetworkbehavior(inputs/targets)?ReinforcementLearning

Networkisonlyprovidedwithagrade,orscore, whichindicatesnetworkperformance?UnsupervisedLearning

Onlynetworkinputsareavailabletothelearning algorithm.Networklearnstocategorize(cluster) theinputs.PerceptronArchitectureWw11,w12,?w1R,w21,w22,?w2R,wS1,wS2,?wSR,=wiwi1,wi2,wiR,=WwT1wT2wTS=Single-NeuronPerceptronDecisionBoundary? Allpointsonthedecisionboundaryhavethesameinnerproductwiththeweightvector.? Thereforetheyhavethesameprojectionontotheweightvector,andtheymustlieonalineorthogonaltotheweightvectorExample-ORORSolutionWeightvectorshouldbeorthogonaltothedecisionboundary.Pickapointonthedecisionboundarytofindthebias.Multiple-NeuronPerceptronEachneuronwillhaveitsowndecisionboundary.Asingleneuroncanclassifyinputvectorsintotwocategories.Amulti-neuronperceptroncanclassifyinputvectorsinto2Scategories.LearningRuleTestProblemStartingPointPresentp1tothenetwork:Randominitialweight:IncorrectClassification.TentativeLearning