數(shù)據(jù)科學導引10.9預處理

1、DataDataSignalHowtosingogtoDigitalDataDataSignalHowtosingogtoDigitalleacontinuousThephenomenonofFastFourieristicalDefinitionofleandleisticsandfeatureFeatureMissingobservationandr數(shù)據(jù)科學導引，歐高炎 et數(shù)統(tǒng)，et抽樣：數(shù)據(jù)科學導引，歐高炎 et數(shù)統(tǒng)，et抽樣：論與應用，勇The1972WaldLecture,Robustistics: etSisticalSourceofrandomnessinSisticalSou

2、rceofrandomnessinWhenthepopulationislarge,needtoleWhenthereisun-controlledorun-controllableerrorin the experiment.Effectivelytocollect 抽樣論,試驗設計EffectivelytousetheExtractinformation,makeinferenceandaccurateandrobustScientificWhatisPopulation(總體 在WhatisPopulation(總體 在統(tǒng)計中，常把研究問題中所關心的對象全le(樣本由于人物的限制，無法對

3、總體所有的對逐個進觀WhatisInformalathematicalWhatisInformalathematicalistics:theleoffiniterealizationsofarandom如：在同一架天平上將一個物體秤n次，得到數(shù)據(jù)則它們的全體，X=(X1,Xn)，稱為樣本。n稱為樣本大小WhatisleofWhatisleofrepresentsoneX(!)21, ,6 , Aleconsistsoftwoindependent(X1,X2)=(X(!1),X(!2)Demo:tosimulateusingcomputerinaidofthe pseudo random nu

4、mber generatorSisticsofGivenSisticsofGivenx1,x9,howtoestimatetheirUse(x1+x2+x3+Sortinincreasingorder:x(1)=x(2)=x(3)= ?2!)D=0!&($ ).+ 8 0?2 B! &+?2!/4%!&+(&?20 : 5#!&-&.(Minkowskidistancein!M&Minkowskidistancein!M&!#$%&$%()$%*+ ,#$-&$-()$-*.+ 0)%$!-1 . ,(lcase:Minkowskidistancein!# &*+Minkowskidistan

5、cein!# &*+,-./0-1/,2,-.30-13,242,-.50-15 !,-./0-1/,32,-.30-13,3242,-.50-&( *+ !8%!Minkowskidistancein! ! ! % Minkowskidistancein! ! ! % % % 特征特征12352045Distancen2ImageSearch(informationDistancen2ImageSearch(informationext,distancentwooftwolanguagesExtractfeatures,computeEuclideandistance/ cosine sim

6、ilarityMysteryofdeepconvolutionalneuralIEEEIEEEStandardfor Floating-Arithmetic(IEEEWhy nP4r7IZob$ Why nP4r7IZob$ 2SiK,-c$_ Y3neOZ XSsa+em*fsa GjnPt2So .01t2S! 2S8 2 0Standardization: Z-MotivedbyGaussianStandardization: Z-MotivedbyGaussianNormalizethefeaturespacetobe+D2 GA 9SC & 1M ? G7 G9SStandardiz

7、ation:MotivedbyUniform methefeaturespaceis$nP6H!a.a5HStandardization:MotivedbyUniform methefeaturespaceis$nP6H!a.a5H! H !_1K9 =:x=)K9O&7+!#$%i -()Oa5*+a &0&,#R#,=C3GTJ8EAB(V12WK9Oa./+PU H WhyWhykv JAfnp+, 1zztr !& 9 # 3 $ 0! .9D: 3 5 9# 1393 : 1& #D:$ 535&$& #D:$ 55:$ # :*:!%: !D. D4 34 / 2-4# D.4 *

8、 6) 00=E$ &FA B814 (!D.6E!#$% 4 &F14 : !DC4 / $1? 5) DifferentwaysofFixed-unsigned:AnN-bitbinaryword,DifferentwaysofFixed-unsigned:AnN-bitbinaryword,retedasunsignedfixed-rational,cantakeonvaluesfromsubsetPofthenon-negativerationalsgivenb=p/2 1,p2PpFloat-Exercise:HowmanydistinctnumbersarethereineachB

9、asicmethodsofBasicmethodsofChallenge: 高維情! $ # % % ) ! MissingandrInprobability(largedeviation)MissingandrInprobability(largedeviation)theor,observeanrisarareArareeventcanhaveabigAlewhohasmissingfeaturecanregardedasanr Butthereasonwhyafeatureismissingcanbe very different.WhythereismissingWhythereism

10、issing!# $ WhythereismissingWhythereismissing叼啄，造成某些產(chǎn)數(shù)據(jù)缺失DealwithMissingIfsomedataismissing,DealwithMissingIfsomedataismissing,howcanweion（插補法MeanNearestneighborOtherCollaborative Bayesian(HMM,KalmanMean?4% 7# + 4% + K+ 4% 2 +/ The Mean?lWf1 0= Y JM ?:mOW Lo ?: H YJMX;QGb5(3Qkn+QKdJMKdJMUVH%cF.le: N

11、ew impact on stock ImpactofImpactofrr !r !r # r # (G+F!%95 EB4A36!c67*1- +F6 $O6!7*? ? :;=& *EB6e78196e8?a *? 8?C*?IrherhestockrhestockrhestockIn IEEE Standard for Floating-Po Arithmetic(IEEE754In IEEE Standard for Floating-Po Arithmetic(IEEE754),WhatisthemachineHowto(machinethestockDemoofPseudorandomnumberrand():uniformDemoofPseudorandomnumberrand():uniformnDemois a