內(nèi)容文稿教程rm-finalreport

UndergraduateProjectReport[LeapMotionBasedHand-writtenTextRecognitionSystem] [GAO [Internetof QMStudent BUPTStudent Project DateTableof Chapter1: Chapter2: Somatosensory TesseractOCR Leap Image Gesture Chapter3:Designand Collecting Image Median TesseractOCR Gesture Remove Chapter4:Resultsand Chapter5:ConclusionandFurther Environmental Hand-writtentextrecognitionandsomatosensoryinteractionare2importanttechnologiesofvirtualreality,andvirtualrealitywill ethemostimportantmodeof puterInteraction.Mostpreviousworkarefocusontheirrespectivefields.ItmeansthattherearefewcooperationbetweenHand-writtentextrecognitionandsomatosensoryinteraction.Therefore,theirrecognitionaccuraciesonlycanrelyontheirrespectivealgorithmstoimprovetherecognitionaccuracy.Thispaperpresentsahand-writtentextrecognitionsystembasedonLeapMotionwhichisasomatosensorydevicewithhighprecision.Thesystemcombinedthetechnologiesofhand-writtentextrecognitionandgesturerecognition.Specifically,throughtrainingthenewhand-writtenlanguagelibraryandapplyingthetechnologiesofimageprocessingandgesturerecognition,thissystemnotonlyachievedarobustalgorithmtoimprovehandwritingrecognitionaccuracy,butalsostrengthenedthecapacitytohandlecomplexgestureforthesomatosensoryequipment.Thepaperhas5sections.Chapter1isIntroductionwhichintroducesthetechnicalcontext,forinstance,theconceptandfeaturesofLeapMotion,gesturerecognitionandTesseractOCREngine.Inthesection,theworksIhavedoneandthefeaturesofthisprojectalsoarepresented.Chapter2isBackground,whichdescribestheknowledgethatrelatestothisproject,forexample,Human-ComputerInteraction,Somatosensoryinteraction,C++programminglanguage,theAPIofLeapMotion,imageprocessing,andOpenCV.Chapter3istheDesignandImplementation,whichintroducesspecificallythealgorithmdesignandsystemimplementation,suchashowtogetuserinput,howtoprocessuserinput,howtotrainlanguagelibrary,howtousegesturerecognitiontoimproverecognitionaccuracy.Chapter4isResultandDiscuss,whichshowstheresultofmyworks,andadvantagesanddisadvantagesofthesystem.Chapter5isConclusionandFurtherWork,whichconcludestheworkIhaveachieved,andintroducestheworkwillbeenhancedinthefuture.Keyword：LeapMotion,Hand-writtentextrecognition，Imageprocessing，Gesturerecognition,TesseractOCREngine手寫文字識別和體感交互是2個重要的虛擬現(xiàn)實技術(shù)，并且虛擬現(xiàn)實將會是最主要的人機交互方式。大部分以前的工作關(guān)注于它們各自的領(lǐng)域，這意味著手寫文字識別和體感交互之間很少有合作。因此它們只能依賴各自的技術(shù)去提高識別準確性這篇展示了一種基于厲動這一高精度體感交互設(shè)備的手寫文字識別系統(tǒng)。這個系統(tǒng)聯(lián)合了手寫文字識別技術(shù)和手識別技術(shù)，詳細的說，通過訓(xùn)練新的手寫語言庫，應(yīng)用圖像處理和手勢識別技術(shù)，這個系統(tǒng)不僅實現(xiàn)了一種健壯的算法來提高手寫文字識別的準確性，而且加強了體感設(shè)備處理復(fù)雜手勢的能力。這篇主要分為5個章節(jié)：第一章是介紹，這個部分簡要的介紹了這個項目的技術(shù)背景比如厲動的概念和特點，eseratOR引擎的概念和特點，以及手勢識別技術(shù)的概念和特點。在這一部分我還會介紹我做了什么工作，以及這個系統(tǒng)的特點。第二章是背景，這個部分主要介紹了與這個項目有關(guān)的相關(guān)知識，比如人機交互，體感交互++編程語言，圖像處理的相關(guān)知識，OpnCV和厲動的AI。第三章是設(shè)計，這個部分會詳細地展示算法設(shè)計和系統(tǒng)實現(xiàn)，比如怎樣得到用戶輸入，如何處理用戶輸入，怎樣訓(xùn)練語言庫，怎樣應(yīng)用手勢識別提高識別準確度。第四章是結(jié)果與討論，這部分會展示這個項目的結(jié)果,優(yōu)點，和缺點。第五章是總結(jié)和未來工作，介紹我做過的工作，以及未來可以調(diào)高的工作。關(guān)鍵字：厲動，手寫文字識別，圖像處理，手勢識別，TesseractOCRChapter1:Virtualrealityisanidealmodeof puterInteraction.Itprovidesmoreefficientinteractiveexperience,anditwillmakeanimportantinfluenceinhumanlifeinthefuture.Hand-writtentextrecognitionandsomatosensoryinteractionare2importanttechnologiesofvirtualreality.Withalongtimedevelopment,thetechnologyofprintedcharacterrecognitionhas advantaged.TherearemanyOCREnginesprovidehighaccuracyofrecognitionforprintedcharacter,likeTesseractOCREngine.Now,somatosensorydevicesandgesturerecognitiontechnologyareverypopular,LeapMotionisasomatosensorydevicewhichcanrecognizeandtrackhands,fingersandfinger-liketoolswithhighprecisionandtrackingframerate.Itcanreportdiscretepositions,gestures,andmotiontoo.LeapMotionControllertracksall10fingersupto1/100thofamillimetre,andithaveafieldofviewofabout150degreesandaZ-axisfordepth,whichmeansyoucanmoveyourhandsandfingersin3D,justlikeyountherealworld.[1]However,theaccuracyofrecognitionforhand-writtencharacterislow.Andthesomatosensorydeviceonlyrecognizesomesimplegestures,likecircle,swipe,keytap,andscreentapTherefore,Idesignedahand-writtentextrecognitionsystembasedonLeapMotiontoenhancetheaccuracyofhand-writtentextrecognitionandtheabilityofhandlingcomplexgestureforLeapMotion.Thissystemcombined3coretechnologiesincludingOCR,imageprocessingandgesturerecognition.Inthisproject,using’sTesseractOCREnginehandlesOCR,andinvokingOpenCVoperatesimageprocessing,andutilizinggesturerecognitionimprovestheaccuracyofOCR.Particularly,firstLeapMotionrecordsuser’sinputthatincludespositionsandtimestamps,andsavestheinputasapicture.ThenOpenCVwasusedtoprocessthepicture.Forexample,usingMedianfilterprotectstheimageedge,andsmoothiesnoises,invokingcvDilate()removesuselessthincurves,anddeleteduselesscontentsbyresizingthepicture.Afterthat,thesystemmakesthepictureprocessedastheinputofTesseractOCREnginethathasahand-writtentraineddata.TheoutputofOCREngineistheinitialresult,iftheinitialresultisaconfusingletter,itwillbere-recognized,throughgesturerecognitionbasedonthepositionsandtimestamps.Chapter2:puterInteractionItisastudythatresearchestheinteractiverelationshipbetweensystemandhuman.Thesystemcanbevariousmachinesandsoftware.Humanandcomputeruseaparticularlanguageandinteractivemodetoachieveinformationexchange.[2]puterInteractionisanimportantfactortothefriendlinessofacomputersystem.ThefunctionsofHCImainlyrelyoninput/outputexternaldevicesandrelativesoftware.ThemainapplicationsofHCIincludecontrollingrelativedevices,understandingandexecutingusers’variouscommandsandrequires.Earlyinteractivefacilitiesarethekeypadanddisplay.Operatorsthroughthekeyboardinputthecommand.Commandsareexecutedimmediayaftertheoperatingsystemreceivestheordersanddisplaysresultsonthescreen.Commandsmayhavedifferentformats,buttheinterpretationofeachcommandisclearandunique.Withthedevelopmentofinformationtechnology,patternrecognitions,likevoicerecognition,opticalcharacterrecognitionandgesturerecognition,havegothugeprogress,whichprovidesavarietyofinteractivemode.Sofar,HCIhasexperiencedthefollowingstagesManualJobControlLanguageandInteractiveCommandGraphicalUserInterfaceInligentinteractionwithmultichannelandSomatosensorySomatosensorytechnologyhasmadeitpossiblethatusingbodymovementsdirectlyinteractwiththesurroundingequipmentortheenvironment,withoutusinganycomplicatedcontroldevice.[4]Itcanletpeopleinteractefficientlyandlivelywiththesystemandcontent.Forexample,whenyoustandinfrontofaTV,ifthereisasomatosensorydevices,likeKinect,candetectyourhandmovements,wecouldcontrolevisionfast-forward,rewind,pause,andterminationandotherfunctions,throughourhandmovements,likeup,down,leftandright.Itisanappropriateexampletosomatosensorycontrolperipheraldevices.Ifweuseourbodymovementstocontrolcharacter'sreaction,directly,playerscangetimmersivegamingexperience.OtherSomatosensoryapplicationsinclude3Dvirtualreality,healthcare,motiondetectionandsoon.C++programminglanguageisageneral-purposeprogramminglanguage,whichdevelopedbasedonLanguageC.[5]C++supportsmultipleprogrammingparadigms,likeobject-orientedprogramming,genericprogrammingandproceduralprogramming.Itappliedinmanyfields,likesystemdevelopment,enginedevelopment.Anditisoneofthemostpopularandpowerfulprogramminglanguage,becauseitsupportsclass,encapsulation,andoverload.RelationshipwithLanguageClanguageisthebasisofC++,C++andClanguagearecompatible,inmanyways.Clanguageisastructuredlanguage,itfocusonalgorithmsanddatastructures.ThedesignprincipleofCprogramishowaprocessobtainstheoutput(orimplementation(things)control)fromaninput.ThecoreofC++ishowtoconstructanobjectmodel,whichcanfitthecorrespondingproblem,sothatyoucangetoutputorimplementation(things)controlbyobject'sstateinformation.Insum,thebiggestdifferencebetweenCandC++isthatthinkingthattheysolvetheproblem,isnotthesame.C++isasefficientasCC++supportsvariousprogrammingstyles,likeproceduralprogramming,data ion,object-orientedprogramming,andgenericprogrammingC++programmingdonotneedcomplexprogrammingC++programminglanguageisflexible.Ithasabundantoperators,datastructure,andstructuredcontrolstatements.Andithastheadvantagesofhigh-levelprogramminglanguageandassemblerlanguage.Forexample,C++programsareefficient,readable,andportable.Generallyspeaking,C++languagehas2mainfeatures,oneiscompatibility,theotheroneisobject-orientedapproach.IthaslanguageC'sfeatures,likesimple,efficient,andexpandslanguageC’typesystem,soC++saferthanC.C++introducesobject-orientedconcepts,whi akethedevelopmentofinteractiveapplicationismoresimpleandfast.ManyexcellentprogramframeworklikeBoost,Qt,MFC,OpenCVisusedC++.ThecorrectnessofcomplexC++programsisquitedifficulttoC++hasabundantdifferentstandards,soproducingareasonably pliantC++compilerhasproventobeadifficulttask.[8]TesseractOCRTesseractisafreeOCRenginewhiaybehasthehighestaccuracy.TesseractOCRenginecanreadprintedtextofover60languages.[9]TesseractOCRenginewasdevelopedbyHPlabin1985.Until1995,itwasoneofthetop3OCRenginesaroundtheworld.However,therewaslittleworkdoneonit,between1995and2006,becauseHPdecidedtoabandontheOCRbusiness.Inlate2005,HPreleasedTesseractforopensource.Sincethattime,Tesseracthas ethemostpopularandaccurateopensourceOCRengineavailable.[10]LeapLeapMotionisasomatosensorydevicewhichcanrecognizesandtrackshands,fingersandfinger-liketoolswithhighprecisionandtrackingframerate.[11]Itcanreportdiscretepositions,gestures,andmotiontoo.InteractionLeapMotionhas2opticalsensorsand1infraredlighsor.Throughthesesensors,LeapMotionhasafieldofviewofabout150degrees,andtheeffectiverangeextendsfromaround25to600milimetters.Inthisperception,LeapMotioncanrecognizeandtrackthemovementofhandsandfigures.[12]Forcollectingdata,theLeapMotionsystememploysaright-handedCartesiancoordinateTheoriginisthecenterofthetopoftheLeapMotionController.Thex-andz-axeslieinthehorizontalplane,withthex-axisrunningparalleltothelongedgeofthedeviceandpointingtotheright.[13]They-axisisperpendiculartothedevice,withpositivevaluesincreasingupwards.Thez-axispointtotheuser.Figure1TheLeapMtionright-handedcoordinatesystemTheLeapMotionAPImeasuresphysicaltieswiththefollowingFigure2TheunitsofphysicaltitiesmeasuredbytheLeapMotionMotiontrackingInthefieldofview,LeapMotiontracksthemovementofhandsandfinger,anditrecordsandupdatesthedatainformofframe.Thedatathateachframecontains,includes:ThelistandinformationofallThelistandinformationofallThelistandinformationofThegestureandLeapMotionControllerdistributesauniqueIDtoeachobject.AndtheIDsneverchange,whentheobjectsstaywithinthefieldofview.BasedontheIDs,wecaninquireinformationabouteachTheLeapMotioncontrollercanrecordandprovideinformationabouteachfingeronahand.Forexample,theinformationincludesfinger’sname,long,position,direction,andspeed.Andtherearemanyfunctions,likefingerType(),fronmost(),length(),tipposition(),whichcangettheinformation,Figure3ThemodeloffingerstrackedbyLeapLeapMotionprovidesAPIstorecognizecertainmovementpatternsoffingersortoolasgestureswhichcouldindicateauserintentorcommand.TheLeapMotionsoftwarepresentsgesturesinaframethatisthesamewayitreportsothermotiontrackingdatalikefingersandhands.[17]LeapMotion’sSDKprovides4gestures,includingCircleGesture,KeyTapGesture,ScreenTapGesture,andSwipeGesture.ThefollowingmovementpatternsarerecognizedbytheLeapMotionFigure4Circle–AfingertrackingaFigure5Swipe–Along,linearmovementofahandanditsfingersFigure6KeyTap–AtapmovementbyafingerasiftapakeyboardFigure7ScreenTap–AtapmovementbythefingerasiftapaverticalcomputerImageImageprocessingreferstotheinformationtechnologythatremovesnoise,andenhances,restores,segments,extractsfeaturesforimages,throughcomputer.Itcanimprovethequalityofimages,forexample,enhancingthelightofimages,transformingcolour,enhancingorrestrictingcertaincontent,makinggeometrictransformationforimages.Itcanextractcertainfeaturesorspecificinformationcontainedintheimage.Thefeatureinformationfacilitatecomputerysisoftheimage.Extractingfeaturesmayincludemanyaspects,suchasthefrequency,grayscaleorcolorcharacteristics,boundarycharacteristics,regionalcharacteristics,texture,shapefeatures,topologicalfeaturesandrelationshipstructure.[21]Itcanfacilitateimagestorageandtransmissionthroughtransforming,encodingandcompressionimageOpticalCharacterRecognitionOpticalcharacterrecognition(OCR)isthetechnologyandprocesswhichconversestheimagesoftypewrittenorprintedtextintomachine-encodedtext.OCRtechnologyrelatestopatternrecognition,artificialinligenceandcomputervision.[22]TheindexesusedtomeasuretheperformanceofanOCRsystemincluderejectionrate,friendlyerrorrate,recognitionspeed,userinterface,productstability,easeofuseandfeasibilityOCRsoftwareoften"pre-process"imagestoimprovethechancesofsuccessfulrecognition.Thefollowingtechniquesof"pre-processes"couldreducethedifficultyoffeatureextractionalgorithm,andcanimprovetheaccuracyofrecognitionBinarization–itcanseparatethetextfromthebackground,anditimprovethespeedoftextrecognition.Weneedconvertanimagefromcolororgreyscaletoblack-and-white.NoiseRemoval–theimagesisnotalwaysgood,whieanthatthereisalotofnoise,sotheimagesshouldberemovednoisebasedonthenoisesfeatureofimagestoimprovetheaccuracyofrecognition,beforetheOCRenginerunsrecognitionprocessingforthecharacters.Tiltcorrection–becausetheinputtedimagesarealwaysdeclining,itisnecessarytodetectandcorrectimages’directionTextfeatureextraction–itisthecoreofOCR,becauseitdirectlyaffectstheresultofOCR.Andthefeaturesisthebasisofrecognition.Thefeaturescanbedividedinto2categories:oneisstatisticalfeature,suchastheproportionofblackandwhitepointwithinthewordregion.Whenstatisticalfeaturematchingisrun,thebasicmathematicaltheoryissufficient.Theothercategoryisstructurefeature,likeCharacterstrokesendpoints,thenumberandlocationoftheintersection,lines,closedloops,linedirection.Itneedsspecificmatchingmethodstomatchfeatures.CharacterBecausethetextfeaturesaredividedintostatisticalfeatureandstructurefeature,therearetwobasicapproachesofcharacterrecognition,whichproduceacorrectcandidatecharacters.Statisticalrecognisersarebasedonthestatisticalfeature,liketheradiobetweenblackpixelandwhitepixelwithinatextregion.Whenthetextregionisseparatedseveralregions,thecombinationofsingleradioofblack/whitepointsmakesupanumericvectorspace.Mathematictheoriesareusedtomatchfeatures,inthestatisticrecogniserStructurerecognisersusestructurefeatureslikelines,closedloops,linedirection,andlineintersectionstorecognisecharacters.[23]Differentstructurefeatureneeddiversematchingmethod.Post-SinceOCRrecognitionratecannotreachonehundredpercent,orwewouldliketostrengthenconfidenceandaccuracyofrecognition,orevenwewanttoaddthedebuggingfeaturestohelpcorrect,post-processing esanecessarymoduleofOCR.“Afterwordprocessing”isanexample,basedonthebeforeandafterthewordstofindthemostlogicalintherecognizedcandidates,todocorrectionfunction.UpdateThecorrectedtextsandtheirfeaturescanbeaddedtothedatabase.Throughupdatingdatabase,theaccuracyofTesseractOCRenginecanstrengthentheconfidenceandaccuracywhenitrecognizethesametextGestureBroadlyspeaking,gesturereferstothemovementofhandsorfingers.Whetheroperatingobjectorcommunicating,gesturesalwayspresenttheintentionofpeopleGesturerecognitionisatechnologythatconvertsgesturestocomputercommandsthroughprocessthegesturesGesturerecognitionisbasedonauser'sgesture,anditidentifythemeaningofthemovement.Itdirectlyasacomputerinputdevice,andthe puterInteractionnolongerneedotherintermediateequipment.Peoplecandefinesomesimplegesturetocontrolsurroundingmachine.Theexecutionofgesturesisadynamicprocedure,forinstance,thechangeofpositionanddirectionofhandinspace.Sothefeaturesofagestureshouldbedescribedfrom2aspects:timeandspace.Basedonthetime-varyingcharacteristicsofgestures,gesturerecognitioncanbedividedinto2classes:staticgesturerecognitionanddynamicgesturerecognition.[24]Theresearchpointofstaticgesturerecognitionistheposturesofhandsandsinglehandshape.Thatmeansitonlyneedtorecognizethefeaturesofhandshape.Forexample,inAmericanSignLanguage,theshapeoffingerspresentstheEnglishletter.Theobjectofdynamicgesturerecognitionareaseriesofconsecutiveactions.Forexample,agesturerecognitiondeviceusesappropriatealgorithmtorecognizethemeaningofthewholeaction.OpenCV(OpenSourceComputerVision)isacross-platformlibraryofprogrammingfunctionsmainlyaimedatreal-timecomputervision.[25]OpenCVislightweightandefficient,becauseitiswritteninC++anditsprimaryinterfaceisinC++,butithasfullinterfacesinPython,Java,.Chapter3:DesignandFigure8CollectingForcollectinginputfromLeapMotionController,firstlyaControllerobjectthatconnectstothedeviceandaListenerobjectthathandlesevensandserver,werecreatedinthemain()function,andthecall-backfunctionswereoverriddenforthissystem.Thenthelistenerobjectwasaddedtothecontrollerobjecttolistenandresponsetheevensofthecontroller.Afterthat,theeventslistenedbylistenerweredeclared.Theinformationabouttheuserinputswascollectedasaframeformat.TheonFrame()callbackfunctionwasoverriddentogettheappropriateinformation.Becausethemotionofoneuser’sfingerisinput,onlythepositionsofthemostfrontfingeroftheuserwerecollected.Thepositionisa3Ddataincludingx,y,andz.thex-axisandy-axisformaplanethatsimulatesapapertorecordthetrackofuser’sfinger.Andthevalueofz-axisaffectsthethicknessofcurves,whiakethetrackshownonthewindowlookslikewrittenwithapen.Inthisway,theinteractioninterface esfriendlier.Andthevalueofz-axisdecidesthatthesystemwhetherrecordsthepositionoffinger.Forrealtimeshowingthetrackofuserinput,manyOpenCVfunctionswereinvokedintheonFrame()function.Forexample,cvLine()wereinvokedtoconnectcurrentpointoffingerwithlastpointtomakeacurve,andcvShowImage()showsthecurvesonacertainwindowGestureswereusedtoachievecertaincommands.IfTYPE_SWIPEgesturewasrecognized,allofthetracksshownonthewindow,andthepositionsrecordedwillbedeleted.IfTYPE_KEY_TAPgesturewasrecognized,thetrackswillbesavedasapicture.Thenthesystemexecutesimageprocessingtothepicture.AndOCRenginewillbeusedtorecognizethecharactersavedintheprocessedimageImageImageprocessingcanimprovetherecognitionaccuracyofOCRandtheeffectiveofrecognition,becauseimageprocessingcanremovenoiseandenhanceimagefeaturesThereare3imageprocessingtechnologiesinthisInimageprocessing,suchasedgedetection,usuallyitneedsanoisereduction.Medianfilterisacommonstepinimageprocessing,anditisparticularlyusefultoprocessspecklenoiseandpepperMedianfilterisanonlinearsmoothingtechnique,whichisusuallyusedtoprotectedgeinformation.Specifically,itusethemedianofalltheneighbourhoodwindowpixeltoreplacethepixelofapoint.Inthissystem,medianfilterfunctionwasprovidedbyOpenCV,anditwasinvokedtosmooththeinputimageAim:deletinguselessthinForrealtimeshowingthetrackandpositionofuser’sfingerinthecertainwindow,therearelotsofuselessthinlinessavedinthepicture.SothissysteminvokeddilationfunctionfromOpenCVtoremovetheuselessthinlinesFigure9BeforeForreducinguselesscontainsandstrengthenstructurefeature,thesystemresizedtheinputtedFirstly,everypixeloftheimagewastraversed.Then,thesystemcalculatedandrecordedthesumofpixelofeverycolumnandrow.Afterthat,thesumofeverycolumnwascomparedwithaparticularvalue127500(thepixelofwhitemultiplythenumberofcolumn),columnbycolumnfromlefttoright.Whenthe2valuesarenotsame,thepositionofcolumnistheleftedgeofresizedimage.Thesamemethodwasusedtocalculatetherightedge,topedge,andloweredge,butthesequenceofcomparisonisfromrighttoleft,fromtoptobottom,andformbottomtotop,respectively.Finally,systemutilizedOpenCV’sresizingfunctiontoresizetheimage.TesseractOCRAlthoughTesseractOCREngineisthemostaccurateopensourceOCRengine,anditsupportmorethan60language.Butithasn’tanappropriatehand-writtentextlibrary.Thatmeansitisnotgoodatrecognizinghand-writtentext.Thesystemcollected4hand-writtenEnglishalphabetastrainingdatatotrainahand-writtentextlibrary.Usingthenewhand-writtentextlibrary,TesseractOCREngineimprovedtheaccuracyofrecognitionforhand-writtenEnglishletters.Figure12correctingtheGestureItisverydifficulttoaccurayrecognizeeveryhand-writtenEnglishcharacterthroughusingTesseractOCREngineandthetrainedhand-writtentextlibrary.SothesystemusedagesturerecognitionalgorithmtoimprovetheaccuracyofrecognitionThegesturerecognitionalgorithmincludesthefollowingstepsRemoveLeapMotionisahighlyprecisionsomatosensorydevices.Itcanrecognizemovementoffingerupto1/100thofamillimetre.Andtheupdatefrequencyofdataisupto200framesperminute.SothepositiondatathatcollectedthroughLeapMotionControllerisverylarge,whichreducedtheprocessefficiency.Duetothehighlyprecision,lotsofuselesspositions,likehandorfingershakinginformation,werealsorecordedintothepositiondata.TheseuselessinformationdirectlyinfluencetheefficiencyandaccuracyofgesturerecognitionInthesystem,thereisanoisereducingsteptohelpimprovetheefficiencyandaccuracyofgesturerecognition.Particularly,theoriginalpositiondatacollectedthroughLeapMotionController,exceptthefirstposition,allofthepositionswithadistancetothepreviouspositionislessthan10mm,shouldbedeleted.Thatmeansinthenewpositiondata,thedistancesbetweenany2neighbouringpointsmustbemorethan10mmFigure13RemoveDetectForrecognizinggesture,thesystemusedthecharacteristicsofthetrigonometricfunctionstoconvertthegestureintoan8-directiongesturesequence.Particularly,everylinesegment,between2positioninthenewpositiondata,convertedintooneof8-direction:up,right-up,right,right-down,down,left-down,left,left-up,andthesystemused1to8torepresentthem.Inthisway,thepositiondonvertedtoadirectionset.Figure148-directioncoordinateThisstepconvertedthe8-directionsgesturesequencetoamoresimplepresentation(e.g.,“65432187”).Inthisway,thesystemcaneasilymatchedtothegesturesFigure15Troughimageprocessing,andtraininghand-writtentextlibrary,mostofthehand-writtencharacterscanberecognizedaccuray.But,therearestillsomecharactersareeasilyconfused,becauseitsstructurefeaturesaretoosimilartoidentify,like‘c’and‘e’.ThesystemusedgesturerecognitiontoimprovetheresultofTesseractOCREngine.Forexample,‘c’and‘e’areeasilyconfusedbyTesseractOCREngine,butitsfirstdirectionalwaysaredifferent.Thefirstdirectionof‘e’onlycanbe2,or3,or4,whilethefirstdirectionof‘o’isalwaysnot2,or3,or4,sowecanusethisfeatureofgesturetorecognize‘c’or‘e’.Usingthegesturesequence,we sorecognizeothereasilyconfusedletter,like‘o’and‘a(chǎn)’EvenwecancalculatethedistanceofgesturesequencetorecognizewhichletterismoreChapter4:ResultsandUsingimageprocessing,gesturerecognition,andTesseractOCREngine,thesystemdesignedandimplementedarobustalgorithm,whiakesthesystemcanaccurayrecognizethehand-writtenlettersinputtedbytheuserthroughLeapMotion.ThesystemprovidesbasicinformationaboutthestateofLeapMotionandrecognitionresultinacommandwindow.Anditcanreal-timeshowsthetrackofuser’sfinger,whichhelpstheusermoreeasilyinput.Itcsodirectlyandaccurayrecognizesmostofuser’sinputs,becausethesystemhastrainedahand-writtenEnglishletterlibraryforTesseractOCREngine.Inordertoidentifysomeeasilyconfusedletter,like‘e’and‘c’,thesystemcanaccurayidentify,byrecognizingthefeaturesofgesture.Figure16SystemFigure17therecognitionofhand-writtenletterFigure18therecognitionofhand-writtenletterFigure19therecognitionofhand-writtenletterLetter‘c’inputedbyuseriseasilyrecognizedasletter‘e’,ifonlyimageprocessingandTesseractOCREngineareapplied.SothissystemapplysgesturerecognitiontechnologytoidentifytheinputisexactlywhichoneletterOriginally,theinput‘c’wasrecognizedas‘e’.ThenitwascorrectedbygesturerecognitionFigure20therecognitionofhand-