非剛性點(diǎn)云配準(zhǔn)在內(nèi)窺鏡光學(xué)相干斷層掃描中用于中耳診斷 Non-rigid Point Cloud Registration for Middle Ear Diagnostics with Endoscopic Optical Coherence Tomography

SpringerNature2021LATEXtemplate

1

arXiv:2304.13618v1[cs.CV]26Apr2023

Non-rigidPointCloudRegistrationfor

MiddleEarDiagnosticswithEndoscopic

OpticalCoherenceTomography

PengLiu1,3*,JonasGolde2,3,Joseph

Morgenstern3,4,SebastianBodenstedt1,ChenpanLi1,Yujia

Hu1,ZhaoyuChen1,EdmundKoch1,2,MarcusNeudert1,4

andStefanieSpeidel1,3

1*TranslationalSurgicalOncology,NationalCenterforTumor

Diseases,Dresden,01307,Germany.

2ClinicalSensoringandMonitoring,TUDresden,Dresden,01307,Germany.

3ElseKr¨onerFreseniusCenter,TUDresden,Dresden,01307,Germany.

4EarResearchCenterDresden,TUDresden,Dresden,01307,Germany.

*Correspondingauthor(s).E-mail(s):peng.liu@nct-dresden.de;

Abstract

Purpose:Middleearinfectionisthemostprevalentin?ammatorydisease,especiallyamongthepediatricpopulation.Currentdiagnos-ticmethodsaresubjectiveanddependonvisualcuesfromanoto-scope,whichislimitedforotologiststoidentifypathology.Toaddressthisshortcoming,endoscopicopticalcoherencetomography(OCT)providesbothmorphologicalandfunctionalin-vivomeasurementsofthemiddleear.However,duetotheshadowofpriorstructures,interpretationofOCTimagesischallengingandtime-consuming.Tofacilitatefastdiagnosisandmeasurement,improvementintheread-abilityofOCTdataisachievedbymergingmorphologicalknowledgefromex-vivomiddleearmodelswithOCTvolumetricdata,sothatOCTapplicationscanbefurtherpromotedindailyclinicalsettings.

SpringerNature2021LATEXtemplate

2Non-rigidPointCloudRegistrationforMiddleEarDiagnostics

Methods:WeproposeC2P-Net:atwo-stagednon-rigidregistra-tionpipelineforcompletetopartialpointclouds,whicharesam-pledfromex-vivoandin-vivoOCTmodels,respectively.Toover-comethelackoflabeledtrainingdata,afastande?ectivegenerationpipelineinBlender3Disdesignedtosimulatemiddleearshapesandextractin-vivonoisyandpartialpointclouds.Results:WeevaluatetheperformanceofC2P-NetthroughexperimentsonbothsyntheticandrealOCTdatasets.TheresultsdemonstratethatC2P-NetisgeneralizedtounseenmiddleearpointcloudsandcapableofhandlingrealisticnoiseandincompletenessinsyntheticandrealOCTdata.Conclusion:Inthiswork,weaimtoenablediagnosisofmiddleearstructureswiththeassistanceofOCTimages.WeproposeC2P-Net:atwo-stagednon-rigidregistrationpipelineforpointcloudstosupporttheinterpretationofin-vivonoisyandpartialOCTimagesforthe?rsttime.Codeisavailableat:/ncttsopublic/c2p-net.

Keywords:MiddleEar,OpticalCoherenceTomography,PointCloud,DeeplearningbasedRegistration

1Introduction

Themiddleearconsistsofthetympanicmembrane(TM)andanair-?lledchambercontainingtheossiclechain(OC)thatconnectstheTMtotheinnerear.Fromafunctionalperspective,themiddleearmatchestheimpedancefromairtothe?uid-?lledinnerear[

1

].Middleeardisorderscontaindeforma-tion,discontinuationofTMandOC,e?usion,andcholesteatomainthemiddleear.Thesemayoccurbecauseofmiddleearinfection(e.g.acuteotitismedia(AOM),chronicotitismedia(COM),otitismediawithe?usion(OME))andtrauma[

2

,

3

].Mostcommonly,seriousandchronicmiddleeardisordermayleadtoconductivehearinglossandinnereardisorder

[4].

Currentmiddleeardiagnosticsmethodsortoolscoveronlyaspectsofthepathologyandcannotdeterminetheoriginandsiteoftransmissionloss,e.g.otoscopy,audiome-try,tympanometry,etc.Asaninnovativeimagetechnology,endoscopicOCTenablestheexaminationofboththemorphologyandfunctionofthemiddleearin-vivobythenon-invasiveacquisitionofdepth-resolvedandhigh-resolutionimages[

5

–

7

].Nevertheless,multiplesourcesofnoise,e.g.theshadowofpre-cedingstructures,reducethequalityofthetargetstructuresfurtherawayfromtheendoscopicprobe,e.g.ossicles.Therefore,thereconstructed3Dmod-elsfromtheOCTvolumetricdataareusuallynoisyanddi?culttointerpret,especiallytheidenti?cationofthedeepermiddleearstructuressuchasincusandstapes(seeFig.

1

).

OuraimistoimprovetheinterpretationoftheOCTvolumetricdatabymergingtheex-vivomiddleearmodelreconstructedfrommicro-ComputedTomography(μCT)scanofisolatedtemporalbonesandthein-vivoOCT

SimulationinBlender3D

non-rigid

rigid

uCTscans

a)

OCTscans

SpringerNature2021LATEXtemplate

Non-rigidPointCloudRegistrationforMiddleEarDiagnostics3

I

II

III

I

II

III

c)

b)

Fig.1:Overviewofdatasamples.Ina),Iistheex-vivotemplatepointcloudreconstructedfromμCTscansontheleft.IIisthesegmentedmiddleearmodel,eachcolorstandsforonestructure.IIIshowsthemodelwithsparselabels.Accordingly,I,IIandIIIinb)presentin-vivoOCTpointcloud,segmentedOCTmodelandlabels.Thesimulationpipelineonthetoplefttakesacompleteex-vivoearpointcloud,performsnon-rigidandrigidsimulation,andproducessyntheticpartialandnoisyin-vivopointcloudshowninc).

model.NotethatasingleμCTmodelwasusedasatemplateand?ttedtoallpatient-speci?cOCTscans.We?rstconvertallex-andin-vivomiddleeardatatopointcloudrepresentationsforeaseof?exiblemanipulationofmid-dleearshapes.Nevertheless,?ndingone-to-onecorrespondencebetweensuchapointcloudpairisstillchallengingduetothenoiseandincompletenessoftheOCTmodel,andthedi?erencebetweenthepatientdataandthetem-plateμCTdata.Totackletheseissues,we?rstemployaneuralnetworkthatsearchesthesparsecorrespondencesforthesourceandtargetpoints,thenapyramiddeformationalgorithmto?tthepointsinanon-rigidfashionbasedonthepredictedcorrespondences.Theneuralnetworkistrainedonrandomlygeneratedsyntheticshapevariantsofthemiddleearthatcontainrandomnoiseandonlypartialpoints.ThisenablesthegeneralizationoftheneuralnetworktonewpatientdataaswellastheadaptiontonoiseandoutliersThemaincontributionsofthisworkarelistedbelow:

1.AgenerationpipelineinBlender3Dthatsimulatessyntheticshapevariantsfromacompleteex-vivomiddleearmodel,andsimulatesnoisyandpartialpointcloudscomparabletoin-vivodata.

2.C2P-Net:atwo-stagednon-rigidregistrationpipelineforpointclouds.ItdemonstratesthatC2P-Netregistersthecompletepointcloudtemplatetopartialpointcloudsofthemiddleeare?ectivelyandrobustly.

SpringerNature2021LATEXtemplate

4Non-rigidPointCloudRegistrationforMiddleEarDiagnostics

2RelatedWork

Inthissection,wereviewmethodsthatanalyze3Dpointcloudsandsolvereg-istrationproblems.Learningthegeometryof3Dpointcloudsisthefoundationofdiverse3Dapplications,butisalsoachallengingtaskduetotheirregularityandasymmetryofpointclouds.Recently,approachesthatfocusonlearningpointfeatureswithneuralnetworkshavebeenwidelyinvestigatedtoovercomesuchissues.Onecategoryofmethodsistoprojectthepointcloudtoaregularrepresentation,e.g.voxelgrids[

8

],2Dimagesfrommulti-views[

9

,

10],

orcom-bined[

11

]that2/3DconvolutionaloperationscanbeperformedontopoftheseintermediatedatainEuclideanspace.Apartfromthese,PointNet

[12]

lever-agesmultilayerperceptron(MLP)toobtainpointwisefeaturesandamax-pooltoaggregateglobalfeaturesbutwithoutlocalinformation.ThisdrawbackisalleviatedbyPointNet++[

13

],whichusesPointNettoaggregatelocalfeaturesinamulti-scalestructure.Inaddition,variousexplicitconvolutionkernelsareapplieddirectlyonthepointcloudstoextractencodings[

14

–16]

inwhichpointfeaturesareextractedbasedonthekernelweights.

Conventionalmethodssolvethepointcloudregistrationtaskasanoptimizationproblemoftransformationparameters.IterativeClosestPoint(ICP)[

17

]iterativelycalculatesarigidtransformationmatrixbasedonupdatedcorrespondencesetfromalastiteration,Non-rigidIterativeClosestPoint(NICP)[

18

]achievesnon-rigidregistrationbyoptimizinganenergyfunc-tionincludinglocalregularisationandsti?ness.CoherentPointDrift(CPD)maximizestheprobabilityofGaussianMixtureModel(GMM)bymovingcoherentGMMcentroidsfromthetwopointclouds.Recently,deeplearning-basedmethodsusingtheaforementionedpointcloudencodingtechniqueshavebeenwidelyinvestigated[

19

–

23

].PointNetLK[

19

]extractssemanticpointcloudfeaturesusingMLP,thensolvesrigidalignmentusingtheLucasandKanadealgorithm[

24

].PR-Net[

20

]followsasimilarwayoffeatureextractionwhileitmapsthefeaturetoregulargridsandde?nestheregistrationtaskasacorrela-tionproblem.RegTR[

21

]andNgeNet[

22

]bothintroduceattentionmechanismstoenablethecommunicationoftheextractedpointcloudfeaturesfromthesourceandtargetandestimateaglobaltransformationfromthepredictedcorrespondenceswithMLPs.Incontrasttothisdirectway,NDP

[23]

decom-posestheglobaltransformationintosub-motionswhichareiterativelyre?nedusingMLPsateachpyramidlevel.Inspiredbythisrecentworkregardingpointcloudanalysisandregistration,weadoptedNgeNet[

22

]andNDP

[23]

asthemaincomponentstoconstructourregistrationpipeline.

3Methods

Thenon-rigidregistrationoftheex-andin-vivomiddleearpointclouds(seeFig.

1

)usingneuralnetworksisrealizedinasupervisedlearningfashion.DuetolackofrealOCTdatawithgroundtruth,andthedi?cultiesof?ndingone-to-onecorrespondencesbetweenthegeneralandex-vivomodelandthenoisyandpartialin-vivomodel,wegeneratedsyntheticin-vivomiddleearpointclouds

SpringerNature2021LATEXtemplate

Non-rigidPointCloudRegistrationforMiddleEarDiagnostics5

basedontheex-vivoμCTmodelusingatwo-stepsimulationpipeline.Forpointcloudregistration,weproposeC2P-Net,whichisatwo-stageregistrationmethod.We?rsttrainaneuralnetworkonthesyntheticsamplestoexplorethecorrespondencealongwithaninitialrigidtransformationmatrix,thenbasedonthis,ahierarchicalalgorithmisperformedtoestimatethenon-rigiddeformation.

3.1SyntheticDataGeneration

WestartwithacompletemiddleearmodelMμCTreconstructedfromμCTvolumeasbasisforthesimulationinBlender3D.Thenon-rigidsimulationisperformedwiththeassistanceoflatticemodi?ersattachedtoeachstructurewithreasonableresolution.Thelatticeverticesareassignedtovariousgroupsaccordingtolength,thickness,andwidthoftheglobalandlocalshapeofastructure.Bytransformingandrotatingvariousverticegroupswithrandomparameterswithboundaryconditions,arandomshapeofastructurecanbegenerated,e.g.arelativelyshortermalleuswithalongerlateralprocess.Thenon-rigidsimulationstepisformulatedasnr=Snr(T,pnr),whereTisthetemplatemiddleear,pnraretheinputparameters,andnristhesimulatednon-rigidshapevariantsofT.Next,armaturemodi?ersareattachedtoeachstructureandconnectedinanend-to-endfashionatthearticulations.Then,rigidsimulationisaccomplishedbytransformingandrotatingthearmaturebones.WedenotetherigidsimulationasSr.Thus,arandomshapevariantofthemiddleearcanbeobtainedwhichisconsideredasthein-vivoearmodelofanewpatient:=Sr(nr,pr).

Inpractice,thein-vivoOCTmodelisusuallynoisyandonlypartiallyvisible,whichlimitstheregistrationmethodsin?ndingaccurateandrobustcorrespondences.Tosimulatetherealdata,weextractrandompartialpatchesforeachstructureoftheshapevariantbyacombinedprobability,whichiscalculatedasthedistanceofeachvertextothesupportpointofeachstructureandrandomGaussiannoise.Additionally,forthesakeofsimulatingincompleteposteriorstructure,e.g.stapes,causedbytheocclusionandshadowfromtheanteriorstructures,e.g.earcanalwall,wedeterminethenumberofpointsoftheposteriorstructuresbythedistancetotheexternalearandarandomfactor.Furthermore,uniformrandomdisplacementsareappliedtotheverticestogeneratethe?nalnoisyandpartialin-vivopointcloudPinv(seeFig.

1

).

3.2C2P-Net

WedelineatethearchitectureofC2P-NetinFig.

2

.Itconsistsoftwocompo-nentsdedicatedtotwostages:initialrigidregistrationandpyramidnon-rigidregistration.Givenanex-vivopointcloudasatemplateextractedfromaμCTmodel:Pexv={xieR3}i=1,2,...,N,andapartialpointcloudofthesimulatedin-vivoshapevariant:Pinv={yjeR3}j=1,2,...,M,weadoptedtheNeighborhood-awareGeometricEncodingNetwork(NgeNet)[

22

]tosolvetheinitialrigidregistrationtask.Thisstageisformulatedas:

CorrespondenceExploration

Shareddecoder

Sharedencoder

exp

KPCo

nv

PConv

exp

K

inp

SpringerNature2021LATEXtemplate

6Non-rigidPointCloudRegistrationforMiddleEarDiagnostics

Crossattention

DeformationEstimation

…

rigid

non-rigid

Fig.2:SchematicarchitectureofC2P-Net.Theinputofthepipelineistheex-vivo(blue)andin-vivo(orange)pointcloud.Toexplorethecorrespondencebetweenthetwoinputs,NgeNetistrainedonsyntheticsamplestoextractthefeaturesforeachpointatdi?erentlevels.Theseareintegratedbyavot-ingalgorithm.Afeaturematchingroutineisperformedtoobtainthesparsecorrespondencesetaswellasarigidtransformationmatrix.Ontopofthepreviousoutputs,NDPmapstheinputpointcloudstosinusoidalspaceanddecomposestheglobalnon-rigiddeformationateachpyramidlevelintosub-motions.Withtheincreaseofsinusoidalfrequency,thenon-rigiddeformationofeachpointiscontinuouslyre?nedateachlevel.Theregisteredpointcloudisshownontherightside.

τ,σ=NgeNet(Pexv,Pinv)where(u,v)eσ(1)

whereτeSE(3)istherigidtransformationmatrixwhichalignsPexvwithPinv,and(u,v)eσisthesparsecorrespondencesetwhereuandvareindicesforpointsinPexvandPinv.Duetothemulti-scalestructureandavotingmech-anismintegratingfeaturesfromdi?erentresolutions,NgeNethandlesnoisewellandpredictscorrespondencerobustly.

Basedonthepreviouspredictedcorrespondencesetandtherigidlyalignedsourceandtargetpointclouds,weemploytheNeuralDeformationPyra-mid(NDP)[

23

]topredictthenon-rigiddeformationofthegivenpointcloudpair.NDPde?nesthenon-rigidregistrationproblemasahierarchicalmotiondecompositionproblem.Ateachpyramidlevel,theinputpointsfromlastlevelaremappedtosinusoidalencodingswithdi?erentfrequencies:Γ(pk-1)=(sin(2k+kepk-1),cos(2k+kepk-1)),kisthecurrentlevelnumber,k0controlstheinitialfrequencyandpk-1isanoutputpointfromthelastlevel.Lowerfrequen-ciesatshallowerlevelsrepresentrigidsub-motion,whilehigherfrequenciesatdeeplevelsemphasizenon-rigiddeformations.Inthisway,asequenceofsub-motionsisestimatedfromrigidtonon-rigid,andthe?naldisplacement?eldisthecombinationofsuchasequence.Formally,wedenotethestageas:

φest=NDPn,m(exv,Pinv,σ)(2)whereexvisthesourcepointcloudPexvtransformedbyτ,nisthenumberofpyramidlayersoftheNDPneuralnetwork,misthemaximaliterationwithinasinglepyramidlayer,andφestisthepredicteddisplacement?elddescribinghoweachpointshouldmovetothetarget.Combinedlossesarecalculatedat

SpringerNature2021LATEXtemplate

Non-rigidPointCloudRegistrationforMiddleEarDiagnostics7

eachiteration,includingcorrespondencelossandregularizationloss,andback-propagatedtoupdatetheweightsofeachMLP.Ofwhich,thecorrespondencelossLCDisde?nedastheChamferdistance(

3

)betweenexvwhichismaskedbythecorrespondenceσandPinv.

CD(A,B)=

IAIyj_BIBIxi_A

1minIxi_yjI+1minIxi_yjI

xi_Ayj_B

LCD=CD(v,Pinv)

(3)

(4)

v={exv[u]I3v:(u,v)eσ}arethemaskedex-vivopointsthathavecorrespondenceinthetargetin-vivopointcloud.

4Evaluation

C2P-Netpredictsadisplacement?eldto?ttheex-vivopointcloudtem-platetothetargetin-vivopointcloudwhichisnoisyandpartial.Attrainingtime,C2P-NetemploysanAdamoptimizerandExpLRschedulerandusessupervisedlearningtolearnonasyntheticdatasetwith20,000in-vivoshapevariantsofthemiddleear.Duringinference,ittakesaround3.5sforC2P-Nettoreacttoanewin-vivoshapeonRTX2070Super.Sincetheneuralnetworkistrainedonsyntheticdata,itiscrucialtoinvestigatethegeneralizationoftheneuralnetworksonunseensamplesaswellastheperformanceonrealOCTdata.Thus,weconducttwoexperimentsonsyntheticdataandrealOCTdatawithvariousmetrics:meandisplacementerror(

5

),Chamferdistance(

3

),andlandmarkserror(

6

).Furthermore,wecompareourresultswithotherpopu-larnon-rigidregistrationmethods,e.g.Non-rigidICP(NICP)andCoherentPointDrift(CPD).

N

MMDE=|φest_φgt|(5)

i

Nisthesizeofthetestdataset,φestistheestimateddisplacement?eldofasample,andφgtisthecorrespondinggroundtruth.

InSilico.Asyntheticdatasetisgeneratedusingthepipelinedescribedinsection

3.1

withthesameboundaryconditions,andthetargetshapevariantsareunseentotheneuralnetworkduringtraining.Foreachtargetshape,weestimatedthedisplacement?eldfortheidenticaltemplatepointcloudusingC2P-Nettrainedon20,000samples.Themeantargetdisplacement?eldforthesyntheticdatasetis1.54mm.Fig.

3a

illustratesthattheneuralnetworktendstopredictthedeformationoftheex-vivopointcloudwithhigheraccuracyifitobtainsmoreinformationaboutthetargetin-vivoOCTpointcloud.Withfewexceptions,Fig.

3b

showswhentheinitialregistrationerrormadebyNgeNetislow,the?nalglobalnon-rigiddisplacement?eldfromNDPtendstobesmall.

meandisplacementerror(mm)

meandisplacementerror(mm)

SpringerNature2021LATEXtemplate

8Non-rigidPointCloudRegistrationforMiddleEarDiagnostics

Table1:Experimentsresultsonsyntheticdata(SYN)andrealOCTdata(REAL),includingmeandisplacementerror(MMDE)[mm],landmarkserror(ML)[mm]andChamferdistance(MCD)[mm]

SYN

REAL

MMDE↓

MCD十

ML↓

ML↓

MCD十

ICP*

2.01

1.85

1.15

4.74

3.03

NICP

461

0.22

0.87

4.39

0.68

CPD

8.79

1.07

0.96

2.17

0.059

C2P-Net(ours)

0.781

0.582

0.424

0.808

2.43

*rigidregistrationmethod(s).

十baselinemethodstendtofallintolocalminimaandignorethegeometry,thoughtheCDislower.

3.0

1.6

1.4

2.5

1.2

2.0

1.0

1.5

0.8

1.0

0.6

0.5

0.4

0.0

0.2

0.300.320.330.340.360.380.390.40

visiblepointsratio

0.250.500.751.001.251.501.752.00

initialregistrationerror(mm)

(a)

(b)

Fig.3:a)showstherelationbetweentheMDEofsyntheticsamplesandthevisiblepointsratio,whichiscalculatedastheratiobetweenthepointnum-berofatargetpointcloudtothecorrespondinggroundtruthpointcloud:IPinvI/II.Thedecreasingtrendshowsthatwithmorepointsavailableinthetargetin-vivopointclouds,theneuralnetworktendstopredictbetterdefor-mation.b)depictstheMDEtotheinitialrigidregistrationerrorwhichisproducedbytheNgeNet.Thebettertheinitialrigidregistrationandcorre-

spondencesetpredictedbyNgeNet,thelowertheglobalnon-rigidregistrationerrorfromNDPbecomes.

InVivo.ArealOCTdatasetwith9samplesofthehumanmiddleearwascollectedwithahandheldOCTimagingsystem[

5

]andsegmentedmanuallybysurgeonswithin3DSliceralongwithasetofsparselandmarksforeachstructureofasample(seeFig.

1

).Sincethereisnogroundtruthcorrespon-denceavailableforthesourceandtargetpointcloudstocalculateMMDE,weintroduceanothermetricbasedonthesparselandmarks:

īAРNīAРAРDC2P-Net(ours)

SpringerNature2021LATEXtemplate

SYN

RE2j1

RE2j2

Non-rigidPointCloudRegistrationforMiddleEarDiagnostics9

Fig.4:RegistrationresultsofbaselinemodelsandC2P-Netonsynthetic(SYN)andrealOCTsamples(REAL).Bluepointcloudsarethedeformedex-vivoearmodels,i.e.theshapetemplate,calculatedbyalltheinvestigatedmethods.Theorangepointcloudsstandfortheregistrationtarget,i.e.in-vivopointclouds.Fromthevisualization,wecanseebaselinenon-rigidmethods,NICPandCPD,tendtosqueezethepointsfromtheex-vivotemplatelargelytomatchthetargetpointclouds,sincetheyonlyfocusonthepositionoflocalpoints,whileC2P-Netretainstheglobalgeometryofthemiddleearduetothelearnedcorrespondenceknowledgebetweentheex-andin-vivopointclouds.

ML=1min|li,k_lj,k|k=1,2,...,K(6)

ILexvIli,k_Lexvlj,k_Linv

whereLexvarethelandmarksontheex-vivopointcloudfromμCT,whichisalsomarkedmanually,Linvarethecorrespondinglandmarksonthetargetin-vivopointcloud,andthelandmarkpointsbelongtoKdi?erentsegmentations. Tab.

1

itemizestheregistrationresultsofC2P-Netandbaselinemodelsonbothdatasets.WecanobserveourregistrationpipelineoutperformstheothermethodsintermsofMMDEandML.However,baselinemodelstendtohavesmallerMCD,whichmeansthesourcepointcloudsaredeformedlargelyto?tthetargetregardlessoftheanatomicalgeometryofthemiddleear.Furthermore,thisissueisdemonstratedvisuallyinFig.

4

,whichdepictsseveralregistrationresultsfromC2P-NetandbaselinemodelsonrealOCTsamples.Hence,itismanifestedthatC2P-Netisabletoregisterthepartialtargetpointcloudwithoutlosingtheunderlyinggeometryofsourcepointclouds,whichisthecriticalinformationclinicianswanttoobtainwithregistration.Onthe

SpringerNature2021LATEXtemplate

10Non-rigidPointCloudRegistrationforMiddleEarDiagnostics

contrary,thebaselinemodelsonlyfocusonthespatialpositionoflocalpointsregardlessoftheneighbouringandglobalgeometryofthemiddleeartemplate.

5Conclusion

Inthiswork,weproposetoimprovetheinterpretationofOCTdataforsurgicaldiagnosisbyfusingtheknowledgefromex-vivoandgeneralμCTdatawiththein-vivonoisyandincompleteOCTdata.Tothisend,weproposeC2P-NetwhichisbasedonNgeNetandNeuralDeformationPyramid.Ittakestheex-andin-vivo3Dpointcloudsasinputandexploresthesparsecorrespondencebetweenthetwo,thenalignstheminanon-rigidfashion.Duetothelackoftrainingdata,afastande?ectivesyntheticsimulationpipelinewasdesignedusingBlender3D,whichproducesnoisyandpartialpointcloudsasseenin-vivobasedontherandomlygeneratedshapevariantsofthemiddleear.Ourneuralnetworkwastrainedbasedon20,000syntheticsamples,andittookaround3.5stopredictthedisplacement?eldforagivenin-vivopointcloudatinferencetime.

ToassesstheperformanceofC2P-Net,experimentsonsyntheticandrealOCTdatasetswerecarriedout.SincetherealOCTdataisnoisyandincom-plete,weinvestigatedvariousmetricsincludingmeandisplacementerror,landmarkerror,andChamferdistanceandcomparedC2P-Nettotheotherbaselinemodelsfrombothstatisticsandgeometryaspects.FromTab.

1

andFig.

4

,wecanseethatC2P-NetisabletodealwiththepartialOCTdataandretaintheanatomicalstructureduringnon-rigidregistration,whileothernon-rigidmethodsdonotunderstandtheglobalgeometryofthepointcloudsandfocusonlyonlocalpoints.

Futureworkwillfocusonimprovingregistrationaccuracy.Thiscanbeachievedbyimprovingthesimulationpipelinewithrealisticnoisetofurtherbridgethegapbetweensyntheticandrealdata.Furthermore,thesegmentationinformationcanbeemployedbythenetworktoimprovethepredictionofcorrespondences.Apartfromthis,inferencetimecanbefurtherdecreasedbyexploringtheoptimalcon?gurationofthenetworkfortheOCTsamples.

Acknowledgments.FundedbytheElseKr¨onerFreseniusCentreforDigital

Health.

References

[1]Zwislocki,J.:Normalfunctionofthemiddleearanditsmeasurement.

Audiology21(1),4–14(1982)

[2]Won,J.,Monroy,G.L.,Dsouza,R.I.,Spillman,D.R.,McJunkin,J.,

Porter,R.G.,Shi,J.,Aksamitiene,E.,Sherwood,M.,Stiger,L.,Boppart,S.A.:HandheldBriefcaseOpticalCoherenceTomographywithReal-TimeMachineLearningClassi?erforMiddleEarInfections.Biosensors