版權(quán)說明:本文檔由用戶提供并上傳,收益歸屬內(nèi)容提供方,若內(nèi)容存在侵權(quán),請進行舉報或認領(lǐng)
文檔簡介
Speci?csofMedicalDataMiningforDiagnosisAid:ASurvey
SarahItania,b,*,FabianLecronc,PhilippeFortempsc
aFundforScienti?cResearch-FNRS(F.R.S.-FNRS),Brussels,Belgium
bFacultyofEngineering,UniversityofMons,DepartmentofMathematicsandOperationsResearch,Mons,Belgium
cFacultyofEngineering,UniversityofMons,DepartmentofEngineeringInnovationManagement,Mons,Belgium
Abstract
Dataminingcontinuestoplayanimportantroleinmedicine;speci?cally,forthedevelopmentofdiagnosisaidmodelsusedinexpertandintelligentsystems.Althoughwecan?ndabundantresearchonthistopic,cliniciansremainreluctanttousedecisionsupporttools.Socialpressureexplainspartlythislukewarmposition,butconcernsaboutreliabilityandcredibilityarealsoputforward.Toaddressthisreticence,weemphasizetheimportanceofthecollaborationbetweenbothdataminersandclinicians.Thissurveylaysthefoundationforsuchaninteraction,byfocusingonthespeci?csofdiagnosisaid,andtherelateddatamodelinggoals.Onthisregard,weproposeanoverviewontherequirementsexpectedbytheclinicians,whoareboththeexpertsandthe?nalusers.Indeed,webelievethattheinteractionwithcliniciansshouldtakeplacefromthevery?rststepsoftheprocessandthroughoutthedevelopmentofthepredictivemodels,thusnotonlyatthe?nalvalidationstage.Actually,againstacurrentresearchapproachquiteblindlydrivenbydata,weadvocatetheneedforanewexpert-awareapproach.Thissurveypaperprovidesguidelinestocontributetothedesignofdailyhelpfuldiagnosisaidsystems.
Keywords:DataMining;Medicine;DiagnosisAid;ExplainableArti?cialIntelligence
1.Introduction
Asoneofthetrendiestresearchtopicsofourcentury,DataMining(DM)makeskeycontribu-tionstothescienti?candtechnologicaladvanceinaconsiderablenumberof?elds(
Gupta
,
2014
;
PhridviRajandGuruRao
,
2014
).Coinedduringthenineties,thedisciplineissubjecttoatoughcompetitionforthedevelopmentofalgorithmsalwaysmorepowerful,whichaimatprocessingdata
*Correspondingauthor.UniversityofMons,DepartmentofMathematicsandOperationsResearch,RuedeHoudain,9,7000Mons,Belgium.
Emailaddresses:sarah.itani@umons.ac.be(SarahItani),fabian.lecron@umons.ac.be(FabianLecron),philippe.fortemps@umons.ac.be(PhilippeFortemps)
2
Numberofpublications
1200
1000
800
600
400
200
0
199019952000200520102015
Year
Figure1:EvolutionoftheannualnumberofpublicationsrelatedtomedicaldataminingintheScopusdatabase(Sco
-
pus
)onaquarterofacentury,from1990to2015
toinfersomeknowledgeintheformofpatternsand/orrelationships(
BellazziandZupan
,
2008
).Theassociatedtechniquesarederivedfromthe?eldsofbothstatisticsandMachineLearning(ML),thelatterwhichaimsatdevelopingcomputationalmethodsabletoextractgeneralizationsfromasetofdata(
Giudici
,
2005
).
MedicalapplicationsfeatureamongtheconcernsoftheDMcommunity,withasigni?cantin-creaseinresearchinterestoverthelastyears(seeFigure
1
).Thisinteractioncomesindi?erentdisciplines(
Bellazzietal.
,
2011
):atthecellularandmolecularlevel(bioinformatics);atthetis-sueandorganlevel(imaginginformatics);atthesinglepatientlevel(clinicalinformatics);atthepopulationandsocietylevel(publichealthinformatics).
Forhalfacenturynow,diagnosispredictionhasbeenaveryactiveresearchareaofclinicalinformatics(
Wagholikaretal
.,
2012
).Inthisregard,withtheadventofDM,researchhasprogres-sivelyshiftedawayfromthestatisticalapproachlongconsideredasastandardpractice.Actually,underahypothetico-deductiveprocess,statisticalanalysesaredriventocheckahypothesisstatedbeforehandanddatasamplesarecollectedforthisspecialpurpose(
Yooetal.
,
2012
).Thisstatis-ticalapproachissurelyadaptedtoraisedi?erencesbetweenpathologicalandcontrolgroups,butnottosetanindividualassessment,i.e.aclinicalexaminationpersubject.Incontrast,enrichedbyMLtechniques,DMinductivelyprocessesavoluminousamountofdata,tobothextractknowledgeanddeveloppredictivemodelsabletohelpindiagnosingpathologies(
Vieiraetal.
,
2017
;
Yooetal.
,
2012
;
BellazziandZupan
,
2008
).Insuchaprocess,statisticsmay?nditsplaceinfeatureengineer-
3
ing,beforethestageofmodelbuildingwhichismainlybasedonMLmethodsofclassi?cationorregression(
Esfandiarietal.
,
2014
).
Inthatrespect,itisthroughdataminingthatrecentworksweredevotedtotheearlydetectionofcancer,e.g.see
LyuandHaque
(
2018
);
Aliˇckovi′candSubasi
(
2017
);
Cichoszetal.
(
2016
);
Nahar
etal.
(
2016
);
Esfandiarietal.
(
2014
);
Krishnaiahetal.
(
2013
);
Parvinetal.
(
2013
);
Guptaetal
.(
2011
).Otherpathologies,suchascardiacandpulmonarydiseases,diabetes,hypertension,meningi-tisformbesidesasigni?cantpartoftheresearchformoreprecisediagnoses(
Esfandiarietal.
,
2014
).Severalpsychiatricdisorders,suchasAttentionDe?citHyperactivityDisorder(ADHD)(
Itanietal.
,
2018a
;
Abrahametal.
,
2017
;
Milhametal.
,
2012
),Alzheimer(
Papakostasetal
.,
2015
),autism(
Kos-
mickietal
.,
2015
),schizophrenia,depressionandParkinson(
Wooetal.
,
2017
)arealsotheobjectofextensiveinvestigation.
Asprobablyperceivedbymostofresearchers,andcertainlybytheauthorsofthepresentpaper,diagnosticdecisionsupportsystemsthathavebeenproposedsofararenotunanimouslyapprovedbyclinicians(
Wagholikaretal
.,
2012
).Suchsystems,andtheunderlyingpredictivemodels,arenotablyfoundasbeingfarfromthe?eldreality.Itisthusmostlikelythatdataminersarenotenoughattentivetothespeci?csofmedicaldiagnosticdecisionsupport.Inparticular,thoughtheDMcommunitywassensitizedaboutthedistinctivenatureofmedicalapplications(
CiosandMoore
,
2002
),thepredictiveperformanceremainspracticallythelonelyparameterwithinthescopeofdataminers,whichencouragescompetition.Thistrendhasbeenaccentuatedwiththegreateravailabilityofopenmedicaldatabases,sharedbydi?erentmedicalandresearchcentersworldwide(
DiMartino
etal.
,
2017
;
Wooetal.
,
2017
;
DiMartinoetal.
,
2014
;
Esfandiarietal.
,
2014
;
Mennesetal.
,
2013
;
Ihleetal.
,
2012
;
Kerretal.
,
2012
;
Milhametal.
,
2012
;
Polineetal.
,
2012
).Someofthesedatasetswerelaunchedattheoccasionofo?cialcontests,e.g.theADHD-200collection(
Milhametal.
,
2012
).Infocusingalmostexclusivelyonperformance,theseresearchworks(1)misschallengesofbetterperceivingandunderstandingtheissuespropertothemedical?eld,(2)areexposedtotheriskofyieldinginconsistentmodels,sincenotably,recentstudiesshowedthattheremaybenologicbehindthepredictionsofaccuratemodels(
Ribeiroetal.
,
2016
).
Itisourstrongconvictionthattheclinicianshavetobeinvolvedinthewholedevelopmentprocessoftheirdecisionsupportsystems.Indeed,theybringexpertiseandknowledgetocontributetointelligentandexpertsystems.Thatiswhy,inthepresentpaper,wewillshedlightuponthespeci?csofmedicaldataminingfordiagnosisaidandraisetherelateddatamodelinggoals.Forsuchapurpose,wewilladdressthefollowingquestions.
4
(1)Howcandecisionsupportmodelsbemoreattractivetoclinicians?Whataretheexpressedrequirementsinthisregard?
(2)Whataretheobjectivescorrespondingtosuchrequirementsintermsofmathematicalmod-eling?
(3)Inwhatwaymedicaldata,particularlyinthiseraofopenmedicaldataproliferation,makesdataminingmorechallenging?
(4)Towhatextentarethecurrentdataminingtechniquesabletosatisfytheclinicians’needsandtohandletheparticularnatureofmedicaldatasimultaneously?
Inansweringthesequestions,weareledtodescribeacomprehensiveexpert-awareapproachwhichstandsoutfromtheexistingliterature,throughthreemaincontributionsexposedbelow.
·Becauseofthelimitede?ectivenessofsomemodels,
Karpatneetal
.(
2017
)pushforatheory-
guideddatascience.SuchDMmodelsaregroundedintheoreticalbases,inthedomainsofPhysicsandChemistrymainly.Inthecontextofmedicaldiagnosis,wecanadoptasimilarapproach,notguidedbytheory,butratherbytheexperts’domainknowledge.Ourpaperlaysthebasesforsuchanapproach,inbuildingakindofbridgebetweenboththemedicalanddataminingdomains.
·Wenotonlyexpressthattheissueofdiagnosisaidisofaparticularnature,wealsopropose
thetranslationoftheassociatedspeci?csintomodelinggoals.Indeed,mostofthepapersthathaveinterestonthespeci?csofthemedicaldomainhaveawidescope,andarethusnotspeci?callyfocusedondiagnosis,butalsoonprognosisandmonitoringnotably,whichinvolvesthatmodelingisnotdiscussedwithenoughdepth(
BellazziandZupan
,
2008
;
Cios
andMoore
,
2002
;
Lavraˇc
,
1999
).Besides,webringamorerecentpointofviewcomparedtothepapersthatspeci?callyaddressedaidedmedicaldiagnosis(
Wagholikaretal
.,
2012
;
Kononenko
,
2001
).
·WedonotprovideanoverviewofDMtechniquesandtherelatedworks;thiswaswidelyproposedinprevioussurveys(
Kalantarietal.
2018
;
Kourouetal.
2015
;
Esfandiarietal.
2014
;
Wagholikaretal
.
2012
;
Yooetal.
2012
).WeratherquestiontheexistingDMtechniques,giventhemodelinggoalsraisedfollowingtheunderstandingoftheproblemanddata.Thisallowsustoraisesomesolidfutureresearchdirections.
5
PREDIcTEDAs>
N
P
Negative(N)
TN
FP
Positive(P)
FN
TP
Figure2:Confusionmatrix
Thepaperisorganizedasfollows.Insection
2
,weexposethematerialsweconsideredtostructureandmakeoursurvey.Theresultsarepresentedinsection
3
anddiscussedinsection
4
.Finally,weconcludethisreportinsection
5
.
2.Materials
2.1.Terminology
Medicaldiagnosisistheresultofachallengingtaskwhichconsistsofcollectingandconciliatingdi?erentinformation(
Donner-Banzho?etal.
,
2017
;
HommersomandLucas
,
2016
;
Miller
,
2016
).Thelatterincludethesymptoms(subjectivedata)andthesigns(objectivedata)ofthetroubleprovidedbyclinicalexaminationsandlaboratorytests.Inquestofexplanationsforthesesymptomsandsigns,theclinicianscometotheconclusionoftheexistence/absenceofatrouble,i.e.thediagnosis.
Atestisoneamongotherelementsthatmotivatesadiagnosis(
Gordis
,
2014
;
CiosandMoore
,
2002
).Thepredictionsofaclinicaltestareofseveraltypes.Apatientwith(respectivelywithout)thediseaseDpredictedassuchisdesignatedastruepositive(resp.truenegative).Incaseofwrongpredictions,thepatientsarefalsepositivesorfalsenegativesrespectively.LetTP(resp.TN)denotethenumberofTruePositives(resp.TrueNegatives)andFP(resp.FN)thenumberofFalsePositives(resp.FalseNegatives);thesequantitiesareusuallyexposedinamatrixofconfusion(seeFigure
2
)(
Wittenetal.
,
2005
).Di?erentscalarmetricsarecomputedfromTP,TN,FPandFNtoassesstheperformanceofclinicaltests;theyareexposedinTable
1
(
LalkhenandMcCluskey
,
2008
;
Akobeng
,
2007a
,
b
).Letusnotethatpositiveandnegativepredictivevaluesdependontheprevalenceofthedisease(
Akobeng
,
2007a
):theyareeasilydeducedfromtheknowledgeofsensitivityandspeci?city,whicharefreefromsuchanin?uence.
Whenseveraltestsarerequiredtocheckthepresenceofamedicalcondition,thesetestsmaybeassessedgloballyintermsofnetsensitivityandnetspeci?city.Thevaluesoftheseindicatorsdependonthewayinwhichthetestswereadministered,i.e.sequentiallyorsimultaneously(
Gordis
,
2014
).Figures
3
and
4
presentthemechanismsofsequentialandparalleltesting.Forillustration
6
Test2(tp2,tn2)
Positive
Test2
(tp2,tn2)
Test2
(tp2,tn2)
METRIc
DEFINITIoN
FoRMuLA
Accuracy(A)
Rateofsuccessfulpredictions
A=TP+TN
TP+FP+TN+FN
Sensitivityor
truepositiverate(tp)
>Abilitytodetectpatientswithagivendisease.
>Probabilitythatapatientwithdis-easetestspositive.
tp=
Speci?cityor
truenegativerate(tn)
>Abilitytodetectpatientswithoutagivendisease.
>Probabilitythatapatientwithoutdiseasetestsnegative.
tn=
PositivePredictiveValue(PPV)
Chancethatapatient,predictedashavingagivendisease,istrulyso.
PPV=
NegativePredictiveValue(NPV)
Chancethatapatient,predictedasfreefromagivendisease,istrulyso.
NPV=
Table1:Performancemetricsofscreeningtests
Negative
Test1
(tp1,tn1)
Figure3:Sequentialtesting
Positive
Negative
Test1
(tp1,tn1)
NegativeNegative
Negative
Test1
(tp1,tn1)
PositiveNegative
Positive
Figure4:Paralleltesting
7
purposes,theexamplepresentsthecaseoftwotests;theassociatedreasoningmaybegeneralizedtosituationsinvolvingmoretests.Incaseofsequentialtesting,apatientissubmittedtoanotherroundofexaminationifhe/shetestedpositive,inordertosettlede?nitelyhis/hermedicalcondition.Ifthepatienttestspositivefollowingasecondroundofexamination,thesubjectisdiagnosedwiththediseaseinquestion.Thus,ifoneofbothtestspresentsanegativeresult,thepatientisconsideredasdisease-free.Theassociatednetsensitivityandspeci?cityareexpressedas:
tp=tp1.tp2andtn=tn1+tn2-tn1.tn2.
Incontrast,incaseofparalleltesting,apatientisconsideredasnegativeoncealltestscon?rmthisconditionsimultaneously.Inthiscase,theassociatednetspeci?cityandsensitivityaregivenby:
tn=tn1.tn2andtp=tp1+tp2-tp1.tp2.
Inthesamewaythatacliniciancanaskfortheopinionofanexpertconfrere,he/shecanresorttomodelsfordiagnosisaid.Theonlydi?erencebetweenbothscenariosrestsontheexternalnatureofthediagnosticsupport,eitherhumanorcomputerized.Thedataofoneorseveraltest(s)arepotentialinputsfordiagnosisaidmodels.Itshouldbenotedthatnon-interpretedoutcomesoftesting(e.g.acholesterollevel,ascan)constitutethemodelinputs,andnotthevalueofthetest(s),i.e.positiveornegative.Actually,itistheroleofthepredictivemodeltodetermineapatient’smedicalconditioninoutput.
Inlightoftheforegoing,inthepresentsurvey,whatwerefertoasamodelisdi?erentfromatest,thelatterbeingapotentialinputoftheformer.Amodelprovidesarecommendationofdiagnosis;atestprovidesaresultthatallows,amongotherpotentialinformation,tomakeadiagnosis.
2.2.Theknowledgediscoveryprocess
Theextractionofknowledgeforthepurposeofdiagnosisaid?tsintoaKnowledgeDiscoveryProcess(KDP).Sinceitspioneerformalizationby
Fayyadetal
.(
1996
),alternativemodelswereproposed,eitheracademically-orindustrially-minded(
KurganandMusilek
,
2006
).Inparticular,theKDPwasadaptedformedicalapplicationsandillustratedfortheissueofdiagnosisaidby
Cios
etal.
(
2007
,
2000
).Theassociatedstepsaresummarizedbelow.
UnderstandingoftheproblemTheprocessisinitiatedbytheproblemstatement,thede?ni-tionoftheobjectives,andthesu?cientappropriationofadomain-speci?cvocabulary.Obvi-
8
ously,interactionswithdomainexpertsareessential.Atthislevel,thechoiceofdataminingtechniquesispartiallyforeseengiventheexpressedrequirements.
UnderstandingofthedataThisstepconsistsofcollectingandexploringdata,i.e.observingandanalyzingtheinformation.
PreparationofthedataThecreationoftargetdatasets(
Fayyadetal
.,
1996
)involvesnotablynoiseremovalaswellascheckingthecompletenessandconsistencyofdata.Then,dataareprocessedthroughengineering,selectionandpossiblereductionofpertinentfeatures.
DataminingThisprocessreceivestheprepareddatasets,andextractsknowledge,i.e.patterns,relationships(
BellazziandZupan
,
2008
).
EvaluationofthediscoveredknowledgeTheresultsarecloselyconsidered:theyareexpectedtobringnewandinterestingelements,tobeunderstoodandtomakesense.Here,domainexpertshavetoplayanimportantroleintheirabilitytointerpretandassesstheresults.
UseofthediscoveredknowledgeItcanleadtoactiontaking,decisionmakingorsystemsde-ployment(
Fayyadetal
.,
1996
).
TheKDPisnotstrictlyaone-wayprocessasitisnotexcludedtoreconsidertheworkofpreviousstages:thisallowstoreinforcetheconsistencyoftheresults(
Ciosetal.
,
2007
).Forexample,the?nalevaluationmayaskforre?ningtheresults.Ortobetterunderstandthedata,are-understandingoftheproblemmaystrengthenthedomain-speci?cknowledge.
2.3.Acceptancecriteria
Onedi?cultyrelatedtomedicalDMisthatitmaytargetdi?erentpublicswiththeresultingnecessitytoaddressdi?erentexpectations.
Actually,aDMapproachmayberequestedinthemedical?eldbyresearchersandspecialistsinordertostudyagivenpathologythroughtheidenti?cationofexplanatoryfactors.Inthatcase,theextractedknowledgeisvalidatedifitcarriesacertainlevelofcredibility,measuredbymeansofcriteriarelatedtostatisticalpowernotably.Ifendorsedbythescienti?ccommunity,suchresultsmaybetakenintoconsideration(directlyorindirectly)bycliniciansfacedwithadiagnosistask.
Assuggestedinsection
2.2
,theextractedknowledgemayalsobedeployedintheformofacomputerizeddiagnosisaid.Despitetheyarethelonelyusersofsuchtechnologies,thecliniciansarein?uencedintheirexpectations,e.g.bythepatientswhoplacealotofhopeinafairdiagnosis.
9
Di?erentmodelsweredevelopedinane?orttoexplainhowaclinicianmayacceptatechnologyandintegrateittohis/herworkingpractices(
Andargolietal
.,
2017
;
Ketikidisetal.
,
2012
;
Holden
andKarsh
,
2010
;
YarbroughandSmith
,
2007
).ThemostpopularistheTechnologyAcceptanceModel(TAM),introducedby
Davisetal.
(
1989
)andrevisedby
VenkateshandDavis
(
2000
)(TAM2).Enjoyedforitsconcisestructure,themodeldepictsthepsychologicalprocesswhich,in?uencedbymaterialandsocialfactors,leadstotheintentionofusingacomputerizedapplicationindi?erent
contexts(
YarbroughandSmith
,
2007
).
VenkateshandDavis
(
2000
)reportthattheacceptanceoftechnologyisacquiredinpracticeonceitsusefulnessandeaseofusearebothperceivedbytheuser.Moreover,theeaseofuseisoneofthefactorsin?uencingtheuser’sperceptionoftheusefulnessoftheapplication.Theperceptionofusefulnessrestsalsoonsocialfactors:thesubjectivenorm,i.e.theuser’s(professionalorprivate)surroundings’opinionregardinghis/herdecisiontoadopt(ornot)theapplication,andtheimage,i.e.thesocialstatustheapplicationprovidestotheuser(
Munetal.
,
2006
;
Chismarand
Wiley-Patton
,
2002
).
Thesubjectivenormimpactsdirectlytheintentionofuse.Thisin?uenceisexertedontheclini-cianbyhis/herpatientsbutalsobytheprofessionalenvironment.Indeed,thephysicianissensitivetotheopinionofconfreres,particularlyofreferencepeopleinthedomain,eventhoughthisopinionmaybecontrarytothephysician’sbeliefs(
Munetal.
,
2006
).Asforthein?uenceofthepatients,thestudyof
Sha?eretal.
(
2013
)showstheyoftentendtodemonizecomputerizeddiagnosticsupport.Conversely,noncomputer-assistedpracticesareperceivedasapledgeofprofessionalism;maytheclinicianresorttotheopinionofanexpertconfrereisevenperceivedasanintelligentact.Yetinbothlastcases,thecliniciansmightbasetheirdecisiononelementsprovidedintheliteratureandextractedfromaDMapproach.Thus,theinvolvementofcomputinginthediagnosticprocess,ifonlytohaveanadvice,wouldinitselfleadthephysician’simagetotakeahittowardscolleaguesand/orpatients(
Munetal.
,
2006
).
Inthepresentsurvey,wewillhighlightthespeci?csofDMtodevelopdiagnosticdecisionsupportmodelswhichmeettherequirementsoftheclinicians.Wewilldealwithhowtomakecomputerizeddiagnosisaidful?llcriteriaofoutputqualityandresultsdemonstrabilityadvocatedbyTAM.Nev-ertheless,itmustberecognizedthatadoptingasuitableapproachofmodelingdoesnotguarantee
exclusivelytheacceptanceofthemodelssincesomerelatedfactors(e.g.subjectivenorm)donotfallwithinDMconcerns.
10
cKnowledgeDiscoveryProcessc
NatureofMedicalData
OverviewofDMTechniques
PerformanceEvaluation
Speci?csofMedicalDM
√
√
√
√
√√
√√
√
√
√
√
√
√
√
√
√√
Selectedtechniquesfordatamininginmedicine
Machinelearningformedicaldiagnosis:history,stateoftheartandperspective
Theuniquenessofmedicaldatamining
Predictivedatamininginclinicalmedicine:currentissuesandguidelines
Introductiontotheminingofclinicaldata
Clinicaldatamining:a
溫馨提示
- 1. 本站所有資源如無特殊說明,都需要本地電腦安裝OFFICE2007和PDF閱讀器。圖紙軟件為CAD,CAXA,PROE,UG,SolidWorks等.壓縮文件請下載最新的WinRAR軟件解壓。
- 2. 本站的文檔不包含任何第三方提供的附件圖紙等,如果需要附件,請聯(lián)系上傳者。文件的所有權(quán)益歸上傳用戶所有。
- 3. 本站RAR壓縮包中若帶圖紙,網(wǎng)頁內(nèi)容里面會有圖紙預(yù)覽,若沒有圖紙預(yù)覽就沒有圖紙。
- 4. 未經(jīng)權(quán)益所有人同意不得將文件中的內(nèi)容挪作商業(yè)或盈利用途。
- 5. 人人文庫網(wǎng)僅提供信息存儲空間,僅對用戶上傳內(nèi)容的表現(xiàn)方式做保護處理,對用戶上傳分享的文檔內(nèi)容本身不做任何修改或編輯,并不能對任何下載內(nèi)容負責(zé)。
- 6. 下載文件中如有侵權(quán)或不適當(dāng)內(nèi)容,請與我們聯(lián)系,我們立即糾正。
- 7. 本站不保證下載資源的準(zhǔn)確性、安全性和完整性, 同時也不承擔(dān)用戶因使用這些下載資源對自己和他人造成任何形式的傷害或損失。
最新文檔
- 激光測距計儀表采購合同范本
- 設(shè)備采購與安裝協(xié)議
- 苗木采購合同格式范文
- 家具選購合同全解析策略
- 高利貸借款合同書樣本格式
- 投資合同協(xié)議撰寫
- 酒店用品供應(yīng)商采購協(xié)議
- 機械設(shè)備采購合同格式模板
- 房屋地基買賣合同模版
- 設(shè)計印刷服務(wù)合同協(xié)議書
- 勞務(wù)派遣用工管理辦法
- 部編人教版道德與法治八年級上冊 引用的名言警句1
- 藏傳佛教因明學(xué)通論
- 監(jiān)控驗收單完整版本
- 2024湖南株洲市天元區(qū)招聘社區(qū)專職工作者筆試歷年典型考題及考點剖析附答案帶詳解
- 弱電智能化工程技術(shù)方案
- TZSA 225-2024 高導(dǎo)熱膜用石墨烯材料應(yīng)用指南
- 第七課《循環(huán)程序》教學(xué)設(shè)計 2023-2024學(xué)年新世紀(jì)版(2018)初中信息技術(shù)八年級上冊
- 人教版八年級音樂上冊 第二單元 《動物世界》片頭曲教案
- 曲式與作品分析智慧樹知到期末考試答案章節(jié)答案2024年內(nèi)蒙古藝術(shù)學(xué)院
- 人工智能與未來教育智慧樹知到期末考試答案章節(jié)答案2024年麗水學(xué)院
評論
0/150
提交評論