市場中介和資產(chǎn)管理者利用人工智能和機(jī)器學(xué)習(xí)

Theuseofartificialintelligenceandmachinelearningbymarketintermediaries

andassetmanagers

FinalReport

TheBoard

OFTHE

InternationalOrganizationofSecuritiesCommissions

FR06/2021

September2021

PAGE\*roman

iii

Copiesofpublicationsareavailablefrom:

TheInternationalOrganizationofSecuritiesCommissionswebsite

?InternationalOrganizationofSecuritiesCommissions2021.Allrightsreserved.Briefexcerptsmaybereproducedortranslatedprovidedthesourceisstated.

Contents

Chapter

Page

1

Executivesummary

1

2

Backgroundandscope

4

3

HowfirmsareusingAIandMLtechniques

6

4

IdentifiedrisksandharmsposedbytheuseofAIandML

9

5

Firms’responsetothepotentialrisksarisingfromtheuseofAIandML

14

6

Guidance

17

A1

HowregulatorsareaddressingthechallengescreatedbyAIandML

22

A2

Guidancepublishedbysupranationalbodies

34

A3

FeedbackStatement

38

PAGE

27

Chapter1-ExecutiveSummary

Background

ArtificialIntelligence(AI)andMachineLearning(ML)areincreasinglyusedinfinancialservices,duetoacombinationofincreaseddataavailabilityandcomputingpower.TheuseofAIandMLbymarketintermediariesandassetmanagersmaybealteringfirms’businessmodels.Forexample,firmsmayuseAIandMLtosupporttheiradvisoryandsupportservices,riskmanagement,clientidentificationandmonitoring,selectionoftradingalgorithmsandportfoliomanagement,whichmayalsoaltertheirriskprofiles.

Theuseofthistechnologybymarketintermediariesandassetmanagersmaycreatesignificantefficienciesandbenefitsforfirmsandinvestors,includingincreasingexecutionspeedandreducingthecostofinvestmentservices.However,thisusemayalsocreateoramplifycertainrisks,whichcouldpotentiallyhaveanimpactontheefficiencyoffinancialmarketsandcouldresultinconsumerharm.Theuseof,andthecontrolssurrounding,AIandMLwithinfinancialmarketsis,therefore,acurrentfocusforregulatorsacrosstheglobe.

IOSCOidentifieditsworkontheuseofAIandMLbymarketintermediariesandassetmanagersasakeypriority.TheIOSCOBoardapprovedamandateinApril2019forCommittee3onRegulationofMarketIntermediaries(C3)andCommittee5onInvestmentManagement(C5)toexaminebestpracticesarisingfromthesupervisionofAIandML.

1

Thecommitteeswereaskedtoproposeguidancethatmemberjurisdictionsmayconsideradoptingtoaddresstheconductrisksassociatedwiththedevelopment,testinganddeploymentofAIandML.

PotentialrisksidentifiedintheConsultationReport

IOSCOsurveyedandheldroundtablediscussionswithmarketintermediariesandconductedoutreachtoassetmanagerstoidentifyhowAIandMLarebeingusedandtheassociatedrisks.ThefollowingareaswerehighlightedintheConsultationReportreleasedinJune2020

2

wherepotentialrisksandharmsmayariseinrelationtothedevelopment,testinganddeploymentofAIandML:

Governanceandoversight;

Algorithmdevelopment,testingandongoingmonitoring;

Dataqualityandbias;

Transparencyandexplainability;

Outsourcing;and

Ethicalconcerns.

1 BoardPriorities-IOSCOworkprogramfor2019,March25,2019,availableat:

/library/pubdocs/pdf/IOSCOPD625.pdf

2 Theuseofartificialintelligenceandmachinelearningbymarketintermediariesandassetmanagers,IOSCOBoardConsultationReport,June2020,availableat:

/library/pubdocs/pdf/IOSCOPD658.pdf

IOSCOGuidance

BasedontheresponsesreceivedtotheConsultationReport,thisfinalreportprovidesguidancetoassistIOSCOmembersinsupervisingmarketintermediariesandassetmanagersthatutiliseAIandML.

TheguidanceconsistsofsixmeasuresthatreflectexpectedstandardsofconductbymarketintermediariesandassetmanagersusingAIandML.Althoughtheguidanceisnotbinding,IOSCOmembersareencouragedtoconsiderthesemeasurescarefullyinthecontextoftheirlegalandregulatoryframeworks.IOSCOmembersandfirmsshouldalsoconsidertheproportionalityofanyresponsewhenimplementingthesemeasures.

TheuseofAIandMLwilllikelyincreaseasthetechnologyadvances,anditisplausiblethattheregulatoryframeworkwillneedtoevolveintandemtoaddresstheassociatedemergingrisks.Therefore,thisreport,includingthedefinitionsandguidance,maybereviewedinthefuturetoremainuptodate.

Measure1:Regulatorsshouldconsiderrequiringfirmstohavedesignatedseniormanagementresponsiblefortheoversightofthedevelopment,testing,deployment,monitoringandcontrolsofAIandML.Thisincludesadocumentedinternalgovernanceframework,withclearlinesofaccountability.SeniorManagementshoulddesignateanappropriatelyseniorindividual(orgroupsofindividuals),withtherelevantskillsetandknowledgetosignoffoninitialdeploymentandsubstantialupdatesofthetechnology.

Measure2:RegulatorsshouldrequirefirmstoadequatelytestandmonitorthealgorithmstovalidatetheresultsofanAIandMLtechniqueonacontinuousbasis.ThetestingshouldbeconductedinanenvironmentthatissegregatedfromtheliveenvironmentpriortodeploymenttoensurethatAIandML:

behaveasexpectedinstressedandunstressedmarketconditions;and

operateinawaythatcomplieswithregulatoryobligations.

Measure3:Regulatorsshouldrequirefirmstohavetheadequateskills,expertiseandexperiencetodevelop,test,deploy,monitorandoverseethecontrolsovertheAIandMLthatthefirmutilises.Complianceandriskmanagementfunctionsshouldbeabletounderstandandchallengethealgorithmsthatareproducedandconductduediligenceonanythird-partyprovider,includingonthelevelofknowledge,expertiseandexperiencepresent.

Measure4:Regulatorsshouldrequirefirmstounderstandtheirrelianceandmanagetheirrelationshipwiththird-partyproviders,includingmonitoringtheirperformanceandconductingoversight.Toensureadequateaccountability,firmsshouldhaveaclearservicelevelagreementandcontractinplaceclarifyingthescopeoftheoutsourcedfunctionsandtheresponsibilityoftheserviceprovider.Thisagreementshouldcontainclearperformanceindicatorsandshouldalsoclearlydeterminerightsandremediesforpoorperformance.

Measure5:RegulatorsshouldconsiderwhatlevelofdisclosureoftheuseofAIandMLisrequiredbyfirms,including:

RegulatorsshouldconsiderrequiringfirmstodisclosemeaningfulinformationtocustomersandclientsaroundtheiruseofAIandMLthatimpactclientoutcomes.

RegulatorsshouldconsiderwhattypeofinformationtheymayrequirefromfirmsusingAIandMLtoensuretheycanhaveappropriateoversightofthosefirms.

Measure6:RegulatorsshouldconsiderrequiringfirmstohaveappropriatecontrolsinplacetoensurethatthedatathattheperformanceoftheAIandMLisdependentonisofsufficientqualitytopreventbiasesandsufficientlybroadforawell-foundedapplicationofAIandML.

Chapter2-BackgroundandScope

PreviousIOSCOworkinthisarea

IOSCOhasundertakenseveralworkstreamsontheuseofAIandMLinfinancialmarkets,including:

CommitteeonEmergingRisks(CER):TheCERundertookamandateontheuseofnoveltechnologiesdeployedbyregulatorstoincreasetheefficiencyandeffectivenessofsupervisoryandoversightprogramsandpublishedareportinFebruary2017.

3

CERexaminedtheregulatoryuseoftoolssuchasbigdataanalyticsanddatavisualisationtechnologies;AIandML,anddeeplearningtechnologies;anddistributedledgertechnologies.

CommitteeonRegulationofSecondaryMarkets(C2):C2publishedareportinApril2013onTechnologicalChallengestoEffectiveMarketSurveillanceIssuesandRegulatoryTools.

4

Thereportmaderecommendationstohelpmarketauthoritiesaddressthetechnologicaldifficultiesfacingeffectivemarketsurveillance.

IOSCOFintechNetwork:TheIOSCOFintechNetworkwasestablishedinMay2018tofacilitatethesharingofknowledgeandexperiencesamongIOSCOmembers.TheIOSCOFintechNetworkconsideredtheethicalimplicationsoftheuseofAIandMLtechnologies.

IOSCOMandate

Buildingonitspreviouswork,IOSCOreleasedaConsultationReportontheuseofAIandMLbymarketintermediariesandassetmanagersinJune2020,proposingguidancetoaddressthepotentialrisksandharmsthatmaybecausedbytheuseofAIandMLbythesemarketintermediariesandassetmanagers.Theproposedguidancelookedtohelpensurethatmarketintermediariesandassetmanagershave:

appropriategovernance,controlsandoversightframeworksoverthedevelopment,testing,useandperformancemonitoringofAIandML;

staffwithadequateknowledge,skillsandexperiencetoimplement,oversee,andchallengetheoutcomesoftheAIandML;

robust,consistentandclearlydefineddevelopmentandtestingprocessestoenablefirmstoidentifypotentialissuespriortofulldeploymentofAIandML;and

appropriatetransparencyanddisclosurestotheirinvestors,regulatorsandotherrelevantstakeholders.

3 IOSCOResearchReportonFinancialTechnologies(Fintech),February2017,availableat:

/library/pubdocs/pdf/IOSCOPD554.pdf

4 TechnologicalChallengestoEffectiveMarketSurveillanceIssuesandRegulatoryTools,August2012,availableat:

/library/pubdocs/pdf/IOSCOPD389.pdf

ThisFinalReportconfirmstheguidanceproposedintheConsultationReport,amendingittotakeaccountofresponsesreceivedasappropriate.

DefiningthetermsAlandMLforthisreport

ArtificialIntelligence

ThetermArtificialIntelligence,firstcoinedbydatascientistJohnMcCarthy

5

in1956,isdefinedas“thescienceandengineeringofmakingintelligentmachines”,orsimply,thestudyofmethodsformakingcomputersmimichumandecisionstosolveproblems.AIincludestaskssuchaslearning,reasoning,planning,perception,languageunderstandingandrobotics.AIinthefinancialservicesindustryisstillinitsrelativeinfancyandispoisedtobecomemorecommon,andwiththatwillcomelegal,ethical,economicandregulatorychallenges.

MachineLearning

ThetermMachineLearningisasubsetandapplicationofAI,whichfocusesonthedevelopmentofcomputerprograms-designedtolearnfromexperiencewithoutbeingexplicitlyprogrammedtodoso.

TherearethreecategoriesofMLalgorithms–supervisedlearning,unsupervisedlearningandreinforcementlearning.Thesecategoriesareusedbasedonthetypeofdataavailableandthelevelofhumaninterventionrequiredinprovidingfeedback.DeepLearninginvolvestrainingneuralnetworks(computingsystems)withmanylayersofunits,inspiredbythestructureofthehumanbrainandcanalsoincludeanyofthesecategories:

6

Supervisedlearning:thealgorithmisfedaninitialsetofdatathathasbeenlabelled.Basedonthistrainingset,thealgorithmwilllearnclassificationrulesandpredictthelabelsfortheremainingobservationsinthedataset.

Reinforcementlearning:thealgorithmisfedaninitialsetofdatathathasnotbeenlabelledandisaskedtoidentifyclustersofobservationsunderpinnedbysimilarcharacteristics.Asitchoosesanactionforthedatapoints,itreceivesfeedbackthathelpsitlearn.

7

Unsupervisedlearning:thealgorithmdetectspatternsinthedatabyidentifyingclustersofobservationsunderpinnedbysimilarcharacteristics–ituncoversthestructureofthedataonitsown.

5 WhatisAI?availableat:

/artificial-intelligence/index.html

6Deeplearningisamethodthatanalysesdatainmultiplelayers,startingwithlearningaboutsimpleconceptsthenlearningmorecomplexconcepts.Deeplearningcanbeusedforallthreecategoriesofmachinelearningalgorithms.

Adaptedfrom‘WhatisMachineLearning?’at

/cloud/learn/machine-

learning.

Afamousexampleofthisisthe“move37”inthegameofGo:whenGoogle’sAlphaGo5algorithmwaspittedagainstprofessionalGoplayerLeeSedolinMarch2016,makingamoveonthe37thturnthatwaspreviouslyunimaginable.Thisalgorithmuseddeeplearning,aformofMLtechniquethatefficientlylearnsassociationsandstatisticalpatternsfromaverylargedataset.

7 RSutton,ABarto,ReinforcementLearning:anintroduction,MITPress,1998.

Chapter3–HowfirmsareusingAIandMLtechniques

AIandMLusebymarketintermediariesandassetmanagers

Marketintermediariesandassetmanagers’useofAIandMLisgrowing,astheirunderstandingofthetechnologyanditsutilityevolves.TheriseintheuseofelectronictradingplatformsandtheincreaseofavailabledatahaveledfirmstoconsidertheuseofAIandMLinseveralareas,includingfortheirtradingandadvisoryactivities,aswellasforriskmanagementandcompliance.

TheIOSCOfirmengagementrevealedthatwithinfinancialmarkets,AIandMLarebeingadoptedtoaugmentexistingprocessesandactivities,withaviewtoreducingcostandincreaseefficiency.AIandMLarefreeingupresourcestofocusonmorecognitiveaspects,suchasstrategy,portfolioselectionandgeneratinginvestmentideas.Marketintermediariesaredeployingthistechnologyin:

Advisoryandsupportservices;

Riskmanagement;

Clientidentificationandmonitoring;

Selectionoftradingalgorithm;and

Assetmanagement/Portfoliomanagement.

TheuseofAIandMLbyassetmanagersappearstobeinitsnascentstagesandismainlyusedtosupporthumandecision-making.AIandMLtechniquesarebeingusedto:

Optimiseportfoliomanagement;

Complementhumaninvestmentdecision-makingprocessesbysuggestinginvestmentrecommendations;and

Improveinternalresearchcapabilities,aswellasbackofficefunctions.

SomeassetmanagersarealsobeginningtouseAIandMLfororderexecution,brokerselectionandorderroutingpurposes(includingthroughmethodssuchasalgo-wheels).

8

Advisoryandsupportservices

AccordingtoIOSCO’sindustryengagement,mostrobo-advisorsorautomatedinvestmentadvisorsusesimple,rule-based(i.e.,deductive)algorithms,althoughsomearebeginningtoutilisepredictiveMLalgorithms.WhereMLisusedtoprovideadvisoryservices,mostfirmshavemanualinterventionprocesses.Theautomatedadvicesystemis,therefore,usuallylimitedtogeneratingpotentialadviceorassetallocationfortheinvestmentadvisertoreview.TheinvestmentadvisercanthenusethisAI-generatedadviceasappropriateand,wheresuitable,tomakearecommendationtotheclient.

8Algo-wheelsmayperformdifferentfunctionsindifferentpartsoftheworld.Inthiscontext,wedefinealgo-wheelstomeanasoftware/modelthataggregatesdatatoselectthestrategyandbrokerthroughwhichtorouteordersbeforegeneratingareportthatsetsoutthereasonbehindhowandwhereaparticulartradewasmade.

Riskmanagement

Riskmanagementinvolvesusingdatatopriceandmanageexposure,includingcredit,market,operationalandliquidityrisk.MarketintermediariesareharnessingML-basedriskmanagementsystemsforcreditriskmonitoring,whichcouldhelpprovideanearly-warningindicatorofpotentialcustomerdefaultsandcanhelpcreateadynamicmeasurementofacustomer’sriskprofiletobetterunderstand,forexample,whentowriteoffadebt.

MLisimprovingtheefficiencyofbackofficeprocessingandreportingfunctionswithinmarketintermediaries.Itisalsoincreasinglyusedtovisualisemarketriskbyanalysingvolatilitytrends,andtogaugeliquidityriskbyanalysingmulti-dimensionalriskandexposuredata.MLalgorithmsareincreasinglyusedtomonitorstaffe-mailsbyleveragingadvancedpatternrecognition.

Somemarketmakers,whoprovideliquiditytomarketparticipantsbyfacilitatingtransactionsareadoptingMLmodelsandreinforcementlearningtominimisetheirinventoryriskandmaximisetheutilityoftheirbalancesheet.

Similarly,someassetmanagersareseekingtoharnesstheadvantagesofthesetechniquesinriskmanagement.Somehedgefundsandassetmanagersareautomatingriskmanagementandcomplianceprocessesbytrackingthebehaviourofindividualportfoliomanagers,automatingexecutionqualityreportsandassessingmarketliquidityrisk.

Clientidentificationandmonitoring

MLhasallowedmarketintermediariestoautomatetheirclientonboarding,frauddetection,moneylaunderingandcyber-attackmonitoring.MarketintermediariesshouldgenerallyundertakeKnowYourCustomer(KYC)checksbeforeonboardingclientsandsellingthemproductsandservices.KYCentailscollectionandverificationofcomprehensivepersonalinformationfrompotentialclients,involvingprocessingunstructuredmetadata.

InductivereasoningalgorithmshelpaccuratelyidentifyfakephotoIDs,whilerecognisingdifferentphotosofthesameperson.MLcanalsobeusedforscreeningandmonitoringclientsandtransactionsagainstsanctionsorotherlists,todetectevidenceofpossiblemoneylaundering,terroristfinancingandotherfinancialcrimes.

Selectionoftradingalgorithms

Manymarketintermediariescurrentlyofferasoftwaresolutiontotheirclientsthatselectsanappropriatetradingstrategyand/orabrokerdependingonthemarketsituationandtradingobjectivesforbestexecutionpurposes,oftennamedanalgowheel.Algo-wheelsseektoclassifyhistoricaltradingandperformance,predicttheperformanceofstrategiesandbrokeralgorithms,andrecommendwhentouseaparticularalgorithm.Usingalgo-wheelstooptimallyexecutesimplerordersallowsthetradertofocusonmorecomplextradeflows.

Predictivedataanalyticsareenablingtheidentificationofpotentialmarketconditionsconducivetoaflashcrashtypeevent.

Assetmanagement/Portfoliomanagement

Supervisedlearning,whereafunctionisinferredfromlabelledtrainingdata,hasbeenusedforsmall-scalepatternrecognitionandsimplepredictionmodelstoaidtradingdecisionswithinassetmanagersandmarketintermediariesforseveralyears.

Marginpressureandcompetitionisdrivinginnovationamongstassetmanagers.Tocompete,someactivemanagersthattraditionallyemphasisedtheirfundamentalresearchcapabilitiesarebeginningtoexpandalreadyexistingquantitativeapproachesbyleveragingdiversifieddatasources–suchassocialmedia,geospatialdata,andothermetadatatoenhanceinternalresearch.

Incertaincases,thesemethodsarebeingusedforasset-allocationandpricingsuchasidentifyingrelationshipsinmetadatawhichcouldbeusedtogeneratetradeideasoralphasignalsandforecastassetpricesbasedonhistoricalpricesaswellascurrenttrends.Moreover,assetmanagersmayapplythesetechniquestopricenewinvestmentproductscompetitively.Theydosobyusingdataonexistinginvestmentproductswithsimilarstructuresand/orconstituentassets.Otherapplicationsincludeinvestmentcompliancechecks,transferagencyactivitiesandclientservicing.

Chapter4–IdentifiedPotentialRisksandHarmsPosedbytheUseofAIandML

IOSCO’sindustryengagementrevealedthattheevolutionandincreasingadoptionofAIandMLmayraise(eitherintentionallyorunintentionally)anumberofconductconcernsformarketintermediariesandassetmanagers,regarding:

Governanceandoversight;

Algorithmdevelopment,testingandongoingmonitoring;

Dataqualityandbias;

Transparencyandexplainability;

Outsourcing;and

Ethicalconcerns.

Governanceandoversight

FirmsimplementingAIandMLmostlyrelyonexistinggovernanceandoversightarrangementstosignoffandoverseethedevelopmentanduseofthetechnology.Inmostinstances,theexistingreviewandseniorleadership-levelapprovalprocesseswerefollowedtodeterminehowrisksweremanaged,andhowcompliancewithexistingregulatoryrequirementswasmet.AIandMLalgorithmsweregenerallynotregardedasfundamentallydifferentfrommoretraditionalalgorithmsandfewfirmsidentifiedaneedtointroducenewormodifyexistingproceduralcontrolstomanagespecificAIandMLrisks.

Somefirmsindicatedthatthedecisiontoinvolveseniorleadershipingovernanceandoversightremainsadepartmentalorbusinesslineconsideration,ofteninassociationwiththeriskandITordatasciencegroups.Therewerealsovaryingviewsonwhethertechnicalexpertiseisnecessaryfromseniormanagementparticipatinginandoverseeingcontrolfunctionssuchasriskmanagement.Despitethis,mostfirmsexpressedtheviewthattheultimateresponsibilityandaccountabilityfortheuseofAIandMLwouldliewiththeseniorleadershipofthefirm.

SomefirmsnotedthatthelevelofinvolvementofriskandcompliancetendstofocusprimarilyondevelopmentandtestingofAIandMLratherthanthroughthelifecycleofthemodel(i.e.,implementationandongoingmonitoring).Generally,onceimplemented,somefirmsrelyonthebusinesslinetoeffectivelyoverseeandmonitortheuseoftheAIandML.Respondentsalsonotedthatrisk,complianceandauditfunctionsshouldbeinvolvedthroughoutallstagesofthedevelopmentofAIandML.

ManyfirmsdidnotemployspecificcompliancepersonnelwiththeappropriateprogrammingbackgroundtoappropriatelychallengeandoverseethedevelopmentofMLalgorithms.Withmuchofthetechnologystillatanexperimentalstage,thetechniquesandtoolkitsatthedisposalofcomplianceandoversight(riskandinternalaudit)functionscurrentlyseemlimited.Insomecases,thisiscompoundedbypoorrecordkeeping,resultinginlimitedcompliancevisibilityastowhichspecificbusinessfunctionsarereliantonAIandMLatanygivenpointintime.

Algorithmdevelopment,testingandongoingmonitoring

Itisimportantthatfirmshaverobustandwellunderstooddevelopmentandtestingframeworksinplace,regardlessofwhethertheyareusingAIandMLortraditionalalgorithms.

Overall,IOSCO’sengagementshowedthatinmostcasesthereisnotanestablishedframeworkforspecificallydevelopingAIandML.Instead,manyfirmsusethesamedevelopmentandtestingframeworksthattheyusefortraditionalalgorithmsandstandardsystemdevelopmentmanagementprocesses.

Firmsthatusealgorithmsshouldconsidermaintaininganappropriatedevelopmentandtestingframework,whichisconsistentlyappliedacrossallrelevantaspectsofthebusiness.ThisisparticularlyimportantwherefirmsareusingAIandMLwithintheiralgorithmictradingstrategies.

Algorithmsrelyonqualitydataandmostfirmsrecognisetheneedforqualitydatainputs.Excessiveimmaterial,or“noisy”dataisunwanteddatathatdoesnotcontributetoarelationshipandmaycauseMLalgorithmstomissthesignalinthedataandbehaveunexpectedly.

Robustdevelopmentandtestingcontrolsarenecessarytodistinguishsignalsandstatisticallysignificantdatafromthenoise.Unliketraditionalalgorithms,asmoredataisprocessedbyMLalgorithms,theymaybehaveunpredictablyastheyareexposedtonewdatapatterns.MLalgorithmsshouldthereforealsobecontinuouslymonitoredthroughouttheirdeploymenttohelpensuretheydonotbehaveinexplicablyowingtoasubtleshiftintheoperatingconditionsorexcessivenoise.Whilesomefirmsnotedtheyreviewandreviseexistingmodelsasnecessary,somefirmsfocuslessonmanagingmodelsinthepost-productionphasesothattheyperformastheyshouldovertime.Unliketraditionalalgorithms,MLalgorithmscontinuallylearnanddevelopovertime.Itisimportantthattheyaremonitoredtoensurethattheycontinuetoperformasoriginallyintended.

Dataqualityandbias

TheperformanceofAIandMLisinherentlydependentonthequalityofthedatasetparticularlywhenbuildingthemodel.AccordingtoafewoftherespondentstotheConsultationReport,thequalityofthedatasetsusedinthelearningphasecanhaveamaterialimpactonthepotentialoutcomesandperformanceofAIandMLapplications.Intheviewoftherespondents,assuchitisakeyrisk.Accordingtosomerespondents,anotherriskrelatedtodatasetsisthesourcingofsufficientlylargedatasetstotraintheapplications,particularlywhenrelatedtoinvestmentdecisions.

Learnedbiasinthedatasetcanimpactthedecisionsmadebysuchalgorithmsandmayresultindiscriminativedecisionsandprovideundesirableoutcomestomarketparticipants.Forexample,askingquestionsphrasedinacertainwayorinacertainsequencemayleadtoaresponsethatintroducesimplicitorexplicitbiasfromtherespondents.Suchadataset,whereabiasmayhavebeenintroducedbyeitherthequestionerorbytherespondents,willinfluencetheconclusionsreachedbythealgorithm.Anyoutputbasedonsuchabiaswilllikelydegrade

theperformanceofthealgorithmmorequicklyovertimeandcouldresultinconsumerdetriment.

9

Biasmayalsobeinadvertentlyintroducedduringdatacleansing–apre-processingstepoftennecessarytoimprovedataquality.CleaningdatabeforeapplyingMLcanincreasethesignal-to-noiseratioallowingformoremeaningfulinterpretationstobederived.However,cleansingdatainvolvessubjectivedecisions,whichmayinadvertentlyintroduceotherbiases.

Transparencyandexplainability

TheeffectiveuseandadoptionofAIandMLrequirealgorithmsthatarenotonlyaccuratebutarealsounderstandablebyfirms(includingfrontline,complianceandriskpersonnel),marketcounterparties,clientsandregulators.Itispossiblethatrisksareintroducedifoutcomescannotbefullyexplainable.Whileincreasedtransparencyinfirms’useofAIandMLcouldimprovepublicunderstandingandconfidenceintheuseofthetechnology,excessivetransparencycouldcreateconfusionoropportunitiesforindividualstoexploitormanipulatethemodels.Theleveloftransparencywillalsodifferdependingontheaudience;forexample,aregulatormayrequiremoredetailedinformationthanaclient.TheseconsiderationsneedtobebalancedindeterminingtheappropriateleveloftransparencyintheuseofAIandML.

Itisimportantthatfirmsappropriatelydiscloseinformationabouttheirserviceofferings,tohelpclientsunderstandthenatureandrisksofproductsandserviceofferings,sothattheycanmakeinformeddecisions.ApplyingunexplainableMLalgorithmstorefineatradingstrategycouldexposethefirmtounacceptablelevelsoflegalandregulatoryrisk.Firmsofferingautomatedinvestmentservices,includingroboadviceandautomatedportfolioconstructionshouldappropriatelydisclosethenatureoftheuseofMLandtheautomationintheirserviceoffering.

SomeMLmodelsoperateasa“blackbox”withlimitedclarityonthereasoningbehindt