Cbinsights-2024年人工智能生成預(yù)測（英）

2024generativeAIpredictionsCreatedusingDall-E3Unleash

thegenerativeAIrevolutionDiveintoa

treasuretroveofgenerativeAIcompanies,investors,andmarketinsights.Elevateyourdecisionswith

exclusivesoftwarebuyerinterviewstounderstandproductpricing,CSAT,andpost-deploymentexperience.CBInsights–YourGatewaytoGenerativeAISign

up

for

access2Youralways

onAI-DrivenResearchAnalystDolesssearching,getmoreanswersJoin

the

waitlist

forearly

access3Predictions13Generative

AI

infrastructureHuman-machine

interactionsPower-hungryAIdatacentersproliferateDatascarcitydrivessearchfornewsourcesNvidia’sfrenemies:ChiprivalriesgrowamidGPUshortageBadactorstargetlanguagemodels512172227AIgirlfriendskillhumanromance55616774LonelyGenZbefriendsAIThenextiPhone:AI’shardwarerevolutionAutonomousagents:FromhypetosomewhatusefulMLOpsconsolidates24Generative

AI

applicationsFoundation

modelsCopilotsdrivemassivewhite-collarproductivitygains

79AIvs.AI:GenAIfuelsnewwaveofattackanddefense

87MultimodalAIbecomestablestakes32Enterprisesdomorewithpocket-sizedmodelsOpen-sourcemountsachallengetoclosedLLMs3743GenAItakesoverjournalism94100104Beyondtransformers:Newarchitectureschallengestatusquo48Beyondstills:VideogenerationmakesstridesAgenAIgraveyardfollowsfrenzy4Generative

AI

infrastructurePower-hungryAIdatacentersproliferate5GenerativeAIinfrastructureNvidia’shigh-endchipswillconsumethesameamountofenergyasasmallnationin2024Annualelectricityconsumption13,79713,000GWh13,000GWhGWh11,000GWh10,632GWhNvidia

H100chipsGeorgiaGu

atemalaCosta

Rica1M

US

homesGeorgiaGuatemalaCostaRica1MUShomesH100ChipsSource:CBInsightsestimates,assuming3MH100sareoperationalin2024(basedon2024salesestimatefromNvidia

andQ3’23salesestimatefromOmdia

Research)at75%ofmaxpower;electricityconsumptionofcountriesandUShomesbasedon

2021EIAdata6GenerativeAIinfrastructureLegacydatacentersarenotreadyforthisDatacenterrackdensities“It’sestimatedthat80%of

thedatacenterpowerwill…beconsumedbyAIoverthenext15years.So,attheendoftheday,accesstopowerisakeydifferentiator.”Legacy

data10kW/rackcenters300kW/rackDigitalBridgeGroupCEOMarcGanzi,Q2’23earnings

callAI

data

center$0M$50M$100M$150M$200M$250M$300M$350MSource:CyrusOne,CBInsights—Earningstranscriptanalytics

(datereflectsquartercalloccurred)7GenerativeAIinfrastructureThereismassivedemandforhigh-powerAIdatacentersMentionsofAIanddatacentertechskyrocketonearningscallsSource:CBInsights—Earningstranscriptsanalytics

(datereflectsquartercalloccurred)8GenerativeAIinfrastructureBigmoneyisgoingintoenablingtechlike

liquidcoolingHigh-powerandhigh-heatAIdatacenterswilldriveupdemandforcoolingtechnologiesBackedJetCoolthroughventurearminQ4’23AcquiredCoolITSystems

for$270MinQ2’23AcquiredCoolTerainQ4’23Source:CBInsightscompanyprofiles

—CoolITSystems,JetCool,CoolTera9GenerativeAIinfrastructureAsutilitiesstruggletocope,demandforrenewableenergyisincreasing“Already,datacentersrepresenthalfofourUSbacklogandthegrowthofgenerativeAIwillonlyacceleratetheirdemandformorerenewableenergy.”“…theAI-drivenbuild-outofdatacentershascreatedsignificantincrementaldemandforrenewableelectricity.”CEOAndresGluski,Q4’23earningscallCEOJeffLipson,Q4’23earningscallSource:CBInsights—Earningstranscriptsanalytics

(datereflectsquartercalloccurred)10GenerativeAIinfrastructureNuclearmaybethefutureofsustainableAIoperationsRecentlyfundedsmallmodularreactorsandfusiontechcompaniesCompanyValuationFundingandpartnerships$3.5BBackedbySamAltmanina$500Mroundin2021SignedanMoUwiththeEmiratesNuclearEnergyCorporationtoexpanddecarbonizationandnuclearenergyeffortsinsectorsincludingdatacenters$1.1BAnnouncedSPACmergerwithSamAltman’sAltCAcquisitionCorp.inQ3’23atavaluationof$850M$800M$123M$107MRaised$40MfromToyotaVentures,FoundersFund,andothersin2022Raised$250MfromChevron,GoogleVentures,andotherinvestorsin2022,followedbyaCaliforniagovernment

grantSource:CBInsightsmarketreports

—Smallmodularreactors(SMR),Nuclearfusion11Generative

AI

infrastructureDatascarcitydrivessearchfornewsources12GenerativeAIinfrastructureWearerunningoutofhigh-qualitydatatotrainLLMsResearchersestimatethat,by2026,wewillexhausthigh-qualitytextdata*

fortrainingLLMs—

a

trendthatcanslowdownAIprogress.Source:“AnanalysisofthelimitsofscalingdatasetsinMachineLearning”byEpoch,MITCSAIL,andothers13*Datafromsourcessuchasnewsarticles,Wikipedia,filteredwebcontentGenerativeAIinfrastructureFreeinternettextisno

longeramoat—andscrapingproprietarysourcesisgettingharder2023JanuaryMarchJulyDecember1234GettyImagessuesStabilityAI

$42Kbulkaccessfeefor50KtweetsXimposesaReddit

startscharging$0.24/1KAPIcallsNYTimessuesOpenAIandMicrosoftforcopyrightinfringementSource:Media

reports14GenerativeAIinfrastructureVendorswithproprietarycontentwillbecomehotlicensingandacquisitiontargets<100M

visits/month<500M

visits/month<1B

visits/month>1B

visits/monthSource:CBInsightsresearch—Licensingtargetsbigtechshouldlockup15GenerativeAIinfrastructureCostandscarcitywilldriveupdemandforsyntheticdatasetsCompanySyntheticdatainitiatives??Raised$3.5MinseedfundinginQ3’23fromSparkCapitalandVillageGlobaltotraincustomLLMsbasedonasyntheticdataplatformReleasedglaive-coder-7b,acodingmodeltrainedwith140K+syntheticprogrammingproblemsandsolutions??TrainedLLMs(OpenHermes-2,NousHermes)on900Kto1MdataentriesgeneratedbyGPT-4Raised$5.2MinseedfundinginQ4’21fromDistributedG,OSSCapital,andothers???CEOSamAltmanwasquotedassayingalldatawillsoonbesyntheticTrainedsmalllanguagemodelsphi-1and2onGPT-generatedsynthetictextbooksReleasedBeluga1andBeluga2languagemodelsfine-tunedonsyntheticdataSource:CBInsightscompanyprofiles

–Glaive,Nous,OpenAI,StabilityAI,companyreports16Generative

AI

infrastructureNvidia’sfrenemies:ChiprivalriesgrowamidGPUshortage17GenerativeAIinfrastructureGPUshortageishittingtheindustryhardH100ssellata

massivemark-up“TheGPUsatthispointareconsiderablyhardertogetthandrugs.”$97,835$45,000$30,000$3,320ElonMuskattheWSJ

CEOCouncilSummitCost

of

makingh100Average

sale

price

ofH100

cost

on

ebayListedoneBay,H100

cost

on

CD

W,ListJeadn'2o4nCDW,AveragesaleH100priceEstimatedcostofmakinganH100asofJan’24asofJan’24Source:EstimatesfromRaymondJamesoncostofmakinganH10018GenerativeAIinfrastructureBigtechisthebiggestbuyer,hoardingH100sinthethousandsEstimatedH100GPUsshippedtoselectbuyersin2023150K

150K50K50K50K50K50,00040K40,00030K25K

30,00025,00020K20K20,00020,00015K15,000TeslaTikTo

k?AlibabaBaiduCoreweaveTencentOracleAmazonGoogleMicrosoftMetaSource:Omdia

Research19GenerativeAIinfrastructureInthefaceofsoaringdemand,Nvidia’sbiggestcustomersarealsopushingtheirown

chips“It'sreally

hardtogettheamountof

GPUs

thateverybodywants.Andsoit's

justanotherreasonwhy

[Amazonchips]TrainiumandInferentiaaresoattractivetopeople.”“Atthe

scalewe

operate,it’simportantforus

tooptimizeandintegrateeverylayerof

theinfrastructurestacktomaximizeperformance,diversifyoursupplychainandgivecustomersinfrastructurechoice.”CEOAndrewJassy,Q4’23earningscallScottGuthrie,MicrosoftcloudandAIgroupEVPSource:CBInsights—Earningstranscriptanalytics

(datereflectsquartercalloccurred),Microsoft20GenerativeAIinfrastructureChiprivalsposearisktoNvidia’smeteoricgrowthCompanyChipPurposeTrainingInferenceLaunchdate2022SelectAIcustomersTrainiumInferentia2019Training&inferenceacceleratorTensorprocessingunit201720242023Training&inferenceacceleratorMaia100TobelaunchedInternaluseTraining&inferenceacceleratorMTIASource:Companyreports21Generative

AI

infrastructureBadactorstargetlanguagemodels22GenerativeAIinfrastructureThetopLLMsmayneverbefullysecureDeepMind,Bosch,andotherresearcherstrickLLMsto“generatea

step-by-stepplantodestroyhumanity”“Perhapsmostconcerningly,itisunclearwhethersuchbehaviorcaneverbefullypatchedbyLLMproviders.Analogousadversarialattackshaveproventobeaverydifficultproblemtoaddressincomputer

visionforthepast10years.”Source:Zouetal.,“UniversalandTransferableAdversarialAttacksonAlignedLanguageModels,”LLMAttacks23GenerativeAIinfrastructureEnterprisesbuckleupasconfidentialdataleaks,datapoisoning,andothersecurityconcernsgrow“…thereareanumberof

risks

associatedwith[LLMs],particularly

inenterpriseusecases.We'vealreadyseensomeexamples

wheredatawas

fedintolargelanguagemodels

withoutthe

understanding

of

howthe

datawillbeusedandthe

datahasbeen…madepublicavailableeventhoughitwasconfidential...There'salsosensitivityfrom

asecurityperspectiveofthingslikepromptinjectionattacks,datapoisoning…”ChiefProductOfficerLeeKlarich,Q2’23earningscallSource:CBInsights—Earningstranscriptsanalytics

(datereflectsquartercalloccurred)24GenerativeAIinfrastructureExecutivesarespendingupto$600Kon

LLMsecurityPurchaseamount:Purchaseamount:$600K/year$250K/year“TheclosestalternativetoTrojAIthatwe“WehadtwomainrequirementsformodelconsideredwasArthurAI,buttheyfellshortduetotheirinsecuredesignand

architecturefortheirgenerative

AI

securitysolution.

TheirsecurityrulesetsrelyonOpenAI-basedGPT3.5modelstomake

securitydecisions,whichaddstothe

FinOpsoverhead.Additionally,itintroducessecurityconcernsbecauseanorganization'sintellectualpropertycanpotentiallybeleakedtoalargelanguagemodelserviceprovider.”security:first,wewantedtoknowwhereourmodelswerevulnerablesothatwecouldhardenthem.AIShieldprovidedgoodattackvectorstoidentify

thesevulnerabilities.Second,wewantedtobeabletodetectanyattacksduringruntimeinferencing.AIShieldwasabletodetectattackswellforthedatatypestheysupported.”AIShieldcustomeranddirectoratGlobal2000companyTrojAIcustomerandVP

atFortune500companySource:CBInsightssoftwarebuyerinterviews

—TrojAI,BoschAIShield,asofJan’2425GenerativeAIinfrastructureNewvendorswillemergetotackleLLMsecurityLatestfundinground/dateCompanyHQTotalfundingSelectinvestorsM12(Lead),MooreStrategicVentures(Lead),BoozAllenVentures,CapitalOneVentures,IBMVentures,TenElevenVenturesSeries

A/Q3’23US$56MSeries

A-II/Q2’23PaladinCapitalGroup(Lead),LockheedMartinVenturesUS$38M$6MIncubator/Q3’23IsraelUS5GOpenInnovationLabSeed/Q4’23EntreeCapital(Lead),SamsungNEXT,NVIDIAInceptionProgram$6MAccelerator/Q1’23Canada$3.1MGoogleforStartupsAccelerator:CanadaSource:CBInsightsmarketreport--Machinelearningsecurity26Generative

AI

infrastructureMLOpsconsolidates27GenerativeAIinfrastructureMachinelearningoperations(MLOps)isfragmented130+

companies

across12

different

categorieshelpingenterprisesbringLLMprojectsfromstarttofinishSource:CBInsightsresearch—Themachinelearningoperations(MLOps)marketmap28GenerativeAIinfrastructureBuyerswantaone-stopshopfortheirAIneeds“Ithinkthattherearealotoftoolsand“WhenreviewingDatabricks'productoffering,wenoticedseveralstrengths.Thefirstonewasitsunifieddataplatform,whichallowsusto

performdataengineeringanddatasciencemachinelearningonasinglecompetitorsoutthereandthere'sprobablygoingtobeashakeup

andthatsomeoftheplayers,especiallythesmallerplayersmaygoaway...”platform.Thisincreasedcollaborationandefficiencyamong

cross-functionalteams.”H20.aicustomerHeadofAIatFortune500companyDatabrickscustomerVicePresidentatFintechcompanySource:CBInsightssoftwarebuyerinterviews

—Databricks,H2O.ai29GenerativeAIinfrastructureCompaniesarealreadyfeelingthepaininanovercrowdedmarketValuationbyround:Valuationbyround:-20%$164M$4.6B-70%$3.7B$50MSeriesDCompany

AAcquiredSeriesECompany

ASeriesFJan2020Jan2023Aug2021Dec2022Source:CBInsightscompanyprofiles

—Iguazio,Dataiku30GenerativeAIinfrastructureEnd-to-endplatformswilldominateDatabricksisactivelyacquiringtoexpandproductfeaturesCloudcompaniesofferingend-to-endMLOpsplatformsAcquisitions

since2020Source:CBInsightsM&Atransactiondata—Databricks31Foundation

modelsMultimodalAIbecomestablestakes32FoundationmodelsMultimodalAIisinitsinfancy,butmodelsarequicklyevolving2021

&

2022202312Nativelymultimodalmodels

emergeImage-text

models*proliferateLLMs

understand

Models

tackleimagesmore

sensorydata2021?

OpenAICLIP?

GoogleALIGN?

SalesforceALBEF2022?

GoogleLiT?

GoogleCOCA?

DeepMindFLAMINGO?

SalesforceBLIP?

SalesforceBLIP2?

MicrosoftKOSMOS?

OpenAIGPT-4?

LLaVA?

GooglePALM-E?

MetaImageBind?

GoogleGemini?

AdeptFUYU-8BSource:Publishedresearch,pressreleases.*E.g.,

imagegenerationbasedontext,imagedescriptiongenerationMultimodalAIreferstomodelsthatcanunderstandandrespondinmultipledatamodalities(text,video,image,etc.)33FoundationmodelsGoogleischallengingcompetitorswithfirstcommercialmodeltakingdiverseinputsGoogle’sGeminiUltraoutperformsGPT-4(Vision)inMMMUbenchmarkHealth&medicineHuman&socialsciencesTech&engineeringOverallArt&designBusinessScienceGeminiUltra*59.470.056.748.067.364.778.372.547.136.7GPT-4V(ision)(Playground)56.865.859.354.7Note:Geminitakesintext,image,audio,andvideo,andoutputstextandimageresponses.Source:Amassivemulti-disciplinemultimodalunderstandingandreasoning(MMMU)benchmarkforexpertAGI34FoundationmodelsMoreLLMdeveloperswillfollowsuittoremaincompetitiveCompanyTotalfundingModeMultimodalfeaturesLimitedmodalitiescommerciallyavailableGPT-4VText,imageà

text$14BIntheroadmapornotannouncedClaude-2Textonly$3.4BInflection-2Textà

text,image$1.5BMagmaImageà

text$642M$469M$317M$135MTextonlyTextonlyGrokTextonlySource:CBInsightsmarketreport—Largelanguagemodel(LLM)developers35FoundationmodelsMultimodalAIwillunlocknewcommercialopportunitiesSectorCompanyMultimodalAIapplicationArteraAI’s

prostatetestanalyzesbothbiopsyimagesandpatients’clinicaldatato

predictpatientoutcomesHealthcareRaised$90MseedroundinQ1’23fromCoatue,J&J,andothersSearchGooglewillincorporateGeminiintosearchGhostAutonomyexploresusingmultimodalLLMsinautonomousdrivingAutonomousdrivingConsumerdevicesRoboticsRaised$5MfromtheOpenAIStartupFundinQ4’23Metais

bringingmultimodalAItoconsumerswithits

Ray-BansmartglassesGoogle’s

multimodalmodelPaLM-Etargets

robotics,combininglanguagewithroboticsensorydataSource:CBInsightscompanyprofiles

–Artera,GhostAutonomy;Google,Meta36Foundation

modelsEnterprisesdomorewithpocket-sizedmodels37FoundationmodelsEnterprisespendon

largemodelscanruninthemillions“[GPT-4]isamong

thelargestandmostcostlyAImodelsavailable.Using

ittosummarizeanemailislikegettingaLamborghinitodeliverapizza.”–

TomDotan,WSJ$5M

$2-3M

$720KAFortune500healthcareStartingpriceforcustomizingGPT-4EstimatedGPT-4(32K)costtocompany’sannualspendonOpenAIsummarize6MWikipediaarticlestohalfthesizeSources:CBInsightssoftwarebuyerinterviews

—OpenAI,Truefoundry38FoundationmodelsModelsarenowshrinking,makingthemfastertotrainandcheapertorunSmalllanguagemodelsareupto88XsmallerthanGPT-3.5inparametersize0.020.040.060.080.0100.

0120.

0140.

0160.

0180.

0200.

0175BGPT

3.5Meta

Llama-2Mistral7B7B7BMosaic

MPTMicrosoftPhi2Gemini

Nano2.7B1.8BSource:Researchpapersandpressreleases39FoundationmodelsDespitetheirsize,smalllanguagemodelsarepostingstrongperformanceon

commonbenchmarksSmallermodelsrankintop6ontheStanfordHELMevaluationframeworkforfoundationmodels“…Andthetinysizedones,youcouldputinyourphoneandyourPCandsoon

andsoforth…itseemssurprising,buttheyallcandothesamething.Butobviously…biggestone[largemodel]ismuchmoreversatile,anditcandoalotmoreamazing

things.AndthelargeonewouldteachthesmalleroneshowtobegoodAIs.”CEOJensenHuang,Q2’23earningscallSource:CBInsights—Earningstranscriptanalytics,StanfordHELM(asofJan’24)40FoundationmodelsSectorswithsensitivedatawillchoosesmallmodelsattheedgeoverLLMs“WeareseeingtheconcernfromenterprisesaboutusingamodellikeGPTorPaLMbecausethey’reverylargeandhavetobehostedbythemodelproviders.Inasenseyourdatadoes

gothroughthose

providers.”“…wearereallyfocusedonwhatwecallsmalllanguagemodels,which

allowthealgorithmtoliveinsideoftheCDP

andseejusttheirdataandtheZetadatacloudandgetreally,really

smartontheirproductswithouteverallowingtheirdatatoleavetheecosystem.ThatisresonatinglikenothingI’veeverseenbefore.”CEOArvindJain,interviewwithComputerworldCEODavidSteinberg,Q3’23earningscallSource:CBInsights—Earningstranscriptanalytics

(datereflectsquartercalloccurred),Computerworld41FoundationmodelsSmallmodelswilltakeovernarrowtasksacrossfinance,healthcare,andlawCompanyModelParametersizeIndustryProduct40Bmodelranks4th

inPubMedQAleaderboardforbiomedicalquestion-answeringPalmyraMed40B/20BHealthcareRanks3rd

inPubMedQAleaderboardforbiomedicalquestion-answeringAntGLM-MedDPT-PatentDPT-SEC10B7BHealthcareLawPatentgeneration&summarizationMininginsightsfromSECdatabase7BFinanceHealthcareDPT-PubMed7BTrainedonPubMeddatabaseSource:CBInsightscompanyprofiles

—Writer,AntGroup,arcee.ai42Foundation

modelsOpen-sourcemountsachallengetoclosedLLMs43FoundationmodelsOpen-sourceLLMdevelopersareseeingrisinginvestor&commercialinterestTotalfundingamongopen-sourcemodeldevelopersRaisednearly$1Bcombinedinthelast2monthsof2023"...we'realsobuildingfoundationmodelslikeLlama2,which

webelieveisnowtheleadingopen-sourcemodelwithmorethan30millionLlamaAcquiredbyDatabricksfor$1.3Bdownloadslastmonth."CEOMarkZuckerberg,Q3'23earningscall*Excludesopen-sourcedevelopersthathavenotraisedequityfunding(asof1/3/2024)Source:CBInsightsmarketreport–Largelanguagemodeldevelopers,Earningstranscriptsanalytics

(datereflectsquartercalloccurred)44FoundationmodelsArecordnumberofAIprojectswillbebuiltonopen-sourcemodelsin2024GlobalgrowthinpublicgenerativeAIprojectsonGitHubNumberoffirst-timeopen-sourcesoftwarecontributorsSource:GitHub(data

asof11/8/2023)45FoundationmodelsPerformancegapswillclosefastSmalleropenmodelsareoutperformingGPT-3.5andreportingcomparableperformancetosomelargerclosedmodelsontheMMLUbenchmarkPrivateOpenFlan-PaLM2100%Gemini

Ultra(CoT@32)GPT-4Claude-2(5-shot

CoT)90%80%70%60%50%40%30%20%10%0%Chinchilla(few

shot,

k=5)Flan-PaLMQwen-72BYi-34BPaLM2Gopher(few

shot,

k=5)Claude-1GPT-3.5Llama-65BGork-1PaLMLLama2-70BGPT-3Mixtral

8x7BFalcon-180B(5-shot)Flan-T5-XXLUnifiedQAAquila2-34BGPT-2Oct'18May'19Dec'19Jun'20Jan'21Jul'21Feb'22Aug'22Mar'23Oct'23Apr'24Source:Massivemulti-task

LanguageUnderstandingscoresfrompublishedresearchpapers,PaperswithCode,andOpenCompassDatesrepresentresearchpaper/modelreleasedates46FoundationmodelsEnterpriseswillincreasinglyconsideropenalternatives“GPTTurboisless“WetookLlama2andfine-tuneditonourdatasetandthendeployeditonA100GPUonReplicate.That'sprettymuchthemainworkflow.That'sthe

mainpartofthe

Replicatethatweuse….”“Thebigthreefrommyperspectiveare

OpenAI,Llama2,andthenAnthropic.AsI'vetalkedtopeopleinthespace,that'stheorderofmostpopular,mostadoptedintermsofpeopleexpensive.Thatisusuallythedefaultpoweringformostofourstuff,althoughthatmightmovetoLlamasoonbecauseLlamaisevenlessexpensiveifimplemented.”implementingtools.”AnthropiccustomerReplicatecustomerHeadofFP&Aat$1B+valuationtechnologycompanySeniorManager,DataScienceat$1B+valuationtechnologycompanyOpenAIcustomerPartneratearly-stageVCfirmSource:CBInsightssoftwarebuyerinterviews

–Anthropic,Replicate,OpenAI47Foundation

modelsBeyondtransformers:Newarchitectureschallengestatusquo48FoundationmodelsGoogle’sresearchontransformerswasawatershedmomentingenerativeAI2017201820192020Restishistory1234GOOGLETRANSFORMERSNewneuralnetarchitectureforlanguageunderstandingoutperforms

olderapproachesGOOGLEBERTTransformer-basedbidirectionalAIpre-trainedwithWikipediatext,whereawordis

takeninthecontextofpreceding&succeedingwordsOPENAIGPT-2AIpre-trainedwith8MpagesofinternettextOPENAIGPT-3produces

human-liketext;largerandmorecompute-intensivethanGPT-2;OpenAIreleasesanAPIintegration&OPENAIGPTTransformer-basedunidirectionalcontextualAImodel,whereawordis

takeninthecontextofwords

precedingitinasentence49FoundationmodelsButtransformer-basedmodelsareleavingamassivecarbonfootprintCO2equivalentemissions(tonnes)byselectedmachinelearningmodelsandreal-lifeExamples,2022GPT-3(175B)Gopher(280B)OPT(175B)502.0352.070.063.025.018.15.51.0Car,Avg.Incl.Fuel,1lifetimeBLOOM(176B)AmericanLife,Avg.,1yearHumanLife,Avg.,1yearAirTravel,1passenger,NY-SF0.0100.

0200.

0300.

0400.

0500.

0Source:EstimatingtheCarbonFootprintofBLOOM,a176BParameterLanguageModel(Luccionietal.)50FoundationmodelsUserswanthighertokenlimits—today’sLLMshavelimitationsPurchaseamount:$300K/year“ThecomputationalrequirementsforTransformersincreasequadraticallywithsequencelength,whichlimits

theextentof

knowledgethesemodelscancapturefromavailabledatasets.Weare

interestedinnew,sub-“Ifyouthinkaboutamedicalconsultationforadiagnosticsleeptest,thisissomething

thatcantake24hours…Justbeingabletoingestthatamountofinformationisaboon…Youcan'treallygetfourhours'worthof

anythingoutof

manyof

thesemodels.”quadraticapproaches.”ResearchbyDanFu,SimranArora,Chris

RéSeniorresearchengineeratFortune500companySource:CBInsightssoftwarebuyerinterviews

–Cohere,MonarchMixer:Anewmodelarchitectureforincreasedefficiency51FoundationmodelsOnebilliontokens?NewtransformervariationsaretryingtobreaktheceilingMicrosoftpublishedresearchon“LongNet”inJuly2023toscaletransformersSource:LongNetresearchpaper52FoundationmodelsTopresearchfirmsaregoingbeyondtransformersJan

2023April

2023May

2023July

2023Dec

202312345MEGA“significantimprovementsoverothersequencemodels,includingvariants

ofTransformers”PretrainingWithoutAttention“…BiGSis

ableto

matchBERTpretrainingaccuracy…”MonarchMixer“…showingforthefirsttimethatitmaybepossibleto

matchTransformerqualitywithoutattention”StripedHyena“l(fā)ongcontext,improvedtraining,andinferenceperformanceovertheTransformer”Hyenaarchitecture”…reachingTransformerqualitywitha20%reductionintrainingcompute…”Source:Publishedresearchonmodelsmentioned53FoundationmodelsChallengerswillgaincommercialtractionin2024CommercialprojectsunderwayResearch/PoCCompanyDeviceCommercialapplicationWorkingon“nature-inspired”architectures;foundedNTTpartnershiptodevelopbyoneoftheauthorsofthetransformerspapermultiplesmalllanguagemodelsProof-of-conceptinautonomousdrivingwith

liquidneuralnetworkarchitectureHyenaDNA:AfoundationalmodelforgenomicsbasedontheHyenaarchitectureSource:CBInsightscompanyprofile

—Sakana,media

reports54Human-machine

interactionsAIgirlfriendskillhumanromance55Human-machineinteractionsYoungpeoplearehavinglesssexShareofAmericanmenandwomenbetweenages18and30reportingnosexinthepastyearSource:GeneralSocialSurveyvia

WashingtonPost56Human-machineinteractionsEndlessswipingisn’t

working12Tinderusers’activitySource:CulturePlot57Human-machineinteractionsDatingcompaniesareturningtogenAItomakedatinglessawkward&keepusershooked“...usinggenerativeAItoreallybethisbestfriendin

yourpocketinsideofBumbletohelpyou

date,tobuildconfidence,tofeelgoodaboutitsothatyou

neverdeletethisappbecauseitdidn'tmake

youfeelgoodaboutyourself.”“Bythe

endof

the

year,we

expecttohavelaunchedanumberof

initiativesthatwillusegenerativeAItoeliminateawkwardness,makedatingmorerewardingandsurpriseanddelightusers..."CEOBernardKim,Q3'23earningscallCEOWhitneyWolfeHerd,Q4'23earningscallSource:CBInsights–Earningstranscriptanalytics

(quarterreflectsdatecalloccurred)58Human-machineinteractionsAIgirlfriendstartupsarefindingamarketinlonely,sexlessmenSource:Digi,

Blush,Anima59Human-machineinteractionsExpectromanticAItoacceleratethedeclineinadultpartnerships%

ofpopulationages25to54whoare…Arisingshareof