基于ArcGIS的水利大數(shù)據(jù)及應(yīng)用課件

1、基于ArcGIS的水利大數(shù)據(jù)及應(yīng)用團(tuán)隊(duì)簡(jiǎn)介水利大數(shù)據(jù)及其面臨的挑戰(zhàn)基于水利大數(shù)據(jù)的多災(zāi)害信息集成與風(fēng)險(xiǎn)預(yù)警案例主要內(nèi)容123二、水利大數(shù)據(jù)及其面臨的挑戰(zhàn)水利工作關(guān)系到國(guó)計(jì)民生，尤其是我國(guó)水資源 分布存在嚴(yán)重的時(shí)空分布不均特性，旱災(zāi)洪澇 易發(fā)多發(fā)。水利行業(yè)在經(jīng)濟(jì)、生態(tài)、社會(huì)等方 面都扮演著重要角色，對(duì)水利大數(shù)據(jù)的研究具 有重要的現(xiàn)實(shí)意義和應(yīng)用價(jià)值。水利大數(shù)據(jù)是在大數(shù)據(jù)的理論指導(dǎo)及技術(shù)支 撐下的水利科學(xué)和工程的重要實(shí)踐。水利工作及水利大數(shù)據(jù)的重要性水利大數(shù)據(jù)水利大數(shù)據(jù)是指產(chǎn)生于各種水文監(jiān)測(cè)網(wǎng)絡(luò)、水利設(shè)施、用 水單位和水利相關(guān)經(jīng)濟(jì)活動(dòng)，并通過(guò)現(xiàn)代化信 息技術(shù)高效傳輸、分布存儲(chǔ)于各地存儲(chǔ)系統(tǒng)、 但又可

2、以快速讀取集中于云端、實(shí)現(xiàn)深度數(shù)據(jù) 挖掘并可視化的海量多源數(shù)據(jù)總和。ValueVelocityVolume海量快速價(jià)值Variety多樣Veracity真實(shí)交叉性，由于水利和其它領(lǐng)域具有交叉性，因此水利大數(shù) 據(jù)和遙感大數(shù)據(jù)、氣象大數(shù)據(jù)、海洋大數(shù)據(jù)等交叉；時(shí)空分布性，需要依賴先進(jìn)大數(shù)據(jù)技術(shù)進(jìn)行處理分析，包 括分布式大數(shù)據(jù)存儲(chǔ)框架、機(jī)器學(xué)習(xí)等數(shù)據(jù)挖掘方法；多元循環(huán)性，由水的多元循環(huán)決定的水利大數(shù)據(jù)在經(jīng)濟(jì)、 社會(huì)、生態(tài)等領(lǐng)域的價(jià)值循環(huán)。水利大數(shù)據(jù)的外延挑戰(zhàn)一：水利大數(shù)據(jù)的收集與集成水利大數(shù)據(jù)來(lái)源廣泛，不同的監(jiān)測(cè)平臺(tái)得到的 數(shù)據(jù)具有不同的數(shù)據(jù)結(jié)構(gòu)、存儲(chǔ)系統(tǒng)，非結(jié)構(gòu) 化數(shù)據(jù)、半結(jié)構(gòu)化數(shù)據(jù)、結(jié)構(gòu)化數(shù)據(jù)并存

3、；由于觀測(cè)條件的差異，數(shù)據(jù)可信度層次不齊，對(duì)數(shù)據(jù)清洗和質(zhì)量的確保提出了很高的要求；大數(shù)據(jù)的存儲(chǔ)與管理需要新型數(shù)據(jù)庫(kù)的支持， 水利大數(shù)據(jù)的信息化還未與新型數(shù)據(jù)庫(kù)接軌。水利大數(shù)據(jù)面臨的挑戰(zhàn)挑戰(zhàn)二：水利大數(shù)據(jù)的時(shí)空多維度分析水利大數(shù)據(jù)具有明顯的時(shí)空分布特性，時(shí)間、 空間雙維度下的數(shù)據(jù)分析具有難度；水利大數(shù)據(jù)在其應(yīng)用領(lǐng)域講究實(shí)時(shí)性，比如洪 水預(yù)報(bào)等，這對(duì)大數(shù)據(jù)的處理分析速度提出了 高要求；水利大數(shù)據(jù)的深度挖掘有賴于引入先進(jìn)的人工 智能算法，兩者的有效結(jié)合至關(guān)重要。水利大數(shù)據(jù)面臨的挑戰(zhàn)挑戰(zhàn)三：水利大數(shù)據(jù)的共享與安全眾多水利數(shù)據(jù)掌握在政府機(jī)關(guān)部門，為非公 開數(shù)據(jù)，形成數(shù)據(jù)孤島現(xiàn)象；水利數(shù)據(jù)是國(guó)家安全的重要

4、組成部分，水利 數(shù)據(jù)的共享與安全是一個(gè)值得探討的問(wèn)題。水利大數(shù)據(jù)面臨的挑戰(zhàn)三、基于水利大數(shù)據(jù)的多災(zāi)害信息集 成與風(fēng)險(xiǎn)預(yù)警案例介紹基于水利大數(shù) 據(jù)的多災(zāi)害信 息集成與風(fēng)險(xiǎn) 預(yù)警案例介紹1、天、地、空、海，多基多源降水?dāng)?shù)據(jù)采集2、移動(dòng)眾包信息收集可視化云平臺(tái)mPing3、基于水利大數(shù)據(jù)的全球洪水泥石流災(zāi)害預(yù)測(cè)預(yù)報(bào)4、基于概率洪水風(fēng)險(xiǎn)預(yù)報(bào)EF55、城市洪水模型Urban CREST介紹6、全球風(fēng)暴數(shù)據(jù)庫(kù)及CI-FLOW7、中國(guó)區(qū)域多尺度洪水模擬及預(yù)警系統(tǒng)的建立8、基于ArcGIS的FFG介紹9、基于ArcGIS平臺(tái)開發(fā)的ArcCREST介紹基于水利大數(shù)據(jù)的多災(zāi)害信息集成與風(fēng)險(xiǎn)預(yù)警案例介紹3小時(shí)臨近預(yù)

5、報(bào)（250米2.5分鐘）36小時(shí)模型預(yù)報(bào)（1公里小時(shí)）1.天、地、空、海多基多源降水?dāng)?shù)據(jù)采集 雙偏振雷達(dá)衛(wèi)星站點(diǎn)模型PERSIANN 全球衛(wèi)星產(chǎn)品(4km, hourly)Hong et al., 2004, JAM;5顆地球靜止衛(wèi)星（可見光紅外）以及4顆極軌衛(wèi)星（雷達(dá)和被動(dòng)微波）通過(guò)人工神經(jīng)網(wǎng)絡(luò)ANN機(jī)器學(xué)習(xí)訓(xùn)練反演 High Quality 衛(wèi)星降水產(chǎn)品Merge Satellites, ground (Radar & Gauge), and Model (NWP)TRMMAquaDMSPNOAAMETEOSAT(Europe)GOESGMS/MTSAT(Japan)TMPA uses 4

6、 Polar-orbital microwave satellites (NOAA, DoD, NASA) and 5 Geo-IR satellites(GOES8-10, GMS, MYSAT, MeteoSAT); all calibrated by TRMM Preci Radar17+ years (98-16) of data; Most requested TRMM product from NASAWith Huffman et al. 2007 : （1700+ 引用）2005 加入 NASA：多衛(wèi)星聯(lián)合反演共性技術(shù)；(1700+引用)全球天地空標(biāo)準(zhǔn)產(chǎn)品系列：TMPA30-d

7、ay HQ coefficientsInstant- aneous SSM/I TRMM AMSR AMSU3-hourly merged HQHourly IR TbHourly HQ-calib IR precip3-hourly multi- satellite (MS)Monthly gaugesMonthly SGRescale 3-hourly MS to monthly SGRescaled 3-hourly MSCalibrate High-Quality (HQ) Estimates to “Best” Space RadarMerge HQ EstimatesMatch I

8、R and HQ, generate coeffsApply IR coefficientsMerge IR, merged HQ estimatesCompute monthly satellite-gauge combination (SG)30-day IR coefficients26深度學(xué)習(xí)方法研制全球衛(wèi)星產(chǎn)品研制青藏西南部IR云圖相應(yīng)時(shí)段降水情況在深度學(xué)習(xí)中，我們可以將不同頻段的可見光、紅外、微波影像同時(shí)作 為訓(xùn)練數(shù)據(jù)輸入模型，且不需要事先設(shè)定Feature，海量的遙感影像下，讓 模型自己去尋找Feature。5-minute 250mRainfall Dataover USA2.

9、 mPING 美國(guó)版災(zāi)害Crowdsourcing移動(dòng)平臺(tái)技術(shù)2.移動(dòng)眾包信息收集可視化云平臺(tái) mPING Crowd Sourcing Tool and Data750,000+ App Downloads Since Dec 2013硅谷SF IoT/BigData Weather 2.0 Service Inc.Ensemble Coupled Hydro- Landslide Modeling SystemWater Balance ComponentCREST (Variable Infiltration Curve)SAC-SMARunoff RoutingCell-by-cell

10、 linear reservoirLandslide Model EnsembleTRIGRSSLIDE+Surface Flow and InundationSoil Water ContentOther variablesOccurrence and Locations of landslidesRemote Sensing basedPrecipitation EstimatesTopographyLand cover/Land Use3.基于水利大數(shù)據(jù)的全球水洪泥石流災(zāi)害預(yù)測(cè)預(yù)報(bào) National Flash Landslide System3. 基于水利大數(shù)據(jù)的全球水洪泥石流災(zāi)害預(yù)測(cè)

11、預(yù)報(bào)美國(guó)暴雨山洪泥石流災(zāi)害鏈業(yè)務(wù)化系統(tǒng)NFL:NMQ：National Mosaic and Multi-Sensor QPE (NMQ)FLASH：Flooded Locations And Simulated HydrographsLANDSLIDE:SLope-Infiltration-Distributed Equilibrium ModelNMQ Radar Precipitation Observations 250 m/2.5 minFLASH Distributed CRESTHydrologic Models10-11 June 2010, Albert Pike RecAr

12、ea, Arkansas250 mm150200Simulated surface water flow20fatalitiesLANDSLIDELandslide Hotspot ModelsRed: ObservationsPink: PredictionsLandslide predictionIntegrated Hydrologic-Landslide Model iCRESLIDE = CREST + SLIDECoupled Routing and Excess STorage (CREST)Jointly developed by OU/NASARun operationall

13、y overglobeDistributed, fully coupled runoff generation and routingWang amnoddHelong et al. 2011 HSJIntegrated Hydrologic-Landslide Model: iCRESLIDEDevelopment and Application- CREST has been set up at both national and basin scales in China;- iCRESLIDE shows great capability in forecasting shallow

14、landslides around the world;- More flood and landslide event data is needed.250m/5-min resolution of Q2 precipitation forcing and model outputsAddresses service needs in NWS; flash flooding is #1 weather-related killer6/11 12:30am-4am 20 deaths: Little Missouri River Crested from 3 ft to 23.5 ft wit

15、hin 2 hoursInclude data assimilation and probabilistic productsReadily incorporate dual-pol radar products (Q3) and stormscale ensemble forecastsNFL: Real-time, direct prediction of flash floods a realityPhoto source: National Geographic美國(guó)暴雨山洪泥石流災(zāi)害鏈耦合系統(tǒng)核心模型Physically-coupled iCRESTSLIDE (SLope Infil

16、tration- Distributed Equilibrium)020408010012000.210.80.60.460Radius (m)POD FAR CSIValidation with inventory dataRed: Observations Pink: Predictions美國(guó)北卡州 梅肯縣Within 18-m 120-meter buffer zonePOD 0.50.9CSI 0.10.8FAR 0.90.2(Liao et al., 2011, Nat. Hazards )16th hrFS Map vs. Time18th hr21st hrForecastSt

17、reamflow (2010)Recurrence Interval(2010)Inundation (2015)State-Param EstimationDREAM (2010)Observed StreamflowGroundwaterMODFLOWRoutingKinematic wave (2014)Linear reservoir (2010)4.基于概率洪水風(fēng)險(xiǎn)預(yù)報(bào) EF5Ensemble Framework ForFlashFlood ForecastingBest distributed hydrologic System yetPrecip ForcingMRMSTMPA

18、RTWRR/HRRR QPFEvapotranspirationFEWS NET PETHRRR tempVIIRS?Surface RunoffCREST (2010)SAC-SMA (2013)Hydrophobic (2015)SnowmeltSNOW-17 (2015)-2m TempCurrent VersionFutureAdditionEF5: Probability of Flash Flood Forecast (PFFF)基于概率洪水風(fēng)險(xiǎn)預(yù)報(bào)100%50%0%PFFF( RP = 5 yr )The New Features of uCREST Model1-10 Mete

19、r DEM and Urban Drainage SystemUrbanCanopyandHighRiseBuildingImpacton theRainfallInterceptionEnhanced Impervious (pavement, roof etc.) and Non-impervious surface infiltration and Surface Processes (runoff, ET etc)UrbanSewer/PipelineModuleincludedasaspecialInterflowProcess/reservoirHas been tested an

20、d implemented in Oklahoma City and Dallas Metropolitan at spatial resolution5.城市洪水模型UrbanCREST介紹A High-Resolution UrbanCREST Flood Modeling and Mapping System For Urban and Built-up Environments2010 June 14, OKC Flash Flood101 km1Return Period (years) 210200+No FloodingFloodingSevereFloodingUrban-CR

21、EST Flood Model Implemented at Oklahoma City &Dallas Metropolitan137 km6.全球風(fēng)暴數(shù)據(jù)庫(kù)及CI-FLOW Global Storms (2000-2010)*Sellars et al. (2013), Computational Earth Science: Big Data Transformed Into Insight, EOS Trans. AGU, 94(32),277Nov 2011 BAMSThe CI-FLOW Project:A System for Total Water Level Predicti

22、on From The Summit To The SeaCI-FLOW summary paper with Hurricane Isabel, Hurricane Earl, & Tropical Storm Nicole resultsVolume # Number # November 2011BAMSAmerican Meteorological SocietySuzanne Van Cooten, , Yang Hong, et al., 2011: The ci-flow project: a system for total water level prediction fro

23、m the summit to the sea. Bull. Amer.Meteor. Soc., 92, 14271442.已應(yīng)用到美國(guó)北卡羅來(lái)納州、墨西哥灣等易受颶風(fēng)和風(fēng)暴潮影響的海岸帶地區(qū)海洋風(fēng)暴潮與內(nèi)陸洪水監(jiān)測(cè)預(yù)警系統(tǒng)(CI-FLOW)CI-FLOWCoastal and Inland Flooding Observation and WarningTracking the raindrops and disasters from the SKY and the SUMMIT to the seaCI-FLOW: HL-RDHM/SWAN/ADCIRC Coupled ModelPre

24、cipitationSig. Wave HeightsTotal Water LevelsRiver BCsDischargeSurface BCsPressure Wind ForcingSurface BCsWave ForcingHydrodynamic Model (ADCIRC)HydrologicModelAtmospheric ModelWave ModelPrecipitation Source: QPE/QPF Atmospheric Model: NAM or NHC trackHydrologic Model: HL-RDHM, Vflo or CRESTWave Mod

25、el: unstructured SWAN7.中國(guó)區(qū)域多尺度洪水模擬及預(yù)警系統(tǒng)的建立中國(guó)的山洪預(yù)警系統(tǒng)量融合，驅(qū)動(dòng)CREST模型，模擬徑流分布與氣象局以及國(guó)家氣象中心合作開發(fā)多源降水產(chǎn)品和地面臺(tái)站數(shù)據(jù)進(jìn)行雨地貌水動(dòng)力學(xué)模型模擬洪水淹沒(méi)情景的時(shí)空演進(jìn)，實(shí)時(shí)動(dòng)態(tài)提取洪水淹沒(méi)范圍、水深分布和淹沒(méi)時(shí)間分布， 實(shí)現(xiàn)對(duì)洪水的模擬洪水模擬的時(shí)間：199806280501001502002503000500010000150002000025000Date 3/5/19975/8/19977/11/19979/13/199711/16/19971/19/19983/24/19985/27/19987/30/1

26、99810/2/199812/5/19982/7/19994/12/19996/15/19998/18/199910/21/199912/24/19992/26/20004/30/20007/3/20009/5/200011/8/20001/11/20013/16/20015/19/20017/22/20019/24/200111/27/20011/30/20024/4/20026/7/20028/10/200210/13/200212/16/20022/18/20034/23/20036/26/20038/29/200311/1/20031/4/20043/8/20045/11/20047/

27、14/20049/16/200411/19/20041/22/20053/27/20055/30/20058/2/200510/5/200512/8/2005R_Obs in (m3/s)R(v2.1) in (m3/s)rain率定期驗(yàn)證期NSCE=0.897CC=0.947Bias=-1.57%20 年、10 年、5年、2年、1年 一遇洪水外州站CREST模型率定/模擬效果：氣象臺(tái)站數(shù)據(jù)驅(qū)動(dòng)7.中國(guó)區(qū)域多尺度洪水模擬及預(yù)警系統(tǒng)的建立114114.5115115.5116116.51172525.52626.52727.52828.529iMAP 在嘉陵江流域的應(yīng)用結(jié)果7.中國(guó)區(qū)域多尺度洪水

28、模擬及預(yù)警系統(tǒng)的建立9.基于ArcGIS平臺(tái)開發(fā)的ArcCREST介紹 ArcCREST UIPrecip ThiessenEvap ThiessenGeo DataUsed for rainfall sites (Cell-based data need some effort)Parameters distribution need more advanced methodBugs in code, the results are not correctGeo and Hydro data management and operationParameters distribution se

29、ttingModel running and results showArcCREST運(yùn)行結(jié)果分析ArcCRESTv1.0(Uncalib)ArcCRESTv1.0Nash-Sutliffe-0.415460.8121Bias (%)-99.999915.25CC0.79630.8382300200100040050011325374961738597109121133145157169181193205217229241253265277289301313325337349361Discharge(m3)Time(24h)Discharge: ArcCREST vs GageCalibUnCalibActualR=0.70255010015020025030000501001502002