完成第二部分spark速成spark2.1.0版本-7.2sparkgraphx簡介

Spark2.1.0Spark2.1.0：Spark2017329(updated:2017331，，【博客內(nèi)容由廈門大學數(shù)據(jù)庫擁有！返回Spark首頁，，GraphXSparkGraphXSparkRDD.本章節(jié)主要介紹GraphX的抽象模型---屬性圖，并通過實例介紹如何構(gòu)造 屬性圖是GraphX的抽象模型，是一個有向多重圖，帶有每一個頂屬性圖支持平行邊，這簡化了屬性圖的建模場景。每個頂點用唯一的64位長的標識符作為鍵。GraphX沒有為頂點標識符排序，每條邊都有對應(yīng)的源和目的頂?shù)?表示頂點或邊時，GraphX使用特殊數(shù)組來，降低內(nèi)存的消耗classcaseclassUserProperty(valname:String)extendsVertexPropercaseclassProductProperty(valname:String,valprice:Double)extendsVertexProperty//Thegraphmightthenhavethevargraph:Graph[VertexProperty,String]=classGraph[VD,ED]valvertices:valedges:}VertexRDD[VDEdgeRDD[ED]RDD[(VertexId,VD)]和接下來，通過一個實例簡單介紹如何構(gòu)建一個屬性圖 RDDs objectSimpleGraphXdefmain(args:Array[String])日 //設(shè)置運行環(huán) valconf=newSparkConf().setAppName("SimpleGraphX").s valsc=new users頂 valusers:RDD[(VertexId,(String,String))] sc.parallelize(Array((3L,("rxin","student")),(7L,"postdoc")),(5L,("franklin","prof")),(2L,("istoica", //設(shè)置relationships valrelationships:RDD[Edge[String]] sc.parallelize(Array(Edge(3L,7L,"collab"),Edge(5L,3L,"advisor"),Edge(2L,5L,"colleague"),Edge(5L,7L,"pi"))) //定義默認的作者,以防與不存在的作者有relationship valdefaultUser=("JohnDoe", //Buildtheinitial valgraph=Graph(users,relationships,defaultUser)23. *展示圖的屬 printlnstudent graph.vertices.filter{case(id,(name,occupation))=>occupation=="student"}.collect.foreach{ case(id,(name,occupation))=>println(s"$nameis$occ printlnadvisor graph.edges.filter(e=>e.attr===>println(s"${e.srcId}to${e.dstId}att defmax(a:(VertexId,Int),b:(VertexId,Int)):(VertexId,= if(a._2>b._2)aelseb39. println("maxofoutDegrees:"+graph.outDegrees.reduce(max)+"maxofinDegrees:"+graph.inDegrees.reduce(max)+maxofDegrees:"+}EdgeTablesrcIddstId符，還有一個邊屬性。同樣VerticeTable每個頂點也