圖數(shù)據(jù)庫(kù)組會(huì)匯報(bào)

1圖數(shù)據(jù)庫(kù)組會(huì)匯報(bào)報(bào)告人2024年09月25日任務(wù)進(jìn)度

徐柴俊工作進(jìn)度Introduction:動(dòng)機(jī)、方法介紹、例子、貢獻(xiàn)Preliminaries：屬性圖模型、查詢語言、PatternGraphRelatedWork：關(guān)系型數(shù)據(jù)庫(kù)、圖數(shù)據(jù)庫(kù)3徐柴俊工作進(jìn)度Introduction例子視圖創(chuàng)建視圖維護(hù)4n:Commentm:Postr:replyOf*..n:Commentm:Postr:ROOT_POSTn:Comment{id:1}m:Postr:replyOf*..n:Commentm:Postr1:replyOf*..:Comment{id:1}r2:replyOf*0..徐柴俊工作進(jìn)度Introduction例子使用視圖優(yōu)化查詢5c:Commentp:Postr1:replyOf*..f:Forumr2:containerOfc:Commentp:Post@M_r1:ROOT_POSTf:Forumr2:containerOf魏星迪工作進(jìn)度為了在Neo4j里測(cè)試優(yōu)化效果，要把使用視圖優(yōu)化后的PatternGraph反向重寫為Cypher語句。比如match(n:Comment)?[r:replyOf?1..]?>(m:Post)returnn,m這里有兩個(gè)replyOf?1..可以用ROOT_POST優(yōu)化，可以重寫為match(n:Comment)?[r:ROOT_POST]?>(m:Post)returnn,m輸出為優(yōu)化后的查詢語句，測(cè)試時(shí)在需要優(yōu)化的Cypher語句前加上Optimize即可，比如在tugraph中輸入Cypher語句：Optimizematch(n:Comment)?[r:replyOf?1..]?>(m:Post)returnn,m應(yīng)該返回match(n:Comment)?[r:ROOTPOST]?>(m:Post)returnn,mPatterngraphMatch（m）-[r:keyword]->(n),(a)-[z:test]->(b)Anticypher魏星迪工作進(jìn)度Tugraphcypher處理過程魏星迪工作進(jìn)度函數(shù)的輸入為cypher::RTContext?ctx,antlr4::tree::ParseTree?tree,conststd::vector<cypher::PatternGraph>&pattern_graphs,這里的&patterngraphs是優(yōu)化后的PatternGraphs函數(shù)的輸入例如這個(gè)patterngraph其中含有的nodes:m,n,a,b其中含有的relationships： r:keyword z:test此時(shí)通過遍歷patterngraph應(yīng)該得到的cypher語句應(yīng)該是Match（m）-[r:keyword]->(n),(a)-[z:test]->(b)其中r:keyword和z:test可能是通過視圖優(yōu)化后的邊。

魏星迪工作進(jìn)度Neo4j視圖維護(hù)借用neo4j擴(kuò)展apoc觸發(fā)器來維護(hù)視圖match(n:Comment)-[:replyOf*..]->(m:Post)WITHn,mCREATE(n)-[r:ROOT_POST]->(m)魏星迪工作進(jìn)度Neo4j視圖維護(hù)借用neo4j擴(kuò)展apoc觸發(fā)器來維護(hù)視圖{

"match(n:Comment)-[:replyOf*..]->(m:Post)WITHn,mCREATE(n)-[r:ROOT_POST]->(m)":[

[

"match(n:Comment)-[*0]-(viewvetex)-[:replyOf*..]->(m:Post)WITHn,mmatch(n)-[r:ROOT_POST]->(m)WITHn,m,collect(r)asviewdeleteview[0]",

"match(n:Comment)-[:replyOf*..]->(m:Post)-[*0]-(viewvetex)WITHn,mmatch(n)-[r:ROOT_POST]->(m)WITHn,m,collect(r)asviewdeleteview[0]",

"match(n:Comment)-[:replyOf*1..]->(viewvetex)->[:replyOf*0..]->(m:Post)WITHn,mmatch(n)-[r:ROOT_POST]->(m)WITHn,m,collect(r)asviewdeleteview[0]"

],

[

"match(n:Comment)-[:replyOf*0..]->()-[viewedge:replyOf]->()-[:replyOf*0..]->(m:Post)WITHn,mCREATE(n)-[r:ROOT_POST]->(m)"

],

[

"match(n:Comment)-[:replyOf*0..]->()-[viewedge:replyOf]->()-[:replyOf*0..]->(m:Post)WITHn,mmatch(n)-[r:ROOT_POST]->(m)WITHn,m,collect(r)asviewdeleteview[0]"

]

]}rewrite_cypher.json魏星迪工作進(jìn)度Neo4j視圖維護(hù)借用neo4j擴(kuò)展apoc觸發(fā)器來維護(hù)視圖CALLapoc.trigger.add('triggerOnEdgeDelete','UNWIND$deletedRelationshipsASviewedge//只匹配isview屬性為空的邊WITHviewedgeWHEREviewedge.isviewISNULLMATCH(n:Comment)-[:replyOf*0..]->()-[viewedge:replyOf]->()-[:replyOf*0..]->(m:Post)WITHn,mMATCH(n)-[r:ROOT_POST]->(m)WITHn,m,collect(r)ASviewDELETEview[0]',{phase:"before"})魏星迪工作進(jìn)度目前進(jìn)度修改了tugraph中的視圖維護(hù)語句邏輯，從特定點(diǎn)和邊的語句改成了模板類型，訪問器只需要接受視圖語句便可以輸出針對(duì)該視圖的重寫語句neo4j的視圖維護(hù)部分增添邊部分可以成功維護(hù)借助apoc.trigger，并且可以通過python腳本完成多個(gè)視圖觸發(fā)語句的部署問題在刪除部分,本來添加了一條屬性{isview}防止其遞歸調(diào)用觸發(fā)器，在手動(dòng)測(cè)試維護(hù)邏輯，可以正常維護(hù)，但apoc.trigger在刪除時(shí)，phase選用before，neo4j也會(huì)認(rèn)為邊已被刪除，導(dǎo)致后續(xù)維護(hù)邏輯無法進(jìn)行。CALLapoc.trigger.install('myTrigger','UNWIND$deletedRelationshipsASrWITHrWHERENOTr.is_view=true//視圖維護(hù)語句.....

魏星迪工作進(jìn)度neo4j測(cè)試方法框圖魏星迪工作進(jìn)度rewrite_visitor模塊：根據(jù)保存在test_in.txt中的視圖創(chuàng)建語句，生成其對(duì)應(yīng)的視圖維護(hù)語句，其中包括刪除

點(diǎn)，增加邊，刪除邊的維護(hù)語句。輸入：視圖創(chuàng)建語句輸出output.json，視圖維護(hù)語句魏星迪工作進(jìn)度rewrite_visitor生成規(guī)則：對(duì)于點(diǎn)的刪除，要將可變長(zhǎng)的邊進(jìn)行展開再討論node的位置，對(duì)于一條可變長(zhǎng)邊[?x..y]可將其展開為：?Match(n)?[?0]?>(k)?[?x..y]?>(m)?Match(n)?[?1]?>(k)?[?{x?1..{y?1]?>(m)...?Match(n)?[x]?>(k)?[?0..{y?x}

]?>(m)?Match(n)?[?{x+1}]?>(k)?[?0..{y?x?1]?>(m)...?Match(n)?[?y]?>(k)?[?0]?>(m)----------------------------------------------------?Match(n)?[?0]?>(k)?[?{x?1}..]?>(m)?Match(n)?[?1]?>(k)?[?{x?2}..]?>(m)...?Match(n)?[?x..]?>(k)?[?0..]?>(m)對(duì)于無上界的可變長(zhǎng)邊[?x..]可將其展開為：?Match(n)?[*0]?>(src)?[r]?>(dst)?[*{x?1}..]?>(m)?Match(n)?[*1]?>(src)?[r]?>(dst)?[*{x?2}..]?>(m)...?Match(n)?[*i]?>(src)?[r]?>(dst)?[*{x?1?i}..]?>(m)...?Match(n)?[*{x?1}..]?>(src)?[r]?>(dst)?[*0..]?>(m)對(duì)于有上界的可變長(zhǎng)邊[*x..y]可將其展開為：?Match(n)?[*0]?>(src)?[r]?>(dst)?[*{x?1}..{y?1}]?>(m)?Match(n)?[*1]?>(src)?[r]?>(dst)?[*{x?2}..{y?2}]?>(m)...?Match(n)?[*i]?>(src)?[r]?>(dst)?[*{x?1?i}..{y?1?i}]?>(m)...?Match(n)?[*{x?1}]?>(src)?[r]?>(dst)?[*0..{y?x}]?>(m)?Match(n)?[*x]?>(src)?[r]?>(dst)?[*0..{y?x?1}]?>(m)...?Match(n)?[*{y?1}]?>(src)?[r]?>(dst)?[*0]?>(m)魏星迪工作進(jìn)度rewrite_visitor模塊：例：test_in.txt->”match(n:Comment)-[:replyOf*..]->(m:Post)WITHn,mCREATE(n)-[r:ROOT_POST]->(m)”output.json:{"match(n:Comment)-[:replyOf*..]->(m:Post)WITHn,mCREATE(n)-[r:ROOT_POST]->(m)":[[

"optionalmatch(n:Comment)-[*0]-(viewvetex)-[:replyOf*..]->(m:Post)WITHn,m,viewvetexoptionalmatch(n)-[r:ROOT_POST]->(m)WITHn,m,viewvetex,collect(r)asviewdeleteview[0]",

"optionalmatch(n:Comment)-[:replyOf*..]->(m:Post)-[*0]-(viewvetex)WITHn,m,viewvetexoptionalmatch(n)-[r:ROOT_POST]->(m)WITHn,m,viewvetex,collect(r)asviewdeleteview[0]","optionalmatch(n:Comment)-[:replyOf*1..]->(viewvetex)->[:replyOf*0..]->(m:Post)WITHn,m,viewvetexoptionalmatch(n)-[r:ROOT_POST]->(m)WITHn,m,viewvetex,collect(r)asviewdeleteview[0]"],[

"optionalmatch(n:Comment)-[:replyOf*0..]->()-[viewedge:replyOf]->()-[:replyOf*0..]->(m:Post)WITHn,m,viewedgeCREATE(n)-[r:ROOT_POST]->(m)"],["optionalmatch(n:Comment)-[:replyOf*0..]->()-[viewedge:replyOf]->()-[:replyOf*0..]->(m:Post)WITHn,m,viewedgeoptionalmatch(n)-[r:ROOT_POST]->(m)WITHn,m,viewedge,collect(r)asviewdeleteview[0]"]]}魏星迪工作進(jìn)度neo4j_test模塊：根據(jù)保存在output.json中的rewrite_visitor的輸出也就是

視圖維護(hù)語句以及需要運(yùn)行的cypher語句（可以選擇將

其保存在input.txt中）產(chǎn)生新的附帶有視圖維護(hù)語句的實(shí)

際執(zhí)行語句。輸入：視圖維護(hù)語句，需要運(yùn)行的cypher語句輸出new_cypher->帶有視圖維護(hù)語句的cypher語句魏星迪工作進(jìn)度neo4j_test模塊：new_cypher生成規(guī)則：在neo4j_test中定義了如圖中的四個(gè)列表，分別為view(保

存視圖創(chuàng)建語句），delete_nodes(保存視圖刪除點(diǎn)的維護(hù)

語句），add_relas（添加邊的維護(hù)語句），delete_relas

（刪除邊的維護(hù)語句）。其結(jié)果由其中的輸入（output.

json）得到。根據(jù)input.txt中取得的cypher語句中的不同關(guān)鍵字如，delete，create來判別此條cypher語句的目的，并且由此得到不同的添加了視圖維護(hù)語句的new_cypher。test_cypher+$connection_cypher+$UNWIND+∑withviewedge+add_relas[i]ifadd_relasnew_cypher=

$connection_cypher+$UNWIND+∑withviewedge+dele_relas[i]+test_cypherifdele_relas

$connection_cypher+$UNWIND+∑withviewvetex+dele_node[i]+test_cypherifdele_nodesconnection_cypher=with[vars]asdeviewsUNWIND=UNWINDdeviewsasviewedge/viewvetexnew_cypher生成實(shí)例add_relasdelete_relasdelete_nodesoptionalmatch(n:Comment)-[:replyOf*0..]->()-[viewedge:replyOf]->()-[:replyOf*0..]->(m:Post)WITHn,m,viewedgeCREATE(n)-[r:ROOT_POST]->(m)optionalmatch(n:Comment)-[:replyOf*0..]->()-[viewedge:replyOf]->()-[:replyOf*0..]->(m:Post)WITHn,m,viewedgeoptionalmatch(n)-[r:ROOT_POST]->(m)WITHn,m,viewedge,collect(r)asviewdeleteview[0]

"optionalmatch(n:Comment)-[*0]-(viewvetex)-[:replyOf*..]->(m:Post)WITHn,m,viewvetexoptionalmatch(n)-[r:ROOT_POST]->(m)WITHn,m,viewvetex,collect(r)asviewdeleteview[0]","optionalmatch(n:Comment)-[:replyOf*..]->(m:Post)-[*0]-(viewvetex)WITHn,m,viewvetexoptionalmatch(n)-[r:ROOT_POST]->(m)WITHn,m,viewvetex,collect(r)asviewdeleteview[0]","optionalmatch(n:Comment)-[:replyOf*1..]->(viewvetex)->[:replyOf*0..]->(m:Post)WITHn,m,viewvetexoptionalmatch(n)-[r:ROOT_POST]->(m)WITHn,m,viewvetex,collect(r)asviewdeleteview[0]"test_cyphernew_cyphermatch(n:Comment{CommentId:"2061584302086"})-[r:replyOf]->(m:Post{id:"2061584302084"})withrlimit1deletermatch(n:Comment{CommentId:"2061584302086"})-[r:replyOf]->(m:Post{id:"2061584302084"})withrlimit1WITH[r]asdeleviewsUNWINDdeleviewsasviewedgeoptionalmatch(n:Comment)-[:replyOf*0..]->()-[viewedge:replyOf]->()-[:replyOf*0..]->(m:Post)WITHn,m,viewedgeoptionalmatch(n)-[r:ROOT_POST]->(m)WITHn,m,viewedge,collect(r)asviewdeleteview[0]withviewedgematch(n:Comment{CommentId:"2061584302086"})-[r:replyOf]->(m:Post{id:"2061584302084"})withrlimit1deletermatch(n:Comment{CommentId:\"2061584302086\"}),(m:Post{id:\"2061584302084\"})withn,mcreate(n)-[:replyOf{creationDate:1266604713724}]->(m)match(n:Comment{CommentId:\"2061584302086\"}),(m:Post{id:\"2061584302084\"})withn,mcreate(n)-[r:replyOf{creationDate:1266604713724}]->(m)WITH[r]ascreateviewsUNWINDcreateviewsasviewedgeoptionalmatch(n:Comment)-[:replyOf*0..]->()-[viewedge:replyOf]->()-[:replyOf*0..]->(m:Post)WITHn,m,viewedgeCREATE(n)-[r:ROOT_POST]->(m)match(n:Comment{CommentId:\"2061584302086\"})-[r:replyOf]->(m:Post{id:\"2061584302084\"})returnn,r,mmatch(n:Comment{CommentId:\"2061584302086\"})-[r:replyOf]->(m:Post{id:\"2061584302084\"})returnn,r,m魏星迪工作進(jìn)度neo4j_test.py執(zhí)行流程魏星迪工作進(jìn)度特殊情況m:Accountn1n:Loann3VIEW:MATCH(n:Account)-[r1*..2]->(m:Loan)whereall(relinr1whererel.is_viewisnull)withn,mCREATE(n)-[r:ACCOUNT_LOAN{is_view:true}]->(m)optionalmatch(n:Account)-[ANONR2*1]->()-[view_edge]->()-[ANONR3*0]->(m:Loan)WHEREALL(relINANONR2WHERErel.is_viewisnull)

ANDview_edge.is_viewisnull

ANDALL(relINANONR3WHERErel.is_viewisnull)

withn,m,view_edgeCREATE(n)-[r:ACCOUNT_LOAN{is_view:true}]->(m)view_edger1r2魏星迪工作進(jìn)度特殊情況VIEW:MATCH(n:Account)-[r1*..2]->(m:Loan)whereall(relinr1whererel.is_viewisnull)withn,mCREATE(n)-[r:ACCOUNT_LOAN{is_view:true}]->(m)optionalmatch(n:Account)-[ANONR2*1]->()-[view_edge]->()-[ANONR3*0]->(m:Loan)WHEREALL(relINANONR2WHERErel.is_viewisnull)ANDview_edge.is_viewisnullANDALL(relINANONR3WHERErel.is_viewisnull)withn,m,view_edgeoptionalmatch(n)-[r:ACCOUNT_LOAN{is_view:true}]->(m)withcollect(r)asviews,n,m,view_edgedeleteviews[0]m:Accountn1n:Loanview_edger1r2n3魏星迪工作進(jìn)度修改維護(hù)語句

無法維護(hù)上述特殊情況可以解決上述情況，但無法傳遞view_edge可以解決上述情況并傳遞view_edgeoptionalmatch(n:Account)-[ANONR2*1]->()-[view_edge]->()-[ANONR3*0]->(m:Loan)WHEREALL(relINANONR2WHERErel.is_viewisnull)ANDview_edge.is_viewisnullANDALL(relINANONR3WHERErel.is_viewisnull)withn,m,view_edge

optionalmatch(n)-[r:ACCOUNT_LOAN{is_view:true}]->(m)withcollect(r)asviews,n,m,view_edgedeleteviews[0]optionalmatch(n:Account)-[ANONR2*1]->()-[view_edge]->()-[ANONR3*0]->(m:Loan)WHEREALL(relINANONR2WHERErel.is_viewisnull)ANDview_edge.is_viewisnullANDALL(relINANONR3WHERErel.is_viewisnull)withn,m,count(*)asnummatch(n)-[r:ACCOUNT_LOAN{is_view:true}]->(m)withnum,collect(r)asviewsUNWIND

range(0,num-1)

AS

ideleteviews[i]optionalmatch(n:Account)-[ANONR2*1]->()-[view_edge]->()-[ANONR3*0]->(m:Loan)WHEREALL(relINANONR2WHERErel.is_viewisnull)ANDview_edge.is_viewisnullANDALL(relINANONR3WHERErel.is_viewisnull)withn,m,collect(view_edge)asview_edgeswithn,m,size(view_edges)asnum,view_edgesoptionalmatch(n)-[r:ACCOUNT_LOAN{is_view:true}]->(m)withcollect(r)asviews,n,m,num,view_edgesUNWINDrange(0,num-1)asideleteviews[i]維護(hù)加速比0.20.6魏星迪工作進(jìn)度snb_resultsnb測(cè)試案例無優(yōu)化查詢時(shí)間視圖優(yōu)化時(shí)間優(yōu)化后查詢時(shí)間加速比記錄條數(shù)"match(n0:Comment)-[r1:replyOf*..]->(n1:Post)match(n1)<-[r2:replyOf*..]-(n2:Comment)whereid(n0)<>id(n2)returncount(n1)"41.160.0143421.951.874925941136"match(n0:Place{name:$placeName})<-[r1:isPartOf]-(n1:Place)match(n0)<-[r2*2]-(n2:Comment)returncount(n2)"148.630.0126939.103.801257428000"match(n0:Place)<-[r1:personIsLocatedIn]-(n1:Person)match(n1)-[r2]->(n2)-[r3]->(n3:Place)returncount(n3)"6.89320.013711.56364.40852410213"match(n0:Place)<-[r1*2]-(n1:Comment)

match(n1)-[r2:replyOf*..]->(n2:Post)returncount(n2)"21.09740.01267.11322.96596156507"match(n0:Place{name:$placeName})<-[r1]-(n1)<-[r2]-(n2:Person)match(n0)<-[r3*2]-(n3:Comment)returncount(n3)"140.63640.0095132.91281.05818171747840"match(n0:Tag)<-[r1:hasInterest]-(n1:Person)

match(n1)-[r2]->(n2)-[r3]->(n3:Place)wheren0.hasType>$tagTypereturncount(n3)"29.28480.013221.63341.353641324343"match(n0:Comment)-[r1:replyOf*..]->(n1:Post)match(n1)-[r2:postHasCreator]->(n2:Person)match(n2)-[r3]->(n3)-[r4]->(n4:Place)wheren0.creationDate<$creationDatereturncount(n4)"70.95460.009055.92561.2687100784775魏星迪工作進(jìn)度snb_resultsnb測(cè)試案例無優(yōu)化查詢時(shí)間視圖優(yōu)化時(shí)間優(yōu)化后查詢時(shí)間加速比記錄條數(shù)"MATCREATE(p1)-[:knows{creationDate:$creationDate,weight:$weight}]->(p2)"CH(p1:Person{id:$person1Id}),(p2:Person{id:$person2Id})0.01726400.02930250.5891704"result":"[]","time":493,"add_nodes":0,"delete_nodes":0,"add_rel":2,"delete_rel":0"MATCH(n:Person{id:$personId})-[r:likes]->(m:Comment{id:$commentId})DELETEr"0.00715800.01370330.5224040"result":"[]","time":613,"add_nodes":0,"delete_nodes":0,"add_rel":0,"delete_rel":2"MATCH(n:Post{id:$postId})

DETACHDELETEn"0.0128200.01582880.8099682"result":"[]","time":1092,"add_nodes":0,"delete_nodes":1,"add_rel":0,"delete_rel":26魏星迪工作進(jìn)度snb_result魏星迪工作進(jìn)度snb_resultfinbench測(cè)試案例無優(yōu)化查詢時(shí)間視圖優(yōu)化時(shí)間優(yōu)化后查詢時(shí)間加速比記錄條數(shù)"match(n0:Comment)-[r1:replyOf*..]->(n1:Post)match(n1)<-[r2:match(n0:Person)-[r1*2]->(n1:Company)match(n1)<-[r2*2]-(n2:Person)whereid(n0)<>id(n2)returncount(n1)47.30380.009027.62561.71233676match(n0:Company)-[r1]->(n1)-[r2]->(n2:Loan)match(n2)<-[r3:repay]-(n3:Account)returncount(n3)33.02160.010310.01363.29763614007match(n0:Person)-[r1*2]->(n1:Company)match(n0)-[r2:own]->(n2:Account)returncount(n1)26.78360.00996.0644.41685151969match(n0:Medium)-[r1:signIn]->(n1:Account)match(n1)-[r2*..2]->(n2:Loan)returncount(n2)198.19920.0147122.19161.622076828189match(n0:Loan)<-[r1]-(n1)<-[r2]-(n2:Company)match(n2)<-[r3*2]-(n3:Person)returncount(n3)52.05680.012811.59464.489714601110match(n0:Account)<-[r1:own]-(n1:Company)match(n1)-[r2]->(n2)-[r3]->(n3:Loan)returncount(n3)34.25040.01063.83048.94172724147match(n0:Account)-[r1*..2]->(n1:Loan)match(n1)<-[r2]-(n2)<-[r3]-(n3:Company)returncount(n1)129.25340.013129.77024.341739595119魏星迪工作進(jìn)度snb_resultfinbench測(cè)試案例無優(yōu)化查詢時(shí)間視圖優(yōu)化時(shí)間優(yōu)化后查詢時(shí)間加速比記錄條數(shù)MATCH(acc:Account{id:$accountId}),(loan:Loan{id:$loanId})CREATE(acc)<-[:deposit{timestamp:$depositTime,amount:$amt}]-(loan)0.014900.01780.8364"result":"[]","time":15,"add_nodes":0,"delete_nodes":0,"add_rel":9,"delete_rel":0MATCH(acc:Account{id:$accountId})-[r:repay{timestamp:$repayTime}]->(loan:Loan{id:$loanId})deleter0.005500.01300.4236"result":"[]","time":15,"add_nodes":0,"delete_nodes":0,"add_rel":0,"delete_rel":8MATCH(loan:Loan{id:$loanId})DETACHDELETEloan0.008100.01590.5074"result":"[]","time":21,"add_nodes":0,"delete_nodes":1,"add_rel":0,"delete_rel":32魏星迪工作進(jìn)度finbench_result魏星迪工作進(jìn)度snb_resultMATCH(acc:Account{id:$accountId}),(loan:Loan{id:$loanId})CREATE(acc)<-[:deposit{timestamp:$de