Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Insert multiple Vertices at once using Cosmos DB Graph-API

I am looking to quickly insert multiple vertices using Azure Cosmos DB Graph-API. Most of the current Microsoft samples create the vertices one by one and execute a Gremlin query for each, like so:

IDocumentQuery<dynamic> query = client.CreateGremlinQuery<dynamic>(graph, "g.addV('person').property('id', 'thomas').property('name', 'Thomas').property('age', 44)");

while (query.HasMoreResults)
{                    
    foreach (dynamic result in await query.ExecuteNextAsync())  {   
        Console.WriteLine($"\t {JsonConvert.SerializeObject(result)}"); 
    }
    Console.WriteLine();
}


query = client.CreateGremlinQuery<dynamic>(graph, "g.addV('person').property('id', 'mary').property('name', 'Mary').property('lastName', 'Andersen').property('age', 39)");

while (query.HasMoreResults)
{                    
    foreach (dynamic result in await query.ExecuteNextAsync())  {   
        Console.WriteLine($"\t {JsonConvert.SerializeObject(result)}"); 
    }
    Console.WriteLine();
}

However this is less than ideal when I want to create a couple thousand vertices and edges to initially populate the graph as this can take some time.

This is with Microsoft.Azure.Graphs library v0.2.0-preview

How can I efficiently add multiple vertices at once to Cosmos DB so I may later query using the Graph API syntax?

like image 454
kmcnamee Avatar asked Jun 02 '17 14:06

kmcnamee


1 Answers

I've found that the fastest way to seed your graph is actually to use the Document API. Utilizing this technique I've been able to insert 5500+ vertices/edges per second on a single development machine. The trick is to understand the format that Cosmos expects for both edges and vertices. Just add a couple vertices and edges to your graph through the gremlin API and then inspect the format of these documents by going to the Data Explorer in Azure and executing a document query to SELECT * FROM c.

At work I've built up a light ORM that uses reflection to take POCOs for edges and vertices and convert them to the format that you see in the portal. I'm hoping to open source this soon, at which point I'll most likely release a Nuget package and accompanying blog post. Hopefully in the meantime this will help point you in the right direction, let me know if you have more questions about this approach.

like image 175
Jesse Carter Avatar answered Oct 09 '22 13:10

Jesse Carter