Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to store a large graph on disk

I have a large graph of a few 100 GBs so I can't store it in RAM. The graph has multi-edges and there are labels for each edge. I would like to perform queries such as the following:

  1. Show me all edges that come from a specified node.
  2. Show me all edges between two given nodes.
  3. Pick 1000 random pairs of nodes that are connected and show me all edges between them.
  4. Pick 1000 nodes chosen at random.

What's a good way to store a graph on disk for these sorts of queries?

This is for a single high performance PC not a distributed setting. My preferred programming language is Python.

like image 543
graffe Avatar asked Jul 29 '16 20:07

graffe


People also ask

How do you store graph data?

Graph data is kept in store files, each of which contain data for a specific part of the graph, such as nodes, relationships, labels and properties. Dividing the storage in this way facilitates highly performant graph traversals (as detailed above).


1 Answers

You can try to use Neo4J graph database. It has support for Python language and rich query syntax which will give you an ability to perform all the queries you mentioned.

like image 66
SerCe Avatar answered Sep 22 '22 12:09

SerCe