In my work we are building a huge aplication that will use billions of triples, to optmize the space required to store these triples I've been searching for a different way to represent them, any way that is more economic is welcome. Thanks
There are also a whole class of graph storage systems that don't store things as triples like neo4j. But, I wouldn't rule out triple stores just because they store things as triples ;-) Many of today's current solutions already store billions of triples so its not undo-able (although as you get 1 or 2 orders higher than that things get tough). I've personally filled an Allegrograph store with more than 1 billion.
See this thread: http://www.semanticoverflow.com/questions/3332/scalable-owl-rdf-database
I don't think the space required to store billions of Triples is realistically any worse than the space required to store billions of rows in an SQL database.
The general approach that most systems take whether native stores/SQL based is to assign IDs to nodes and store each triple as just 3 node IDs. Given a good choice of Node ID generation and an efficient index between Node ID and Node value you can easily build stores which scale out massively.
As a further optimisation some stores generate Node IDs in such a way that simple value types (eg. integers, booleans, date times etc) have their value encoded directly into the Node ID so there is no need to do the lookup from ID to value (or vice versa when inserting such data)
As RobV says, almost all stores attach internal values/node-id's to elements of a triple. That being said, a lot of space for a triple store is taken by the various indexes that are required to do lookups. In a relational database you can easily reduce the number of indexes based on the datamodel you are using. In triple stores this is a lot harder, and the stores will basically create a large number (6+) indexes on the different ways that the elements of a triple can be ordered.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With