Export whole database in Cypher format (ASCII text)?

Question

Is there a way to export an entire Neo4J database in Cypher, resulting in an ASCII file of the Cypher commands which could be used on an empty Neo4J database to re-create the original database? Since Neo4J is undergoing such rapid development, I worry about using the built-in backup functionality (of the enterprise version).

For example, with Oracle, you can export the whole database in SQL*PLUS DML/DDL commands, which can be useful.

Answer 1

As of Neo4j 2.0 there is a dump command in neo4j-shell that does this. You can dump the results of a specific query or the entire database. By passing the dump command as an argument when starting neo4j-shell you can redirect the output to file to make a 'cypher create script' or to another neo4j-shell session recreating all or parts of the graph in a different database.

Flatten and dump the results of a query

neo4j-sh$ dump MATCH (n:StackOverflow:Question {questionId:18830686})<-[:ANSWERS]-(answer)<-[:WRITES]-(user) RETURN *; 

Dump entire database to a file

usr@term: bin/neo4j-sh -c dump > ./backup/$(date +"%y%m%d_%H%M%S").cypher 

Pipe the dump to another shell session & database

usr@term: db1/bin/neo4j-sh -path db1/data/graph.db/ -c dump | db2/bin/neo4j-shell -path db2/data/graph.db/ 

---

Caveat emptor

There were some issues with doubles and floats being exported in scientific notation, which neo4j-shell couldn't interpret again on import (SO, github) and there were some problems with escaping "quoted strings" (github). I think these are both resolved, so if you have trouble you may want to check out a recent build.

And finally there is one problem that I don't think is resolved yet. Recently schema was included in the dump so create index and create constraint statements are exported as well. However, all the exported statements are framed in one and the same transaction in the output and neo4j won't let you create schema and data in the same transaction. So if you pipe the dump directly into another shell session to recreate the graph, you're likely to run into

> ; ConstraintViolationException: Cannot perform data updates in a transaction that has performed schema updates. neo4j-sh (?)$ commit Failed to commit, transaction rolled back 

It's easy to work around this by redirecting to a file and manually adding commit and begin after the last schema statement. Be sure to put them each on a new line, it should look something like

... create index on :`Person`(`name`) commit begin create (_0:`Person` {`name`:"Paul"}) ... 

Or you can edit the output from neo4j-shell on the fly and add it there, for instance if you dump programmatically and don't want to edit manually. On osx I have used sed like so

db1/bin/neo4j-shell -path db1/data/graph.db/ -c dump | sed -E 's/create (index|constraint) on .*/&\'$'\ncommit''\'$'\nbegin/' | db2/bin/neo4j-shell -path db2/data/graph.db/ 

This adds a commit after each schema statement, which is more than what is necessary (could commit all schema statements together), but hey, it works.

Answer 2

This should solve your problem:

./neo4j-shell -c dump > export_data.cypher 

-c options tells the neo4j-shell that we are passing a neo4j shell command and that we want the output to stdout (outside the neo4-shell) in the bash-shell. We can then pipe the output to any dump file which is export_data.cypher in this case (created automatically in the same directory where you ran the above command).

Not recommended:

./neo4j-shell $dump 

Using the above prints all the cypher statements to stdout inside the shell which you can't pipe out easily, so it is not so useful unless it is very small.

Note: As @jjaderberg pointed out in his answer the dump command without any conditions exports all the Cypher statements for the entire graph.

Answer 3

Dump it using neo4j-shell -c dump > dump.cypher

Then you can try to import it using neo4j-shell -v -file dump.cypher, but for huge dumps you can meet exceptions, when neo4j unable to upload a lot of data in one transaction. It may look like:

at org.neo4j.cypher.internal.frontend.v2_3.bottomUp$BottomUpRewriter.apply(Rewritable.scala:159)

Solution will be to break two big transactions in cypher dump to separate instructions. Here is script to do it:

https://gist.github.com/garmoshka-mo/f27c1884bc1851ebf7b23bf4137095f9

Then you can import, parsing output for warning and errors, if any:

cat dump_converted.cql | neo4j-shell | grep -E 'WARNING:|Unknown\ command'