how to write Spark data frame to Neo4j database

Question

I'd like to build this workflow:

• preprocess some data with Spark, ending with a data frame
• write such dataframe to Neo4j as a set of nodes

My idea is really basic: write each row in the df as a node, where each column value represents the value of the node's attribute

I have seen many articles, including neo4j-spark-connector and Introducing the Neo4j 3.0 Apache Spark Connector but they all focus on importing into Spark data from a Neo4j db... so far, I wasn't able to find a clear example of writing a Spark data frame to a Neo4j database.

Any pointer to documentation or very basic examples are much appreciated.

Answer 1

Read this issue to answer my question.

Long story short, neo4j-spark-connector can write Spark data to Neo4j db, and yes, there is a lack in the documentation of the new release.

Answer 2

you can write some routine and use an opensource neo4j java driver

https://github.com/neo4j/neo4j-java-driver

for example.

Simple serialise the result of an RDD (using rdd.toJson) and then use the above driver to create your neo4j nodes and push into your neo4j instance.