Updating a property of all the nodes in neo4j db resulted in out of memory

Question

My graph db has 3.5 million nodes size of the database is 1.6GB i am trying to update a property for all the nodes via neo4jshell with following query.

Match (p:Person) set p.regId= toInt(p.regId) ;

Before doing this i have added index on Person for property regId. During the execution the following error was thrown

java.lang.MemoryError: GC overhead limit exceeded

Answer 1

All changes performed by a single Cypher statement are executed in the same transaction. A transaction builds up in memory and gets persisted when you close it.

I guess your transaction here grows to large and therefore resulting in a memory error.

The usual strategy to deal with this is to use LIMIT on the cypher statement to have a defined size, report back the number of changes done and run the statement x times until the return value is 0.

In your case:

Match (p:Person) 
where p.regId <> toInt(p.regId)
with p limit 10000
set p.regId= toInt(p.regId) 
return count(p)

Answer 2

Here is a description of what's causing the error. Basically, you're short on memory, an garbage collection isn't finding you any extra free memory.

In the neo4j performance tuning guide there's a lot of guidance on how to tweak memory.

The first thing to try is to give your JVM more memory; for the shell you need to set something like JAVA_OPTS=-Xmx1024m before starting the shell to tweak how much memory the JVM can use, this increases the heap size.