Warmup Neo4j database after spring initialization

Question

I use a Neo4J database with nearly 500k nodes. When I startup my Spring application and do the first query, it takes about 4-5 seconds. This happens just for the first query, so I thought I could do a warmup after spring is initialized to make all subsequent queries faster.

This is my applicationContext.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:util="http://www.springframework.org/schema/util"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
        http://www.springframework.org/schema/context
        http://www.springframework.org/schema/context/spring-context-2.5.xsd">

    <!-- neo4j database -->
    <util:map id="config">
        <entry key="enable_remote_shell" value="false" />
    </util:map>

    <bean id="graphDbFactory" class="org.neo4j.graphdb.factory.GraphDatabaseFactory" />

    <bean id="graphDbBuilder" factory-bean="graphDbFactory"
        factory-method="newEmbeddedDatabaseBuilder">
        <constructor-arg value="/path/to/db" />
    </bean>

    <bean id="graphDbBuilderFinal" factory-bean="graphDbBuilder"
        factory-method="setConfig">
        <constructor-arg ref="config" />
    </bean>

    <bean id="graphDatabaseService" factory-bean="graphDbBuilderFinal"
        factory-method="newGraphDatabase" destroy-method="shutdown" class="org.neo4j.graphdb.GraphDatabaseService"/>

    <context:component-scan base-package="com.app.components" />
</beans>

I saw that one way to wait for Spring beans initialization was to implement an ApplicationListener, and the regular way to warmup the database with Neo was to call the apoc.runtime.warmup() function, so I did as follows:

package com.app.components;

@Component
public class StartupTasks implements ApplicationListener<ContextRefreshedEvent> {

    private static final Logger LOG = LogManager.getLogger(StartupTasks.class);

    @Autowired
    GraphDatabaseService db;

    @Override
    public void onApplicationEvent(ContextRefreshedEvent event) {
        executeTestSelect();
    }

    private void executeTestSelect() {
        LOG.info("Warming up Neo4j...");

        Transaction tx = db.beginTx();
        db.execute("CALL apoc.warmup.run()");
        tx.close();

        LOG.info("Warmup complete.");
    }
}

This one didn't work, everything gets correctly logged but the first neo4j query is still slow.

Since this approach didn't work, I edited the executeTestSelect method to run a real query and process the results like this:

private void executeTestSelect() {
    String textToSearch = "a"; // returns almost all nodes, should make neo4j cache them all
    Transaction tx = db.beginTx();
    Map<String, Object> params = new HashMap<String, Object>();
    params.put("textToSearch", textToSearch );
    Result resultSet = db.execute("MATCH (n:PROD) WHERE n.description CONTAINS {textToSearch} RETURN n",
                params);
    Iterator<Node> results = resultSet.columnAs("n");
    int count = 0;
    while (results.hasNext()) {
        results.next();
        count++:
    }
    tx.close();
    LOG.info("Neo4j cache done. Processed " + count + " nodes.");
}

This time the startup takes 4-5 seconds just to execute the query, but then it prints

Neo4j cache done. Processed 0 nodes.

This is weird, because the exact same query run when the app is fully initialized returns 450k nodes.

What am I missing? Is it possible that when reaching onApplicationEvent the Neo4j DB hasn't been initialized yet and can't execute queries?

How do I correctly warmup a neo4j database right after the complete spring app initialization?

Answer 1

Ok I discovered this by pure luck.

I removed the ApplicationListener and created a @Service class with the @Autowired annotation on the method that does the warmup, it seems to be working.

I discovered this by removing a class field without seeing the @Autowired annotation left here, and it worked. I tried it several times, the warmup is working now. I don't know if it is documented somewhere in the Spring Documentation.

This is my final class:

package com.app.components;

@Component
public class Neo4jWarmup {

    private static final Logger LOG = LogManager.getLogger(StartupTasks.class);

    @Autowired
    GraphDatabaseService db;

    /*
     * this did the trick - the method gets called 
     * after Spring initialization and the DB works as expected
     */
    @Autowired 
    public void neo4jWarmup() {
        executeTestSelect();
    }

    private void executeTestSelect() {
        LOG.info("Warming up Neo4j...");

        Transaction tx = db.beginTx();
        db.execute("CALL apoc.warmup.run()");
        tx.close();

        LOG.info("Warmup complete.");
    }
}