Menu
I am trying to convert some Neo4J Java code code to 2.2+, the code runs an embedded Neo4J server with web frontend.
// configure embedded DB,
// but this doesn't start a server at port 12345 ?
final GraphDatabaseService db = new GraphDatabaseFactory()
.newEmbeddedDatabaseBuilder("/path/to/db")
.setConfig(ServerSettings.webserver_address, "localhost")
.setConfig(ServerSettings.webserver_port, 12345)
.newGraphDatabase();
// add shutdown hook
Runtime.getRuntime().addShutdownHook(new Thread(){
@Override public void run() {
graphDb.shutdown();
}
});
The deprecated code that starts a server reads:
final GraphDatabaseAPI api = (GraphDatabaseAPI)db;
final ServerConfigurator c = new ServerConfigurator(api);
c.configuration().addProperty(
Configurator.WEBSERVER_ADDRESS_PROPERTY_KEY, "localhost");
c.configuration().addProperty(
Configurator.WEBSERVER_PORT_PROPERTY_KEY, 12345);
new WrappingNeoServerBootstrapper(api, c).start();
Leaving this out doesn't start the server. It is unclear to me how to get an embedded server running without using the deprecated methods. Any thoughts?
UPDATE:
So indeed the correct answer seems: change your architecture.
One shouldn't use the embedded server anymore, instead use the server and RESTful API. For the bits that you need the Java API you can write an unmanaged extension (see docs).
Here is a simple example of such an extension https://github.com/histograph/neo4j-plugin
Thanks!
675
You can always do that the other way round. Instead of passing instance of GraphDatabaseService you created to the server try start server first and retrieve instance created by it :) Checked with 3.0.6
String homeDir = "./";
String configFile = "./conf/neo4j.conf";
ServerBootstrapper serverBootstrapper = new CommunityBootstrapper();
int i = serverBootstrapper.start(new File(homeDir), Optional.of(new File(configFile)), Pair.of("dbms.connector.http.address","0.0.0.0:7575"));
NeoServer neoServer = serverBootstrapper.getServer();
GraphDatabaseService graph = neoServer.getDatabase().getGraph();
System.out.println(graph);
UPDATE January 2016
In the comments, you'll notice that certain things about neo4j are changing. The rest of this answer I think is still valid, but if your'e a time traveler coming to this post in mid-2016, look into whether or not the new neo4j binary protocol (AFAIK, called "bolt") is available.
/ENDUPDATE
The bad news is that the wrapping neo server boostrapper is deprecated, and I don't think there is a way of doing this without using deprecated methods. So the simple answer to your question is "no".
Maybe a year ago, I was looking for the same thing as you, but let me share something I've learned in the meantime that might change your question.
So when you use an embedded server, you get certain advantages like the ability to use the java API, but you get some big disadvantages. Putting the DB in the same memory space as your application means you have to juggle memory between the two. And it means you can't upgrade your application without stopping the database.
When you use a server, many things are better like the ability to run cypher without going through your app layer, the use of RESTful services, and so on.
The trouble with the WrappingNeoServerBootstrapper (and why it's probably good that it's deprecated) is that it has the disadvantages of both approaches. Sure, you get REST services and cypher, but you're still in the same memory space as the application.
Architecturally, what we've found with neo4j is that you're just usually going to want to use an external separate server, and then communicate with it via REST services. I think neo4j is working on a binary protocol (ala JDBC but of course not the same thing) so that perhaps more of the java API might be opened up and performance would improve. In the meantime, all of the good libraries for neo4j including spring-data and others can already talk to an HTTP endpoint, so whatever features they provide can generally be done with a separate server. For maintainability of the application, and tweakability of neo4j itself, you're almost always going to be better off running a separate server.
Note that this recommendation mimics what you see with just about every other kind of database (mongo, oracle, whatever). Some offer embedded options for small/specialized applications, but just about all of them assume you'll run a separate server process and have your application talk to it. So this isn't about graphs at all, just about good application design, evolvability, and maintainability.
So for these reasons, it's OK that WrappingNeoServerBootstrapper is deprecated (and probably isn't coming back). My experience suggests this isn't something you should really do anyway.
20
With Neo4J 3.1.1 and czajah's approach, "embedded" mode really can be started just fine (no deprecated calls), together with the advantage of manipulating the graph data from the same JVM instance.
Sample code below requires dependency artifacts neo4j and neo4j-server from org.neo4j group. Server config file is omitted, while working storage folder and suitable config options are specified so that neo4j console and neo4j-shell (which is becoming deprecated in favour of cypher-shell) can be used against the "embedded" instance.
import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.helpers.collection.Pair;
import org.neo4j.server.CommunityBootstrapper;
import org.neo4j.server.NeoServer;
import org.neo4j.server.ServerBootstrapper;
import java.io.File;
import java.io.IOException;
import java.util.Optional;
public class RunNeoRun {
public static void main(String[] args) throws IOException {
// Wherever the Neo4J storage location is.
File storeDir = new File("/tmp/tmp4j");
ServerBootstrapper serverBootstrapper = new CommunityBootstrapper();
serverBootstrapper.start(
storeDir,
Optional.empty(), // omit configfile, properties follow
Pair.of("dbms.connector.http.address","127.0.0.1:7474"),
Pair.of("dbms.connector.http.enabled", "true"),
Pair.of("dbms.connector.bolt.enabled", "true"),
// allow the shell connections via port 1337 (default)
Pair.of("dbms.shell.enabled", "true"),
Pair.of("dbms.shell.host", "127.0.0.1"),
Pair.of("dbms.shell.port", "1337")
);
// ^^ serverBootstrapper.start() also registered shutdown hook!
NeoServer neoServer = serverBootstrapper.getServer();
GraphDatabaseService gdb = neoServer.getDatabase().getGraph();
/* Some transactional code */
try(Transaction tx = gdb.beginTx()) {
gdb.getAllNodes().forEach(
n -> System.out.println(n)
);
tx.success();
}
System.out.println("Press ENTER to quit.");
System.in.read();
System.exit(0);
}
}
Czajah, Hi, your workaround for 3.0.6 also works in a production embedded-mode (older paxos HA) 3.1.0 - thank you.
github project (Maven, Tomcat 8 WAR) at https://github.com/obrienlabs/nbi-neo4j-embedded-aws-war
Add a node to the embedded graph db http://neo4j.ca-central-1.elasticbeanstalk.com/FrontController?action=graph
The browser is on port 7575
With a few modifications I was able to wrap a HighlyAvailableGraphDatabase with a EnterpriseBootstrapper.
There are some non-fatal exceptions around JMX reporting in debug.log likely related to my 8.0.28 tomcat version but the graph db running embedded in Tomcat is OK.
2016-12-21 16:20:00.574+0000 INFO Bolt enabled on 0.0.0.0:7688.
2016-12-21 16:20:09.554+0000 INFO Attempting to join cluster of [127.0.0.1:5001]
2016-12-21 16:20:11.566+0000 INFO Creating new cluster with name [neo4j.ha]...
2016-12-21 16:20:11.607+0000 INFO Instance 1 (this server) entered the cluster
2016-12-21 16:20:12.164+0000 INFO I am 1, moving to master
2016-12-21 16:20:12.293+0000 INFO Instance 1 (this server) was elected as coordinator
2016-12-21 16:20:12.462+0000 INFO I am 1, successfully moved to master
2016-12-21 16:20:12.513+0000 INFO Instance 1 (this server) is available as master at ha://127.0.0.1:6001?serverId=1 with StoreId{creationTime=1482199697648, randomId=7800059877674392627, storeVersion=15531981201765894, upgradeTime=1482199697648, upgradeId=1}
2016-12-21 16:20:14.495+0000 INFO Database available for write transactions
2016-12-21 16:20:31.917+0000 INFO Mounted REST API at: /db/manage
2016-12-21 16:20:53.264+0000 INFO Remote interface available at http://localhost:7575/
register: org.obrienlabs.nbi.graph.service.HaMonitor@1c0f80c9
public class ExtendedHighlyAvailableGraphDatabaseFactory extends HighlyAvailableGraphDatabaseFactory {
private Log log = LogFactory.getLog(ExtendedHighlyAvailableGraphDatabaseFactory.class);
private HaMonitor haMonitor;
@Override
public GraphDatabaseService newDatabase( final Map<String, String> config ) {
EnterpriseBootstrapper serverBootstrapper = new EnterpriseBootstrapper();
List<Pair> pairs = new ArrayList<>();
for(Entry<String, String> entry : config.entrySet()) {
pairs.add(Pair.of(entry.getKey(), entry.getValue()));
}
Pair pairArray[] = new Pair[pairs.size()];
// will resolve to /dir/data/databases/graph.db
serverBootstrapper.start(storeDir, Optional.empty(), pairs.toArray(pairArray));
GraphDatabaseService graph = serverBootstrapper.getServer().getDatabase().getGraph();
// set the paxos HA listener only when dbms.mode=HA
if(graph instanceof HighlyAvailableGraphDatabase) {
haMonitor.setDb((HighlyAvailableGraphDatabase) graph);
HighAvailabilityMemberStateMachine memberStateMachine =
((HighlyAvailableGraphDatabase)graph).getDependencyResolver()
.resolveDependency(HighAvailabilityMemberStateMachine.class);
if ( memberStateMachine != null ) {
memberStateMachine.addHighAvailabilityMemberListener(haMonitor);
log.info("register: " + haMonitor);
}
}
return graph;
} };
}
Spring config
<util:map id="config">
<entry key="ha.server_id" value="1"/>
<entry key="ha.initial_hosts" value="127.0.0.1:5001"/>
<entry key="dbms.mode" value="HA"/>
<entry key="browser.allow_outgoing_connection" value="true" />
<entry key="unsupported.dbms.ephemerall" value="false" />
<entry key="dbms.connector.http.address" value="0.0.0.0:7575" />
<entry key="dbms.connector.bolt.address" value="0.0.0.0:7688" />
<entry key="dbms.connector.http.enabled" value="true" />
<entry key="dbms.connector.bolt.enabled" value="true" />
<entry key="dbms.connector.http.type" value="HTTP" />
<entry key="dbms.connector.bolt.type" value="BOLT" />
<entry key="dbms.connector.http.tls_level" value="DISABLED" />
<entry key="dbms.connector.bolt.tls_level" value="DISABLED" />
<entry key="dbms.security.auth_enabled" value="true"/>
<entry key="dbms.logs.debug.level" value="DEBUG"/>
<entry key="dbms.logs.http.enabled" value="true" />
<entry key="dbms.logs.query.enabled" value="true"/>
<entry key="dbms.shell.enabled" value="true"/>
</util:map>
<bean id="haMonitor" class="org.obrienlabs.nbi.graph.service.HaMonitor"/>
<bean id="graphDbFactory" class="org.obrienlabs.nbi.graph.service.ExtendedHighlyAvailableGraphDatabaseFactory">
<constructor-arg ref="haMonitor" />
</bean>
<bean id="graphDbBuilder"
factory-bean="graphDbFactory"
factory-method="newEmbeddedDatabaseBuilder">
<constructor-arg value="/ec2-user"/>
</bean>
<bean id="graphDbBuilderFinal"
factory-bean="graphDbBuilder"
factory-method="setConfig">
<constructor-arg ref="config"/>
</bean>
<!-- HighlyAvailableGraphDatabase wrapped by an EnterpriseBootstrapper NeoServer created in this constructor -->
<bean id="graphDatabaseService"
factory-bean="graphDbBuilderFinal"
factory-method="newGraphDatabase"
destroy-method="shutdown" />
Transactions go through
2016-12-21 20:51:07.478+0000 INFO [o.n.k.i.s.c.CountsTracker] About to rotate counts store at transaction 9 to [/ec2-user/data/databases/graph.db/neostore.counts.db.b], from [/ec2-user/data/databases/graph.db/neostore.counts.db.a].
2016-12-21 20:51:07.480+0000 INFO [o.n.k.i.s.c.CountsTracker] Successfully rotated counts store at transaction 9 to [/ec2-user/data/databases/graph.db/neostore.counts.db.b], from [/ec2-user/data/databases/graph.db/neostore.counts.db.a].
2016-12-21 20:51:07.483+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [9]: Store flush completed
Settings from the mbean in jconsole are
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] Kernel version: 3.1.0,16a782b42d76ca37db72958eb2565cf6aa671a29
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] Neo4j Kernel properties:
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.logs.query.enabled=true
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.security.auth_enabled=true
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.tls_level=DISABLED
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.shell.enabled=true
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] ha.server_id=1
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.logs.http.enabled=true
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] ha.initial_hosts=127.0.0.1:5001
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.allow_format_migration=false
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.address=0.0.0.0:7575
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.enabled=true
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.tls_level=DISABLED
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.enabled=true
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.mode=HA
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.block_size.labels=56
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.type=BOLT
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.directories.neo4j_home=/ec2-user
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.ephemerall=false
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] browser.allow_outgoing_connection=true
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.logs.debug.level=DEBUG
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.http.type=HTTP
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.block_size.strings=120
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.block_size.array_properties=120
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] unsupported.dbms.edition=enterprise
2016-12-23 13:28:12.708+0000 INFO [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.address=0.0.0.0:7688
7575(jetty server) and 7688 (bolt) ports are open
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7575
tcp46 0 0 *.7575 *.* LISTEN 131072 131072 49013 0
tcp4 0 0 127.0.0.1.7575 127.0.0.1.60685 TIME_WAIT 407296 146988 49013 0
tcp6 0 0 ::1.7575 ::1.60699 TIME_WAIT 407284 146808 49013 0
tcp6 0 0 ::1.7575 ::1.60700 TIME_WAIT 407284 146808 49013 0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7688
tcp6 0 0 ::1.7688 ::1.60704 ESTABLISHED 406582 146808 49013 0
tcp6 0 0 ::1.60704 ::1.7688 ESTABLISHED 398196 146808 48165 0
tcp6 0 0 ::1.7688 ::1.60702 ESTABLISHED 406570 146808 49013 0
tcp6 0 0 ::1.60702 ::1.7688 ESTABLISHED 398185 146808 48165 0
tcp6 0 0 ::1.7688 ::1.60701 ESTABLISHED 407255 146808 49013 0
tcp6 0 0 ::1.60701 ::1.7688 ESTABLISHED 407628 146808 48165 0
tcp46 0 0 *.7688 *.* LISTEN 131072 131072 49013 0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 8080
tcp4 0 0 127.0.0.1.8080 127.0.0.1.60584 FIN_WAIT_2 408104 146988 49013 0
tcp4 994 0 127.0.0.1.60584 127.0.0.1.8080 CLOSE_WAIT 408128 146988 42992 0
tcp46 0 0 *.8080 *.* LISTEN 131072 131072 49013 0
udp4 0 0 *.* *.* 196724 9216 38080 0
/ec2-user/logs/http.log
2016-12-23 02:53:21.505+0000 INFO [REQUEST] [AsyncLog @ 2016-12-23 02:53:21.505+0000] 0:0:0:0:0:0:0:1 - [Thu Dec 22 21:53:21 EST 2016] "/browser/views/frame-cypher.html?null" 200 22972 "http://localhost:7575/browser/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/602.2.14 (KHTML, like Gecko) Version/10.0.1 Safari/602.2.14" 2
query.log
2016-12-23 13:33:39.059+0000 INFO 175 ms: bolt-session bolt neo4j neo4j-javascript/[object Object] client/0:0:0:0:0:0:0:1:64019 server/0:0:0:0:0:0:0:1:7688> neo4j - MATCH (a)-[r]->(b) WHERE id(a) IN {node_ids}
AND id(b) IN {new_node_ids}
RETURN r; - {node_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22], new_node_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22]} - {}
Disclaimer: I believe a proper architecture involves a clustered set of server mode bolt enabled Neo4j servers using causal clustering under the raft protocol. This undocumented/internal API is used as a temporary workaround to get the neo4j browser up on an embedded Neo4j server in 3.1.0 as it used to work under the WrappingNeoServerBootstrapper in 2.3. Use of the embedded server for faster traversals was the use case - performance must be reevaluated to see if a bolt based architecture is more optimal for the traversal API. /michael
42
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
