Shared Memory between two JVMs

Question

Is there a way in Java, for two JVMs (running on same physical machine), to use/share the same memory address space? Suppose a producer in JVM-1 puts messages at a particular pre-defined memory location, can the consumer on JVM-2 retrieve the message if it knows which memory location to look at?

Answer 1

Solution 1:

The best solution in my opinion is to use memory mapped files. This allows you to share a region of memory between any number of process, including other non java programs. You can't place java objects into a memory mapped file, unless you serialize them. The following example shows that you can communicate between two different process, but you would need to make it much more sophisticated to allow better communication between the processes. I suggest you look at Java's NIO package, specifically the classes and methods used in the below examples.

Server:

public class Server {      public static void main( String[] args ) throws Throwable {         File f = new File( FILE_NAME );          FileChannel channel = FileChannel.open( f.toPath(), StandardOpenOption.READ, StandardOpenOption.WRITE, StandardOpenOption.CREATE );          MappedByteBuffer b = channel.map( MapMode.READ_WRITE, 0, 4096 );         CharBuffer charBuf = b.asCharBuffer();          char[] string = "Hello client\0".toCharArray();         charBuf.put( string );          System.out.println( "Waiting for client." );         while( charBuf.get( 0 ) != '\0' );         System.out.println( "Finished waiting." );     } } 

Client:

public class Client {      public static void main( String[] args ) throws Throwable {         File f = new File( FILE_NAME );         FileChannel channel = FileChannel.open( f.toPath(), StandardOpenOption.READ, StandardOpenOption.WRITE, StandardOpenOption.CREATE );          MappedByteBuffer b = channel.map( MapMode.READ_WRITE, 0, 4096 );         CharBuffer charBuf = b.asCharBuffer();          // Prints 'Hello server'         char c;         while( ( c = charBuf.get() ) != 0 ) {             System.out.print( c );         }         System.out.println();          charBuf.put( 0, '\0' );     }  } 

---

Solution 2:

Another solution is to use Java Sockets to communicate back and forth between processes. This has the added benefit of allowing communication over a network very easily. It could be argued that this is slower than using memory mapped files, but I do not have any benchmarks to back that statement up. I won't post code to implementing this solution, as it can become very complicated to implement a reliable network protocol and is fairly application specific. There are many good networking sites that can be found with quick searches.

---

Now the above examples are if you want to share memory between two different process. If you just want to read/write to arbitrary memory in the current process, there are some warnings you should know first. This goes against the entire principle of the JVM and you really really should not do this in production code. You violate all safety and can very easily crash the JVM if you are not very careful.

That being said, it is quite fun to experiment with. To read/write to arbitrary memory in the current process you can use the sun.misc.Unsafe class. This is provided on all JVMs that I am aware of and have used. An example on how to use the class can be found here.

Answer 2

There are some IPC libraries which facilitate use of shared memory via memory-mapped files in Java.

Chronicle-Queue

Chronicle Queue is similar to a non-blocking Java Queue, except you could offer a message in one JVM and poll it in another JVM.

In both JVMs you should create a ChronicleQueue instance in the same FS directory (locate this directory in a memory-mounted FS if you don't need message persistence):

ChronicleQueue ipc = ChronicleQueueBuilder.single("/dev/shm/queue-ipc").build(); 

Write a message in one JVM:

ExcerptAppender appender = ipc.acquireAppender(); appender.writeDocument(w -> {     w.getValueOut().object(message); }); 

Read a message in another JVM:

ExcerptTailer tailer = ipc.createTailer(); // If there is no message, the lambda, passed to the readDocument() // method is not called. tailer.readDocument(w -> {     Message message = w.getValueIn().object(Message.class);     // process the message here });  // or avoid using lambdas try (DocumentContext dc = tailer.readingDocument()) {     if (dc.isPresent()) {         Message message = dc.wire().getValueIn().object(Message.class);         // process the message here     } else {         // no message     } } 

Aeron IPC

Aeron is more than just IPC queue (it is a network communication framework), but it provides an IPC functionality as well. It is similar to Chronicle Queue, one important difference is that it uses SBE library for message marshalling/demarshalling, while Chronicle Queue uses Chronicle Wire.

Chronicle Map

Chronicle Map allows IPC communication by some key. In both JVMs, you should create a map with identical configurations and persisted to the same file (the file should be localed in memory-mounted FS if you don't need actual disk persistence, e. g. in /dev/shm/):

Map<Key, Message> ipc = ChronicleMap     .of(Key.class, Message.class)     .averageKey(...).averageValue(...).entries(...)     .createPersistedTo(new File("/dev/shm/jvm-ipc.dat")); 

Then in one JVM you could write:

ipc.put(key, message); // publish a message 

On the reciever JVM:

Message message = ipc.remove(key); if (message != null) {     // process the message here }