NodeJs shared objects between processes

Question

I am creating a caching module for Nodejs that needs to utilize multiple CPUs using sub processes.

I would like to store data in an index in the master process or, preferably in a subprocess like so:

var index = { key1: 21, key2: 22, key3: 55 }

and another process should be able to search in that index as efficiently as:

if('key2' in index) // do stuff

I assume using IPC would be significantly slower than achieving shared objects. Is this even possible? Thanks.

Answer 1

You might want to try mmap-object. It coordinates shared memory based on a memory-mapped file which means the data is persistent as well as shared among processes. I wrote it so to avoid both the performance and management overhead of redis or a similar solution.

For your application the read-write version may be what you want.

Answer 2

I would use redis in your case. There are ways to configure redis on how frequently it will store its memory data on disk based on how many entries are saved or just time (http://redis.io/topics/persistence).

Another option might be to send requests between instances to save that memory data only on master.

When a non master instance wants to save or load data it will request the master.

Here is some pseudocode:

var SharedMemory = {
  storeObject: function(args, done) {
    if (IAmMaster()) {
      storeObjectToMemory(args);
      done();
    } else {
      sendRequestToMasterForSave(args, done);
    }
  },
  getObject: function(args, done) {
    if (IAmMaster()) {
      done(getObjectFromMemory(args));
    } else {
      getResponseFromMasterForLoad(args, done);
    }
  }
}

However, this is probably gonna be a painful process