How to pipeline in node.js to redis?

Question

I have lot's of data to insert (SET \ INCR) to redis DB, so I'm looking for pipeline \ mass insertion through node.js.

I couldn't find any good example/ API for doing so in node.js, so any help would be great!

Answer 1

In node_redis there all commands are pipelined:

https://github.com/mranney/node_redis/issues/539#issuecomment-32203325

Answer 2

Yes, I must agree that there is lack of examples for that but I managed to create the stream on which I sent several insert commands in batch.

You should install module for redis stream:

npm install redis-stream

And this is how you use the stream:

var redis = require('redis-stream'),
    client = new redis(6379, '127.0.0.1');

// Open stream
var stream = client.stream();

// Example of setting 10000 records
for(var record = 0; record < 10000; record++) {

    // Command is an array of arguments:
    var command = ['set', 'key' + record, 'value'];  

    // Send command to stream, but parse it before
    stream.redis.write( redis.parse(command) );
}

// Create event when stream is closed
stream.on('close', function () {
    console.log('Completed!');

    // Here you can create stream for reading results or similar
});

// Close the stream after batch insert
stream.end();

Also, you can create as many streams as you want and open/close them as you want at any time.

There are several examples of using redis stream in node.js on redis-stream node module