redis bulk import using --pipe

Question

I'm trying to import one million lines of redis commands, using the --pipe feature.

redis_version:2.8.1

cat file.txt | redis-cli --pipe

This results in the following error:

Error reading from the server: Connection reset by peer

Does anyone know what I'm doing wrong?

file.txt contains, for example,

lpush name joe
lpush name bob

edit: I now see there's probably a special format(?) for using pipe mode - http://redis.io/topics/protocol

Answer 1

To use the pipe mode (a.k.a mass insertion) you must indeed provide your commands directly in Redis protocol format.

The corresponding Redis protocol for LPUSH name joe is:

*3
$5
LPUSH
$4
name
$3
joe

Or as a quoted string: "*3\r\n$5\r\nLPUSH\r\n$4\r\nname\r\n$3\r\njoe\r\n".

This is what your input file must contain.

The Redis documentation includes a Ruby sample to help you generate the protocol: see gen_redis_proto.