Redis Queue Incorrect Ordering

Question

I am using redis to store an ordered set of items. Here is a sample code:

object Producer{
  def main(args:Array[String]){
    val jedis = new Jedis("localhost")
    for (i<-1 to 10){
      println("publishing:"+(i))
      jedis.lpush("q1",i.toString)
    }
  }
}
object Consumer {
  def main(args:Array[String]){
    val jedis = new Jedis("localhost")
    while(true){
      while(jedis.llen("q1")>0){
        val msg=jedis.lpop("q1")
        println("processing:"+msg)
      }
    }

  }
}

When I run the producer, I get

publishing:1
publishing:2
publishing:3
publishing:4
publishing:5
publishing:6
publishing:7
publishing:8
publishing:9
publishing:10

while when I run consumer, I get

processing:1
processing:2
processing:4
processing:5
processing:6
processing:7
processing:9
processing:10
processing:8
processing:3

Why order of items are not correct, this is not really FIFO.

Answer 1

This is not really FIFO because the code is wrong.

If you use lpush to queue item, you should use rpop rather than lpop to dequeue them in FIFO order.

redis 127.0.0.1:6379> lpush x 10
(integer) 1
redis 127.0.0.1:6379> lpush x 20
(integer) 2
redis 127.0.0.1:6379> lpush x 30
(integer) 3
redis 127.0.0.1:6379> lpop x
"30"
redis 127.0.0.1:6379> lpop x
"20"
redis 127.0.0.1:6379> lpop x
"10"

Normally you should retrieve the items in reverse order (LIFO). But your two agents probably run concurrently, so the order you get with lpop is not deterministic.

Please note your dequeuing agent is poorly implemented since it takes 100% of the CPU (there is no wait state) and will saturate Redis with llen commands when the queue is empty. Consider using a blocking call such as brpop instead.