How to iterate on concurrentLinkedQueue by multiple threads?

Question

In my application data producing speed (which is stored in in concurrentLinkedQueue) is greater than i can consume with single thread.

I have decided to start with creating 4 threads to consume the data, to prevent my application from "out of memory exception".

Questions :

• Any other better design for the above problem with an example ?
• Can we iterate over concurrentLinkedQueue with multiple threads and delete the elements while iterating ?

Memory consistency effects: As with other concurrent collections, actions in a thread prior to placing an object into a ConcurrentLinkedQueue happen-before actions subsequent to the access or removal of that element from the ConcurrentLinkedQueue in another thread.

Answer 1

I think you should not iterate but create 4 thread each polling data from the queue so that polled data will be deleted or in other words consumed

// your queue
ConcurrentLinkedQueue concurrentLinkedQueue = new ConcurrentLinkedQueue();

    // create 4 Threads
    for (int i = 0; i < 4; i++) {
        new Thread(() -> {
            while (!concurrentLinkedQueue.isEmpty()) {
                // consume element
                var element = concurrentLinkedQueue.poll();

                // do something with element
                // here
            }
        }).start();
    }

Answer 2

You should use the offer and poll methods on a ConcurrentLinkedQueue, rather than directly using an iterator. The iterator is weakly consistent.

while(true) {
  final Item item = queue.poll();
  if (item == null) {
    break;
  }
  // do something with item
}

It is safe for many threads to call offer and/or poll concurrently.