Massive tasks alternative pattern for Runnable or Callable

Question

For massive parallel computing I tend to use executors and callables. When I have thousand of objects to be computed I feel not so good to instantiate thousand of Runnables for each object.

So I have two approaches to solve this:

I. Split the workload into a small amount of x-workers giving y-objects each. (splitting the object list into x-partitions with y/x-size each)

public static <V> List<List<V>> partitions(List<V> list, int chunks) {
      final ArrayList<List<V>> lists = new ArrayList<List<V>>();
      final int size = Math.max(1, list.size() / chunks + 1);
      final int listSize = list.size();
      for (int i = 0; i <= chunks; i++) {
         final List<V> vs = list.subList(Math.min(listSize, i * size), Math.min(listSize, i * size + size));
         if(vs.size() == 0) break;
         lists.add(vs);
      }
      return lists;
   }

II. Creating x-workers which fetch objects from a queue.

Questions:

• Is creating thousand of Runnables really expensive and to be avoided?
• Is there a generic pattern/recommendation how to do it by solution II?
• Are you aware of a different approach?

Answer 1

Creating thousands of Runnable (objects implementing Runnable) is not more expensive than creating a normal object.

Creating and running thousands of Threads can be very heavy, but you can use Executors with a pool of threads to solve this problem.

Answer 2

As for the different approach, you might be interested in java 8's parallel streams.