Java - how to implement threads on a process that waits for data [closed]

Question

I'm fairly new to multi threaded programming and I was hoping to get some insight into the best way to implement the following idea.

Right now my code works like this

It is singularly threaded, so in the time it takes to process each piece of data and write it to the database, new data comes in and gets queued up which is slowing things down too much. I am running on a 4CPU server, but the current setup only uses 1.

I would like separate out the work done in the middle piece over the remaining 3 CPUs. How would I best do this? I thought that I could create a new thread for each new piece of data, but we are talking about over the course of a day hundreds of thousands of new threads. From what I read, the overhead associated with that would be very large. Memory is a concern for me so if the creation of all these threads eats too much memory I will be in trouble. Is it true that a new thread will utilize a less busy CPU, or will it use the same one since it is the same JVM?

The processing and DB write for each new piece of data should not take more than a couple of seconds, if that.

I was also reading about thread pools, but the idea is a bit confusing to me and I can't find a great example.

I was thinking something like this

Please help a multi threaded newb out in coming up with a reasonable design! Thanks in advance :-)

Answer 1

The more important point would be, how many threads are working in parallel (in thus potentially killing a machine). If you are creating one Thread object after another, it could be done more efficiently, but in general the cost for that is (probably) negligible (as Michal pointed out). Leaving that aside (and assuming you want to learn about multi-threading), your design sounds reasonable enough already. Now go have a look what java.util.concurrent can provide you with respect on tools to achieve it:

• ExecutorService: Would be the best choice. Create a fixed thread pool of n worker threads and then, for each incoming thread, post one Runnable doing your processing and storing all the data to the database.

  public class DataProcessor {
      final ExecutorService workerThreadPool = Executors.newFixedThreadPool(5);
  
      public void onNewDataFromTheOutsideWorld(Data d) { 
         workerThreadPool.execute(new ProcessingAndStoreToDBRunnable(d));
      }
  
      public void onShutdown() { 
         workerThreadPool.shutdown();
      }
  }
  

  The ExecutorService will make sure that only the fixed numbers of workers are actually running in parallel,

• Own queue-ing mechanism: When it comes to jobs which have different priorities, you might want to implement your own working mechanism. Note that this is much more complicated, and you should probably stick to the ExecutorService solution if possible.

  The basic idea is to have a BlockingQueue to which data is added and start n worker threads you read from the queue the jobs. The trick is, the queue will block if there are no jobs (thus sending the threads to sleep), if there are more than n jobs, the jobs are stored in the queue until a processing thread is available.

  public class DataProcessor {
      final BlockingQueue<Data> queue = new BlockingQueue<Data>();
  
      public void onInit() {
         for (int i = 0; i < n; i++) 
             new Thread(new WorkerRunnable(queue)).start();
      }
  
      public void onNewDataFromTheOutsideWorld(Data d) { 
         queue.add(d);
      }
  }
  
  public class WorkerRunnable implements Runnable {
      public void run() { 
         while (true) {
             Data d = queue.take();
             processData(d);
         }
      }
   }
  

  As I said, this is much more complicated to realize as I still haven't touched issues like

  • Stopping the worker threads
  • Making sure to handle exceptions and then return on to processing
  • etc.

These are just basic (yet very powerful) tools in a multi-threading environment. If you need more advanced tools, checkout the Guava library, for example with their wonderful concept of a ListenableFuture (which you should use if you need the result of a worker thread.)

You then would have a rather basic design, from where you can add some more sophisticated processing steps as already pointed out in your comments. Also it was pointed out, that it then turn in a rather broad question ;)