Is a "Chain of Threads" a bad solution for this Java application?

Question

I'm running a program where I download large files, parse them and then write the data I have extracted from the file into another file.

The files take a long time to download and parse but the write task only takes a minute or so on average. My solution I threw together was to have three fixedthreadpools of three threads.

ExecutorService downloadExecutor = Executors.newFixedThreadPool(3);
ExecutorService parseExecutor = Executors.newFixedThreadPool(3);
ExecutorService writeExecutor = Executors.newFixedThreadPool(3);

A thread in the download pool downloads the file, then submits a new thread to the parser threadpool, with the filename as a parameter. This is done within the thread itself. The download thread then gets to work downloading another file from a list of file URLs.

Once the parser thread has finished parsing the data I want from the file,it then submits a new thread containing the data to the write threadpool, where it is then written to a .csv file.

My question is if there is a more elegant solution to this. I have not really done much complex threading. Since I have a lot of files to download and parse, I do not want any of the threads being idle at any time. The idea again, is that since parsing a file can take a while, I might as well make seperate threads devoted to downloading those files first.

Answer 1

Why not use only one Thread pool. Download, parse and save must wait anyway for each other so the best seperation of tasks would be to use one thread per file.

Answer 2

This is not a bad practice as many developers do similar sort of coding. But there are something you need to keep in mind.

Number One, You can't expect the performance to increase just because you have more threads. There are optimum number of threads based on the no of CPUs.

Number Two, You must make sure how exceptions are handled.

Number Three, You must make sure you can shutdown all the thread pools in an event where you need to stop the application.