Why cpu bound is better with blocking I/O and I/O bound is better with non blocking I/O

Question

I have been told that for I/O bound applications, non blocking I/O would be better. For CPU bound applications, blocking I/O is much better. I could not find the reason for such a statement. Tried google, but few articles just touches the topic with not much details. Can someone provide the deep depth reason for it?

With this, I want to clear myself with what are the short coming of non blocking I/O as well.

After going through another thread here,a reason I could relate was out was if the I/O process is heavy enough then only we can see significant performance improvements using non blocking I/O. It also states that if the number of I/O operations is large(a typical web application scenario) where there are many requests looking out for I/O requests, then also we see significant improvements using non blocking I/O.

Thus my questions boil down to the following list:

1. In case of a CPU intensive applications, is it better to start a threadpool(or executionContext of scala) and divide the work between the threads of the threadpool.(I guess it has definitely an advantage over spawning your own threads and dividing the work manually. Also using asyn concepts of future, even CPU intensive work can be returned using callbacks hence avoiding the issues related to blocking of multi threading?). Also if there is a I/O which is fast enough, then do the I/O using blocking principles on the threads of thread pool itself. Am I right?

2. What are actually short comings or overheads of using a non blocking I/O technically? Why we don't see much performance gains of using non blocking I/O if the I/O is fast enough or if there are very less I/O operations required? Eventually it is the OS which is handling I/O's. Irrespective of whether the number of I/O's are large or small, let OS handle that pain. What makes the difference here.

Answer 1

From a programmer's perspective blocking I/O is easier to use than nonblocking I/O. You just call the read/write function and when it returns you are done. With nonblocking I/O you need to check if you can read/write, then read/write and then check the return values. If not everything was read or written you need mechanisms to read again or to write again now or later when write can be done.

Regarding performance: nonblocking I/O in one thread is not faster than blocking I/O in one thread. The speed of the I/O operation is determined by the device (for example the hard disc) that is read from or written to. The speed is not determined by someone waiting for (blocking on) or not waiting for (nonblocking on) it. Also if you call a blocking I/O function then the OS can do the blocking quite effectively. If you need to do the blocking/waiting in the application you might do that nearly as good as the OS, but you might also do it worse.

So why do programmers make their life harder and implement nonblocking I/O? Because, and that is the key point, their program has more to do than only that single I/O operation. When using blocking I/O you need to wait until the blocking I/O is done. When using nonblocking I/O you can do some calculations until the blocking I/O is done. Of course during nonblocking I/O you can also trigger other I/O (blocking or nonblocking).

Another approach to nonblocking I/O is to throw in more threads with blocking I/O, but as said in the SO post that you linked threads come with a cost. That cost is higher is than the cost for (OS supported) nonblocking I/O.

If you have an application with massive I/O but only low CPU usage like a web server with lots of clients in parallel, then use a few threads with nonblocking I/O. With blocking I/O you'll end up with a lot of threads -> high costs, so use only a few threads -> requires nonblocking I/O.

If you have an application that is CPU intensive like a program that reads a file, does intensive calculations on the complete data and writes the result to file, then 99% of the time will be spent in the CPU intensive part. So create a few threads (for example one per processor) and do as much calculation in parallel. Regarding the I/O you'll probably stick to one main thread with blocking I/O because it is easier to implement and because the main thread itself has nothing to do in parallel (given that the calculations are done in the other threads).

If you have an application that is CPU intensive and I/O intensive then you'ld also use a few threads and nonblocking I/O. You could think of a web server with lots of clients and web page requests where you are doing intensive calculations in a cgi script. While waiting for I/O on on connection the program could calculate the result for another connection. Or think of a program that reads a large file and could do intensive calculations on chunks of the file (like calculating an average value or adding 1 to all values). In that case you could use nonblocking reads and while waiting for the next read to finish you could already calculate on the data that is available. If the result file is only a small condensed value (like an average) you might use blocking write for the result. If the result file is as large as the input file and is like "all values +1", then you could write back the results nonblocking and while the write is being done you are free to do calculations on the next block.