Is there a vmstat type command that works per-process that allows you to see how much time a process is blocked waiting for I/O, time in kernel and user code?
Using the top command The top command is the easiest and most widely used command for identifying I/O wait time in Linux. It displays a dynamic and real-time view of the system, CPU-level statistics, and a running list of processes managed by the kernel.
strace -p $process_id - if there is not continuous output, that means it is waiting for something. If it waits too long, you could consider that "hung". you could also use the timeout command to set a limit on a command.
To check which processes are actually utilizing the disk IO, run the iotop command with -o or –only option to visualize it. Details: IO: It shows Input/Output utilization of each process, which includes disk and swap. SWAPIN: It shows only the swap usage of each process.
iotop. The iotop command is top-like utility for looking at disk I/O. It gathers I/O usage information provided by the Linux kernel so that you can get an idea which processes are most demanding in terms in disk I/O.
strace will show you how much time is spent in system call, however it won't tell you how much of this time is spent in waiting versus time really spent for I/O. you can select which system call you want to trace, or which type, it is quite powerful
latencytop would be another good tool, because your process can be waiting for I/O because of other processes, or because of some journalling daemon.
blktrace is what your looking for, block layer information, wait/blocking/busy etc..., very in depth, there are quite a few of packages that derive from it, seekwatcher, ...
Some other tool, simular to what sigjuice said, iotop, is also handy, but less informative for serious analysis. Also I believe btrace/blktrace is much more well suited for I/O tracing than oprofile, which is more general and increases load in comparison.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With