What do the counters in /proc/[pid]/io mean?

Question

I'm creating a plugin for Munin to monitor stats of named processes. One of the sources of information would be /proc/[pid]/io. But I have a hard time finding out what the difference is between rchar/wchar and read_bytes/written_bytes.

They are not the same, as they provide different values. What do they represent?

Answer 1

While the proc manpage is woefully behind (and so are most manpages/documentation on anything not relating to cookie-cutter user-space development), this stuff is fortunately documented completely in the Linux kernel source under Documentation/filesystems/proc.rst. Here are the relevant bits:

rchar -----  I/O counter: chars read The number of bytes which this task has caused to be read from storage. This is simply the sum of bytes which this process passed to read() and pread(). It includes things like tty IO and it is unaffected by whether or not actual physical disk IO was required (the read might have been satisfied from pagecache)   wchar -----  I/O counter: chars written The number of bytes which this task has caused, or shall cause to be written to disk. Similar caveats apply here as with rchar.   read_bytes ----------  I/O counter: bytes read Attempt to count the number of bytes which this process really did cause to be fetched from the storage layer. Done at the submit_bio() level, so it is accurate for block-backed filesystems. <please add status regarding NFS and CIFS at a later time>   write_bytes -----------  I/O counter: bytes written Attempt to count the number of bytes which this process caused to be sent to the storage layer. This is done at page-dirtying time.