Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get read/write disk speed in Python?

In a Python program I need to get the accumulated read/write speeds of all disks on the host. I was doing it with subprocess.check_output() to call the following Linux shell command:

$ sudo hdparm -t /dev/sda

This gives as a result:

/dev/sda:
 Timing buffered disk reads: 1488 MB in  3.00 seconds = 495.55 MB/sec

then I can parse the 495.55. OK, so far so good.

But on the man page of hdparm i found this explanation for the -t flag that basically says that when performing measurements no other process should read/write to disk at same time:

Perform timings of device reads for benchmark and comparison purposes. For meaningful results, this operation should be repeated 2-3 times on an otherwise inactive system (no other active processes) with at least a couple of megabytes of free memory. This displays the speed of reading through the buffer cache to the disk without any prior caching of data. This measurement is an indication of how fast the drive can sustain sequential data reads under Linux, without any filesystem overhead. To ensure accurate measurements, the buffer cache is flushed during the processing of -t using the BLKFLSBUF ioctl.

The question is:

How can I ensure that no other process is accessing disk at the same time when measurements are performed?

like image 830
RichArt Avatar asked Oct 18 '16 12:10

RichArt


People also ask

How do I see disk usage in Python?

disk_usage() method in Python is to get disk usage statistics about the given path. This method returns a named tuple with attributes total, used and free.


1 Answers

According to https://unix.stackexchange.com/questions/55212/how-can-i-monitor-disk-io the most usable solution includes the tool sysstat or iostat (same package).

But seriously, since you have sudo permissions on the host, you can check yourself whether any IO intensive tasks are going on using any of the popular system monitoring tools. You cannot kill all IO effectively without your measurements also going nuts. Over a longer time the measurements should give you reasonable results nonetheless, since the deviations converge towards stable background noise.

Aside from that what would you need artificial measurements for? If you simply want to test the hardware capabilities without any RL context, do not mount the disk and test it in binary mode. A measurement while real traffic is going on usually gives you results that are closer to what you can actually expect at load times.

like image 133
Hubert Grzeskowiak Avatar answered Oct 22 '22 05:10

Hubert Grzeskowiak