Menu
A python program I created is IO bounded. The majority of the time (over 90%) is spent in a single loop which repeats ~10,000 times. In this loop, ~100KB data is generated and written to a temporary file; it is then read back out by another program and statistics about that data collected. This is the only way to pass data into the second program.
Due to this being the main bottleneck, I thought that moving the location of the temporary file from my main HDD to a (~40MB) RAMdisk (inside of over 2GB of free RAM) would greatly increase the IO speed for this file and so reduce the run-time. However, I obtained the following results (each averaged over 20 runs):
It would appear that the RAMdisk is slower that my HDD.
What could be causing this?
Are there any other alternative to using a RAMdisk in order to get faster file IO?
816
The main benefit of a RAM drive is its increased read and write speeds compared to an SSD or hard drive. It will be multiple times faster than even the fastest solid-state drive.
DRAM is much faster than any SSD or other solid state storage device. Generally, you can put it to be at least 5-10 times faster, depending on frequency, generation and the SSD you are comparing it with. SRAM is even faster than DRAM, at least a few times more.
A single stick of RAM operating at 2666 MT/s (megatransfers per second) is the equivalent of 21,328 MB/s (megabytes per second) or 21.328 GB/s. In comparison, the fastest available consumer SSDs, like the Sabrent Rocket, have a max transfer rate of 5,000MB/s.
The main advantage of a RAM Disk is speed. As shown in the graph, the speed of the RAM disk as compared to a Hard disk is typically up to 50 times faster for sequential reads and writes, and up to 200 times faster for small 4KB-size transfers.
Your operating system is almost certainly buffering/caching disk writes already. It's not surprising the RAM disk is so close in performance.
Without knowing exactly what you're writing or how, we can only offer general suggestions. Some ideas:
If you have 2 GB RAM you probably have a decent processor, so you could write this data to a filesystem that has compression. That would trade I/O operations for CPU time, assuming your data is amenable to that.
If you're doing many small writes, combine them to write larger pieces at once. (Can we see the source code?)
Are you removing the 100 KB file after use? If you don't need it, then delete it. Otherwise the OS may be forced to flush it to disk.
176
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
