I have to read large files in C using read
function. I was just wondering if it makes any difference what buffer size we keep in terms of performance. The file sizes may reach till tens of GB.
Short version.
It depends. On x86 buffer size of 4096 bytes is a good start (one page size and also Advanced Format block size).
Longer version.
In UNIX it depends on kernel, libc, filesystem, hardware, etc. Not only on versions and compilation options but also on run-time tunables(e.g read ahead setup).
DIY.
Test it! See Advanced Programing in UNIX Environment Chapter 3.9 "I/O Efficiency" for straightforward way of determining the best read-write buffer size for one particular system.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With