Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

optimal buffer size for reading file in C [closed]

Tags:

c

io

I have to read large files in C using read function. I was just wondering if it makes any difference what buffer size we keep in terms of performance. The file sizes may reach till tens of GB.

like image 331
Aman Deep Gautam Avatar asked Nov 17 '12 18:11

Aman Deep Gautam


1 Answers

Short version.
It depends. On x86 buffer size of 4096 bytes is a good start (one page size and also Advanced Format block size).

Longer version.
In UNIX it depends on kernel, libc, filesystem, hardware, etc. Not only on versions and compilation options but also on run-time tunables(e.g read ahead setup).

DIY.
Test it! See Advanced Programing in UNIX Environment Chapter 3.9 "I/O Efficiency" for straightforward way of determining the best read-write buffer size for one particular system.

like image 171
SaveTheRbtz Avatar answered Oct 24 '22 05:10

SaveTheRbtz