Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Non-threadsafe file I/O in C/C++

While troubleshooting some performance problems in our apps, I found out that C's stdio.h functions (and, at least for our vendor, C++'s fstream classes) are threadsafe. As a result, every time I do something as simple as fgetc, the RTL has to acquire a lock, read a byte, and release the lock.

This is not good for performance.

What's the best way to get non-threadsafe file I/O in C and C++, so that I can manage locking myself and get better performance?

  • MSVC provides _fputc_nolock, and GCC provides unlocked_stdio and flockfile, but I can't find any similar functions in my compiler (CodeGear C++Builder).
  • I could use the raw Windows API, but that's not portable and I assume would be slower than an unlocked fgetc for character-at-a-time I/O.
  • I could switch to something like the Apache Portable Runtime, but that could potentially be a lot of work.

How do others approach this?

Edit: Since a few people wondered, I had tested this before posting. fgetc doesn't do system calls if it can satisfy reads from its buffer, but it does still do locking, so locking ends up taking an enormous percentage of time (hundreds of locks to acquire and release for a single block of data read from disk). Not doing character-at-a-time I/O would be a solution, but C++Builder's fstream classes unfortunately use fgetc (so if I want to use iostream classes, I'm stuck with it), and I have a lot of legacy code that uses fgetc and friends to read fields out of record-style files (which would be reasonable if it weren't for locking issues).

like image 439
Josh Kelley Avatar asked Jul 29 '09 13:07

Josh Kelley


2 Answers

I'd simply not do IO a char at a time if it is sensible performance wise.

like image 157
AProgrammer Avatar answered Oct 09 '22 13:10

AProgrammer


fgetc is almost certainly not reading a byte each time you call it (where by 'reading' I mean invoking a system call to perform I/O). Look somewhere else for your performance bottleneck, as this is probably not the problem, and using unsafe functions is certainly not the solution. Any lock handling you do will probably be less efficient than the handling done by the standard routines.

like image 43
William Pursell Avatar answered Oct 09 '22 13:10

William Pursell