Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Default input and output buffering for fopen'd files?

So a FILE stream can have both input and output buffers. You can adjust the output stream using setvbuf (I am unaware of any method to play with the input buffer size and behavior).

Also, by default the buffer is BUFSIZ (not sure if this is a POSIX or C thing). It is very clear what this means for stdin/stdout/stderr, but what are the defaults for newly opened files? Are they buffered for both input and output? Or perhaps just one?

If it is buffered, does output default to block or line mode?

EDIT: I've done some tests to see how Jonathan Leffler's answer affected real world programs. It seems that if you do a read then a write. The write will cause the unused portion of the input buffer to dropped entirely. In fact, the there will be some seeks that are done to keep things at the right file offsets. I used this simple test program:

/* input file contains "ABCDEFGHIJKLMNOPQRSTUVWXYZ" */
#include <stdio.h>
#include <stdlib.h>

int main() {

    FILE *f = fopen("test.txt", "r+b");
    char ch;
    fread(&ch, 1, 1, f);
    fwrite("test", 4, 1, f);
    fclose(f);
    return 0;
}

resulted in the following system calls:

read(3, "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n", 4096) = 27 // attempt to read 4096 chars, got 27
lseek(3, -26, SEEK_CUR)                 = 1        // at this point, I've done my write already, so forget the 26 chars I never asked for and seek to where I should be if we really just read one character...
write(3, "test", 4)                     = 4        // and write my test
close(3)                                = 0

While these are clearly implementation details I found them to be very interesting as far as how the standard library could be implemented. Thanks Jonathan for your insightful answer.

like image 799
Evan Teran Avatar asked Apr 19 '10 18:04

Evan Teran


People also ask

Is PUTC buffered?

"putc" outputs a character to the given file and returns the character that was written. By default, all output streams (including "stdout" and "stderr") are normally buffered.

Can fopen create file?

The fopen function creates the file if it does not exist. r+b or rb+ Open a binary file for both reading and writing. The file must exist.

What is output buffering in Python?

Summary: Python output buffering is the process of storing the output of your code in buffer memory. Once the buffer is full, the output gets displayed on the standard output screen.


1 Answers

A single file stream has a single buffer. If the file is used for both input and output, then you have to ensure that you do appropriate operations (fseek() or equivalents) between the read and write (or write and read) operations.

The buffering behaviour of the standard channels is platform dependent.

Typically, stdout is line buffered when the output goes to the terminal. However, if stdout is going to a file or pipe rather than to a terminal, it most usually switches to full buffering.

Typically, stderr is either line buffered or unbuffered, to ensure that error messages get seen (for example, even if the program is about to crash).

Typically, stdin is line buffered; this means you get a chance to edit your input (backspacing over errors, etc). You would seldom adjust this. Again, if the input is coming from a file (or pipe), the behaviour might be different.

Newly opened files will generally be fully buffered. A particular implementation might change that to line buffering if the device is a terminal.

Your premise - that there are two buffers - is incorrect.


Section 7.19.3 of C99, it says:

At program startup, three text streams are predefined and need not be opened explicitly — standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output). As initially opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.

So, as originally stated, stderr is either unbuffered or line buffered (it is not fully buffered).

like image 93
Jonathan Leffler Avatar answered Sep 26 '22 09:09

Jonathan Leffler