Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can multiple processes append to a file using fopen without any concurrency problems?

Tags:

c

linux

unix

fopen

I have a process opening a file in append mode. In this case it is a log file. Sample code:

int main(int argc, char **argv) {
    FILE *f;
    f = fopen("log.txt", "a");
    fprintf(f, "log entry line");
    fclose(f);
}

Two questions:

  1. If I have multiple processes appending to the same file, will each log line appear distinctly or can they be interlaced as the processes context switch?
  2. Will this write block if lots of processes require access to the file, therefore causing concurrency problems?

I am considering either doing this in its simplest incarnation or using zeromq to pump log entries over pipes to a log collector.

I did consider syslog but I don't really want any platform dependencies on the software.

The default platform is Linux for this btw.

like image 643
Deleted Avatar asked Sep 26 '11 08:09

Deleted


People also ask

Can multiple processes append to the same file?

Two processes successfully appending to the same file will result in all their bytes in the file in order, but not necessarily contiguously. The caveat is that not all filesystems are POSIX-compatible. Two famous examples are NFS and the Hadoop Distributed File System (HDFS).

Can two processes write to the same file at the same time?

no, generally it is not safe to do this! you need to obtain an exclusive write lock for each process -- that implies that all the other processes will have to wait while one process is writing to the file.. the more I/O intensive processes you have, the longer the wait time.

Does Fopen append create a new file?

Open a binary file in append mode for writing at the end of the file. The fopen() function creates the file if it does not exist.


2 Answers

I don't know about fopen and fprintf but you could open the file using O_APPEND. Then each write will go at the end of the file without a hitch (without getting mixed with another write).

Actually looking in the standard:

The file descriptor associated with the opened stream shall be allocated and opened as if by a call to open() with the following flags:

a or ab          O_WRONLY|O_CREAT|O_APPEND

So I guess it's safe to fprintf from multiple processes as long as the file has been opened with a.

like image 182
cnicutar Avatar answered Sep 30 '22 14:09

cnicutar


The standard (for open/write, not fopen/fwrite) states that

If the O_APPEND flag of the file status flags is set, the file offset shall be set to the end of the file prior to each write and no intervening file modification operation shall occur between changing the file offset and the write operation.

For fprintf() to be used, you have to disable buffering on the file.

like image 21
glglgl Avatar answered Sep 30 '22 15:09

glglgl