Can multiple threads/processes read/write from/to non-overlapping regions of a file simultaneously without synchronization?

Question

I'd like to know if Linux read/write system calls support unsynchronized reads/writes (non-appending writes) to non-overlapping regions of a single disk file from multiple threads or processes. Each thread will seek to its own region of the file, and access this region exclusively, never overlapping the regions that other threads are operating on.

Answer 1

I'd like to know if linux read/write system calls supports unsynchronized reads/writes (non-appending writes) to non-overlapping regions of a single disk file from multiple threads or processes. Each thread will seek to its own region of the file, and read/write from/to this region exclusively, never overlapping the regions that other threads are operating on.

POSIX specifies in XSH 2.9.7 that substantially all I/O functions are atomic with respect to each other with regard to their POSIX-specified effects. A long list of the specific functions to which that applies is given, and open(), lseek(), read(), write(), and close() are all on it. Therefore,

If two threads each call one of these functions, each call shall either see all of the specified effects of the other call, or none of them.

That does not depend on any external synchronization, even for operations on file descriptors associated with the same open file description.

There can be multiple open file descriptions for the same file, even in a single process (see, for example, the manual page for open(2)). Given multiple threads performing read() and write() operations on non-overlapping regions of the same regular file, via file descriptors referring to different open file descriptions, POSIX provides no basis to expect those operations to interfere with each other, regardless of external synchronization of the threads involved. In practice, it works fine.

Where you can get in trouble is if the threads involved are trying to use file descriptors referring to the same open file description. These do not have to be the same file descriptor value (so dup()ing a file descriptor does not help here), nor do the threads have to belong to the same process for the situation to arise. Each open file description has a single associated file position, so if two different threads are trying to perform tasks that each require separately setting the file offset and transferring data to or from the file, and if they are using the same open file description, then atomicity of the individual function calls is not sufficient to ensure that the reads and writes are performed at the intended positions. Synchronization is required in that scenario.

Alternatively, as @maximegorushkin observed in comments and @bk2204 observed in another answer, the pread() and pwrite() functions perform positioning and data transfer in a single call. These are also on the list of atomic I/O functions, and they overcome the separation of positioning from data transfer on a per-data-transfer basis. Using them requires extra care and bookkeeping, and there are scenarios that it does not not adequately serve, but it may nevertheless be viable for the particular case in question.

Therefore, if two different threads want to operate on the same file without synchronization then the safest and most general approach is for each to open the file independently. They will not then interfere with each other as long as their I/O operations are restricted to disjoint regions of the file. Even operating on overlapping regions of the file is not out of the question, but that introduces more complex, application-specific considerations.

Answer 2

Yes, this is possible. The pread and pwrite functions allow you to read and write from a file at a given offset without changing the file offset. They're specifically designed to allow reading and writing in multithreaded programs.

You should note that in violation of POSIX, pwrite on a file descriptor opened with O_APPEND will always write to the end of the file.