Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can I allow multiple programs to read from the same file at the same time?

I have an application that reads a set of data files and performs some model computations. The program does not need to modify the data files themselves, so I'm currently opening them with the read-only flag as shown below:

FILE* file;
if(_wfopen_s(&file, fname.c_str(), L"r") == 0)
...

I would like to have several instances of my program running at the same time, using the same set of data, but performing different computations on the data. None of my programs need to modify the data files. As the data files are very large, I can't make separate copies of the data to use with each program.

I assumed that because I'm opening the files with read-only permissions, two programs could be reading from the same file at the same time. Instead, I get various errors along the lines of "the file could not be open because it is being used by another process".

As my development environment is Windows 7, this question suggests it might be a matter of enabling read sharing. However, all of the answers in that thread rely on CreateFile, whereas I'm dealing with legacy code that was written with stdio.h.

Is there a way I can have several programs concurrently read from a file using the fopen class of functions?

like image 266
Troyen Avatar asked Jul 19 '12 18:07

Troyen


People also ask

Can multiple programs read the same file at the same time?

It's okay to ACCESS the file from two different programs, as long as you don't try to MODIFY it from two different programs.

Can two processes open the same file?

The process of opening (and if necessary creating) the file is atomic, so it will never happen that one process will create the file but the other process will open it. Note: this is specific to Windows, other operating systems have different behaviour.

Can two programs write to the same file?

no, generally it is not safe to do this! you need to obtain an exclusive write lock for each process -- that implies that all the other processes will have to wait while one process is writing to the file.. the more I/O intensive processes you have, the longer the wait time.

Can a file be opened multiple times?

Risk Assessment. Simultaneously opening a file multiple times can result in unexpected errors and nonportable behavior.


1 Answers

If you can change the fopen routine then try to replace fopen calls with _fsopen, for shared reading/writing. _fsopen is mscrt-specific.

If you can use CreateFile, and don't want to re-write all the legacy code for read/write, you can also try associating a FILE * with a winapi file handle. Use _open_osfhandle to get a file descriptor from a file handle returned by CreateFile, then use _fdopen to get a FILE * from that file descriptor.

like image 163
pb2q Avatar answered Sep 30 '22 18:09

pb2q