Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Multiple Threads reading from the same file

I have a xml file that needs to be read from many many times. I am trying to use the Parallel.ForEach to speed this processes up since none of that data being read in is relevant as to what order it is being read in. The data is just being used to populate objects. My problem is even though I am opening the file each time in the thread as read only it complains that it is open by another program. (I don't have it opened in a text editor or anything :))

How can I accomplish multi reads from the same file?

EDIT: The file is ~18KB pretty small. It is read from about 1,800 times.

Thanks

like image 954
Pieces Avatar asked Aug 19 '10 23:08

Pieces


People also ask

Can multiple threads read from same file?

Multiple threads can also read data from the same FITS file simultaneously, as long as the file was opened independently by each thread. This relies on the operating system to correctly deal with reading the same file by multiple processes.

Can a file be read by multiple threads in Java?

Unlike many other computer languages, Java provides built-in support for multithreaded programming. A multithreaded program contains two or more parts that can run concurrently. Each part of such a program is called thread and each thread defines a separate path of execution.

What will happen if multiple threads accessing the same resource?

Multiple threads accessing shared data simultaneously may lead to a timing dependent error known as data race condition. Data races may be hidden in the code without interfering or harming the program execution until the moment when threads are scheduled in a scenario (the condition) that break the program execution.

Can two threads write to the same file?

You can have multiple threads write to the same file - but one at a time. All threads will need to enter a synchronized block before writing to the file. In the P2P example - one way to implement it is to find the size of the file and create a empty file of that size.


1 Answers

If you want multiple threads to read from the same file, you need to specify FileShare.Read:

using (var stream = File.Open("theFile.xml", FileMode.Open, FileAccess.Read, FileShare.Read))
{
    ...
}

However, you will not achieve any speedup from this, for multiple reasons:

  1. Your hard disk can only read one thing at a time. Although you have multiple threads running at the same time, these threads will all end up waiting for each other.
  2. You cannot easily parse a part of an XML file. You will usually have to parse the entire XML file every time. Since you have multiple threads reading it all the time, it seems that you are not expecting the file to change. If that is the case, then why do you need to read it multiple times?
like image 131
Timwi Avatar answered Oct 13 '22 20:10

Timwi