Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is Parallel File.Read Faster than Sequential Read?

Tags:

c#

file-io

I just wonder is parallel File.Read using PLINQ/Parallel can be faster? My code is as follows ( .Net 4.0):

public static void ReadFileParallel(List<string> fileName)
{
   Parallel.Foreach(fileName, file=>File.Read(file));
}

public static void ReadFilePLINQ(List<string> fileName)
{
    fileName.AsParallel().foreach(file=>File.Read(file));
}

The reason I ask this is because I thought that file reading is IO bound, so doing parallel won't help, am I right?

like image 234
Graviton Avatar asked Jul 13 '10 14:07

Graviton


2 Answers

It depends.

If your files were in different locations, on different network shares, or on different physical hard drives, then yes, parallel loading will probably help. If they're on a single spinning hard drive, reading the files in parallel will probably hurt your performance significantly due to the extra seek time that you will likely incur from these parallel reads.

If your files are on an SSD, you will probably get slightly less performance, but it would depend on how many files you're reading in parallel and what their sizes are. I imagine that at a certain file size threshold and number of parallel reads, performance will drop significantly. Hard to tell on that one without some experimentation.

like image 82
Dave Markle Avatar answered Sep 20 '22 10:09

Dave Markle


You'd think so, but that's not what measurements show. When file I/O has significant latency, particularly over networks, doing it in parallel can keep the pipe filled.

like image 45
Steven Sudit Avatar answered Sep 22 '22 10:09

Steven Sudit