Why is .NET's File.Open with a UNC path making excessive SMB calls?

Question

I have a block of code that needs to open and read a lot of small text files from a NAS server using UNC paths. This code is part of a module that was originally written in C++ but is now being converted to C#. The C# version is significantly slower. I determined that the call to open the file accounts for nearly all of the performance difference. Using WireShark I found that this is because the System.IO.File.Open call makes far more SMB network requests than similar C++ code.

The C++ code makes this call:

FILE *f = _wfsopen(fileName, L"r", _SH_DENYWR); 

This results in the following sequence of SMB requests:

NT Create AndX Request, FID: 0x0004, Path: \\a\\i\\a\\q\\~141106162638847.nmd NT Create AndX Response, FID: 0x0004 Trans2 Request, QUERY_FILE_INFO, FID: 0x0004, Query File Basic Info Trans2 Response, FID: 0x0004, QUERY_FILE_INFO Read AndX Request, FID: 0x0004, 1327 bytes at offset 0 Read AndX Response, FID: 0x0004, 1327 bytes Close Request, FID: 0x0004 Close Response, FID: 0x0004 NT Create AndX Request, FID: 0x0005, Path: \\a\\i\\a\\q\\~141106162638847.nmd NT Create AndX Response, FID: 0x0005 

The C# code makes this call:

FileStream f = File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.Read); 

This results in the following sequence of SMB requests:

Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path:  Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path:  Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a Trans2 Response, FIND_FIRST2, Files: a Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i Trans2 Response, FIND_FIRST2, Files: i Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a Trans2 Response, FIND_FIRST2, Files: a Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a\\q Trans2 Response, FIND_FIRST2, Files: q Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path:  Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path:  Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a Trans2 Response, FIND_FIRST2, Files: a Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i Trans2 Response, FIND_FIRST2, Files: i Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a Trans2 Response, FIND_FIRST2, Files: a Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a\\q Trans2 Response, FIND_FIRST2, Files: q Close Request, FID: 0x000f Close Response NT Create AndX Request, FID: 0x0018, Path: \\a\\i\\a\\q\\~141106162638847.nmd NT Create AndX Response, FID: 0x0018 Trans2 Request, QUERY_FILE_INFO, FID: 0x0018, Query File Basic Info Trans2 Response, FID: 0x0018, QUERY_FILE_INFO Read AndX Request, FID: 0x0018, 1327 bytes at offset 0 Read AndX Response, FID: 0x0018, 1327 bytes Close Request, FID: 0x0018 Close Response, FID: 0x0018 NT Create AndX Request, FID: 0x0019, Path: \\a\\i\\a\\q\\~141106162638847.nmd NT Create AndX Response, FID: 0x0019 

Why does System.IO.File.Open make all these extra SMB requests? Is there any way to change this code to avoid all these extra requests?

Answer 1

In short, File.Open calls new FileStream() and new FileStream() does a lot of calls:

1. NormalisePath.

   String filePath = Path.NormalizePath(path, true, maxPath); // fullCheck: true 

leads to this code:

1.a: Get full path:

    if (fullCheck) { ...          result = newBuffer.GetFullPathName(); 

GetFullPathName() calls Win32Native.GetFullPathName one or two times (depending on the lentgh of resulting path).

1.b. Trying to expand short path. Your path contains ~ char, so it looks like a candidate for a path expanding:

    if (mightBeShortFileName) {         bool r = newBuffer.TryExpandShortFileName(); 

as a result, Win32Native.GetLongPathName() is called.

2. FileIoPermission.Demand() (for non-trusted only):

   // All demands in full trust domains are no-ops, so skip  if (!CodeAccessSecurityEngine.QuickCheckForAllDemands()) {     ...     new FileIOPermission(secAccess, control, new String[] { filePath }, false, false).Demand(); 

3. Open fileStream (floppy strikes back;)):

    // Don't pop up a dialog for reading from an emtpy floppy drive int oldMode = Win32Native.SetErrorMode(Win32Native.SEM_FAILCRITICALERRORS); try {     ...     _handle = Win32Native.SafeCreateFile(tempPath, fAccess, share, secAttrs, mode, flagsAndAttributes, IntPtr.Zero); 

4. Win32Native.GetFileType()

Not all of them would lead to smb request, but some will do. I've tried to reproduce chatty requests by debugging with source step-by-step (here's manual for enabling the .net source debugging) and checking the log after each step. Resuts are more similar to your's first listing. If you're really interested in finding the real issue, you'll have to do it yourself.

UPD Note that I've checked current (.net 4.5.2) behavior. It was changed multiple times since 2.0 (e.g. FileIOPermission.Demand() originally was called for full-trusted code too), so it depends:)

Answer 2

I don't really have a specific answer to why the .NET implementation is so chatty, but this behaviour would be due to the implementation of System.IO.FileStream as all that File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.Read); is doing is passing the parameters to the FileStream constructor.

public static FileStream Open(string path, FileMode mode, FileAccess access, FileShare share) {     return new FileStream(path, mode, access, share); } 

Changing the behaviour of FileStream would mean that you would basically have to re-implement the FileStream class which will require a lot of effort.

Your other more simpler alternative would be to create a native wrapper that calls the C++ code you gave. Then call the native wrapper from your C# code.