What would cause ANY .NET application to crash immediately... except a project I create and Debug inside Visual Studio?

Question

My software recently got deployed to a customer who said that the application was crashing immediately after it started. After some initial debugging, the customer provided me remote access to one of the computers which was unable to run the application. I found that the crash wasn't specific to my application. Any application which depended on the .NET framework crashed immediately.

Conveniently, Visual Studio 2008 was installed so I created a quick hello world application on it and clicked Debug. The application worked fine. But, then when I tried to execute the generated binaries in the /bin/Debug/HelloWorld.exe directory outside of visual studio it crashed.

List of things i've tried (UPDATED):

• I checked that "Everyone" has Read&Execute permissions for c:\Windows.
• To test that the problem was with the .NET Framework (and not my application), I attempted to download Paint .NET on to the computers. The setup frontend crashed in the same manner.
• Performed a repair of the .NET framework as outlined in http://support.microsoft.com/kb/908077 (Boy was this fun and time consuming). No luck.
• Installed .NET 3.5 SP1 (before it just had .NET 3.5) Note: my application targets 2.0 so I did this more as a long shot... but i learned in the process that .NET 3.5 SP1 also updates the underlying frameworks.
• Ran Aaron Stebner's .NET Setup Verification Tool. This tool indicated that .NET was successfully installed. (I forget if i checked all the versions but at least 2.0 worked).
• Tested some mini hello world applications which were targeted for .NET 2.0 and .NET 3.5 and both crashed in the same way.
• Tried launching .NET apps via windbg cmd line. Doing this did allow me invoke my simple hello world applications. So, simple .NET hello world works when invoked by windbg or by launching via debug in visual studio... but doesn't if i try to execute it standalone.

I created a dump file using WinDbg. It wasn't all that revealing to me.

FAULTING_IP:  mscorwks!PEImage::GetEntryPointToken+21 79f4ff9d f6401010        test    byte ptr [eax+10h],10h

EXCEPTION_RECORD:  0012f710 -- (.exr 0x12f710) ExceptionAddress: 79f4ff9d (mscorwks!PEImage::GetEntryPointToken+0x00000021) ExceptionCode: c0000005 (Access violation)   ExceptionFlags: 00000000 NumberParameters: 2    Parameter[0]: 00000000    Parameter[1]: 00000010 Attempt to read from address 00000010

FAULTING_THREAD:  00000b44
PROCESS_NAME:  MyProcess.exe
ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION}  Breakpoint  A breakpoint has been reached.

EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments are invalid    
DETOURED_IMAGE: 1    
NTGLOBALFLAG:  0    
APPLICATION_VERIFIER_FLAGS:  0    
MANAGED_STACK: !dumpstack -EE OS Thread Id: 0xb44 (0) Current frame:  ChildEBP RetAddr  Caller,Callee

EXCEPTION_OBJECT: !pe cb10b4 Exception object: 00cb10b4 Exception type: System.ExecutionEngineException Message: <none> InnerException: <none> StackTrace (generated): <none> StackTraceString: <none> HResult: 80131506    
MANAGED_OBJECT_NAME:  System.ExecutionEngineException    
CONTEXT:  0012f72c -- (.cxr 0x12f72c) eax=00000000 ebx=00000000 ecx=00000000 edx=0000000e esi=001a1490 edi=00000001 eip=79f4ff9d esp=0012f9f8 ebp=0012fa1c iopl=0         nv up ei pl zr na pe nc cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00010246 mscorwks!PEImage::GetEntryPointToken+0x21: 79f4ff9d f6401010        test    byte ptr [eax+10h],10h     ds:0023:00000010=?? Resetting default scope    
READ_ADDRESS:  00000010     
FOLLOWUP_IP:  mscorwks!PEImage::GetEntryPointToken+21 79f4ff9d f6401010        test    byte ptr [eax+10h],10h    
BUGCHECK_STR:  APPLICATION_FAULT_NULL_CLASS_PTR_DEREFERENCE_SHUTDOWN    
PRIMARY_PROBLEM_CLASS:  NULL_CLASS_PTR_DEREFERENCE_SHUTDOWN
    DEFAULT_BUCKET_ID:  NULL_CLASS_PTR_DEREFERENCE_SHUTDOWN    
LAST_CONTROL_TRANSFER:  from 79ef02b5 to 79f4ff9d    
STACK_TEXT:   79f4ff9d mscorwks!PEImage::GetEntryPointToken+0x21 79ef02b5 mscorwks!PEFile::GetEntryPointToken+0xa0 79eefeaf mscorwks!SystemDomain::ExecuteMainMethod+0xd4 79fb9793 mscorwks!ExecuteEXE+0x59 79fb96df mscorwks!_CorExeMain+0x15c 7900b1b3 mscoree!_CorExeMain+0x2c 7c817077 kernel32!BaseProcessStart+0x23    

SYMBOL_STACK_INDEX:  0    
SYMBOL_NAME:  mscorwks!PEImage::GetEntryPointToken+21    
FOLLOWUP_NAME:  MachineOwner    
MODULE_NAME: mscorwks    
IMAGE_NAME:  mscorwks.dll    
DEBUG_FLR_IMAGE_TIMESTAMP:  471ef729    
STACK_COMMAND:  .cxr 0012F72C ; kb ; dds 12f9f8 ; kb    
FAILURE_BUCKET_ID:  NULL_CLASS_PTR_DEREFERENCE_SHUTDOWN_80000003_mscorwks.dll!PEImage::GetEntryPointToken    
BUCKET_ID:  APPLICATION_FAULT_NULL_CLASS_PTR_DEREFERENCE_SHUTDOWN_DETOURED_mscorwks!PEImage::GetEntryPointToken+21    
WATSON_STAGEONE_URL:  http://watson.microsoft.com/StageOne/MyProcess_exe/2_4_4_39/4a8a192c/unknown/0_0_0_0/bbbbbbb4/80000003/00000000.htm?Retriage=1

Followup: MachineOwner

Edit 1:The event log details for this error say it's a .NET Runtime version 2.0.50727.3053 - Fatal Execution Engine Error (7A097706)(80131506).

_{(source: blakerobertson.com)}

Edit 2 (10-7-09): This issue is still active.

Edit 3 (3-29-10): This update is to let everyone know that I never did successfully solve the problem. The customer who's machine this was on lost interest in solving it and just reimaged the machine :(. Thanks for all the contributions though.

Answer 1

Based on your windbg output it looks like someone has injected a DLL into the process at process-launch, and that the injection isn't designed for whatever version of mscorwks that has been loaded. If this is a casual workstation (e.g. secretary) I would have it confiscated for MIS/IT to inspect for malware. If it is a machine sitting in a server room I would look toward the customer to perform a relocation to another machine.

I don't suspect this would happen to any other customer, and in 8 years .NET development the only thing that can (expectedly) cause the behavior you're describing is an attempt to run a .NET Application on a system with an older version of the framework installed (e.g. lack of support, results in a standard debug/cancel dialog on most versions of Windows) and that is NOT what this problem is. This is also not related to Processor Architecture, Framework Version nor SP level, it is not related to any commercial AV software, nor any commercial network-security software.

It's clearly not something in your code, and I don't see that it is something you can fix for your client. I know of no tool nor series of steps you can use to resolve this issue short of having the customer re-image the target machine. Before they do so, again, have it ghosted by MIS/IT for potential malware (specifically, malware that wouldn't be distributed through the general public.)

For related reading: http://research.microsoft.com/apps/pubs/default.aspx?id=68568

Good luck.