Recently, my asp.net application crashed. I got the event logs on windows, two type exception message:
Application: w3wp.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an internal error in the .NET Runtime at IP 000007FEF9CFDDE3 (000007FEF9B00000) with exit code 80131506.
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2
Faulting module name: clr.dll, version: 4.0.30319.1008, time stamp: 0x517a18ff
Exception code: 0xc0000005 Fault offset: 0x00000000001fdde3
Faulting process id: 0x4adc
Faulting application start time: 0x01cf69e5bbbae403
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll Report Id: e0c90a5f-0455-11e3-8f0e-005056891553
I also got a mini dump. and I load it into windbg. My clr version on my workstation is 4.0.30319.1022, the clr version of the dump is 4.0.30319.1008. I learn from the link , I download the correct clr/sos/mscordacwks.dll verson and put them into clr1008 folder, then copy the clr1008 folder to the windbg exe folder.
After that, I start to input some commands in windbg:
firstly, I load the sos of version 1008: .load clr1008\sos.dll
I tried to load the correct clr of version 1008
0:020> .cordll -u -ve -lp clr1008
CLRDLL: Loaded DLL clr1008\mscordacwks.dll
CLR DLL status: Loaded DLL clr1008\mscordacwks.dll
0:020> .cordll -u -ve -l
CLRDLL: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\mscordacwks.dll:4.0.30319.1022 f:8
doesn't match desired version 4.0.30319.1008 f:8
CLRDLL: Loaded DLL c:\symbols\mscordacwks_AMD64_AMD64_4.0.30319.1008.dll\517A18FF965000\mscordacwks_AMD64_AMD64_4.0.30319.1008.dll
CLR DLL status: Loaded DLL c:\symbols\mscordacwks_AMD64_AMD64_4.0.30319.1008.dll\517A18FF965000\mscordacwks_AMD64_AMD64_4.0.30319.1008.dll
0:020> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
Debugger CompCtrlDb Connection::Open failed 80004005
Unable to open image file: D:\Users\ABC\Downloads\windbg\sym\clr.dll\517A18FF965000\clr.dll
ϵͳÕÒ²»µ½Ö¸¶¨µÄÎļþ¡£
TRIAGER: Could not open triage file : D:\Users\ABC\Downloads\windbg\triage\guids.ini, error 2
Unable to load image C:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll, Win32 error 0n2
Debugger CompCtrlDb Connection::Open failed 80004005
Unable to open image file: D:\Users\ABC\Downloads\windbg\sym\clr.dll\517A18FF965000\clr.dll
ϵͳÕÒ²»µ½Ö¸¶¨µÄÎļþ¡£
TRIAGER: Could not open triage file : D:\Users\ABC\Downloads\windbg\triage\modclass.ini, error 2
Debugger Dbgportaldb Connection::Open failed 80004005
Database Dbgportaldb not connected
FAULTING_IP:
clr+1fde45
000007fe`f9cfde45 41f70000000080 test dword ptr [r8],80000000h
EXCEPTION_RECORD: ffffffffffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 000007fef9cfde45 (clr+0x00000000001fde45)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000001
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: ffffffffffffffff
Attempt to read from address ffffffffffffffff
DEFAULT_BUCKET_ID: INVALID_POINTER_READ
PROCESS_NAME: w3wp.exe
ERROR_CODE: (NTSTATUS) 0xc0000005 - 0x%08lx
EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - 0x%08lx
EXCEPTION_PARAMETER1: 0000000000000000
EXCEPTION_PARAMETER2: ffffffffffffffff
READ_ADDRESS: ffffffffffffffff
FOLLOWUP_IP:
clr+1fde45
000007fe`f9cfde45 41f70000000080 test dword ptr [r8],80000000h
NTGLOBALFLAG: 0
APPLICATION_VERIFIER_FLAGS: 0
MANAGED_STACK: !dumpstack -EE
OS Thread Id: 0x471c (20)
Current frame:
Child-SP RetAddr Caller, Callee
FAULTING_THREAD: 000000000000471c
PRIMARY_PROBLEM_CLASS: INVALID_POINTER_READ
BUGCHECK_STR: APPLICATION_FAULT_INVALID_POINTER_READ
IP_ON_HEAP: fffffffffffffffe
The fault address in not in any loaded module, please check your build's rebase
log at <releasedir>\bin\build_logs\timebuild\ntrebase.log for module which may
contain the address if it were loaded.
FRAME_ONE_INVALID: 1
LAST_CONTROL_TRANSFER: from fffffffffffffffe to 000007fef9cfde45
STACK_TEXT:
00000000`047df1c0 ffffffff`fffffffe : 000007fe`f9c192d4 000007fe`f9b922c0 00000000`022b2ea0 00000000`00000002 : clr+0x1fde45
00000000`047df1c8 000007fe`f9c192d4 : 000007fe`f9b922c0 00000000`022b2ea0 00000000`00000002 00000000`0222b680 : 0xffffffff`fffffffe
00000000`047df1d0 000007fe`f9b922c0 : 00000000`022b2ea0 00000000`00000002 00000000`0222b680 00000001`41e2c328 : clr+0x1192d4
00000000`047df1d8 00000000`022b2ea0 : 00000000`00000002 00000000`0222b680 00000001`41e2c328 00000001`41e2c390 : clr+0x922c0
00000000`047df1e0 00000000`00000002 : 00000000`0222b680 00000001`41e2c328 00000001`41e2c390 00000000`0222b7e0 : 0x22b2ea0
00000000`047df1e8 00000000`0222b680 : 00000001`41e2c328 00000001`41e2c390 00000000`0222b7e0 00000000`0222bcc0 : 0x2
00000000`047df1f0 00000001`41e2c328 : 00000001`41e2c390 00000000`0222b7e0 00000000`0222bcc0 00000000`0222b7e8 : 0x222b680
00000000`047df1f8 00000001`41e2c390 : 00000000`0222b7e0 00000000`0222bcc0 00000000`0222b7e8 00000000`047df2f8 : 0x1`41e2c328
00000000`047df200 00000000`0222b7e0 : 00000000`0222bcc0 00000000`0222b7e8 00000000`047df2f8 00000000`00000010 : 0x1`41e2c390
00000000`047df208 00000000`0222bcc0 : 00000000`0222b7e8 00000000`047df2f8 00000000`00000010 00000000`02224fd0 : 0x222b7e0
00000000`047df210 00000000`0222b7e8 : 00000000`047df2f8 00000000`00000010 00000000`02224fd0 00000000`0222a8c0 : 0x222bcc0
00000000`047df218 00000000`047df2f8 : 00000000`00000010 00000000`02224fd0 00000000`0222a8c0 00000000`00000002 : 0x222b7e8
00000000`047df220 00000000`00000010 : 00000000`02224fd0 00000000`0222a8c0 00000000`00000002 00000000`00000002 : 0x47df2f8
00000000`047df228 00000000`02224fd0 : 00000000`0222a8c0 00000000`00000002 00000000`00000002 00000001`7f8a0e88 : 0x10
00000000`047df230 00000000`0222a8c0 : 00000000`00000002 00000000`00000002 00000001`7f8a0e88 00000001`7f8a0eb0 : 0x2224fd0
00000000`047df238 00000000`00000002 : 00000000`00000002 00000001`7f8a0e88 00000001`7f8a0eb0 000007fe`f9cfba57 : 0x222a8c0
00000000`047df240 00000000`00000002 : 00000001`7f8a0e88 00000001`7f8a0eb0 000007fe`f9cfba57 00000000`0222a8c0 : 0x2
00000000`047df248 00000001`7f8a0e88 : 00000001`7f8a0eb0 000007fe`f9cfba57 00000000`0222a8c0 00000001`00000020 : 0x2
00000000`047df250 00000001`7f8a0eb0 : 000007fe`f9cfba57 00000000`0222a8c0 00000001`00000020 00000001`7f8a0e80 : 0x1`7f8a0e88
00000000`047df258 000007fe`f9cfba57 : 00000000`0222a8c0 00000001`00000020 00000001`7f8a0e80 00000000`0222b7f0 : 0x1`7f8a0eb0
00000000`047df260 00000000`0222a8c0 : 00000001`00000020 00000001`7f8a0e80 00000000`0222b7f0 00000000`00000002 : clr+0x1fba57
00000000`047df268 00000001`00000020 : 00000001`7f8a0e80 00000000`0222b7f0 00000000`00000002 00000001`7f8a0e80 : 0x222a8c0
00000000`047df270 00000001`7f8a0e80 : 00000000`0222b7f0 00000000`00000002 00000001`7f8a0e80 00000000`00000048 : 0x1`00000020
00000000`047df278 00000000`0222b7f0 : 00000000`00000002 00000001`7f8a0e80 00000000`00000048 00000000`00000001 : 0x1`7f8a0e80
00000000`047df280 00000000`00000002 : 00000001`7f8a0e80 00000000`00000048 00000000`00000001 00000000`018215f8 : 0x222b7f0
00000000`047df288 00000001`7f8a0e80 : 00000000`00000048 00000000`00000001 00000000`018215f8 00000000`00000100 : 0x2
00000000`047df290 00000000`00000048 : 00000000`00000001 00000000`018215f8 00000000`00000100 00000000`0222f500 : 0x1`7f8a0e80
00000000`047df298 00000000`00000001 : 00000000`018215f8 00000000`00000100 00000000`0222f500 00000000`00000000 : 0x48
00000000`047df2a0 00000000`018215f8 : 00000000`00000100 00000000`0222f500 00000000`00000000 00000000`047df660 : 0x1
00000000`047df2a8 00000000`00000100 : 00000000`0222f500 00000000`00000000 00000000`047df660 00000001`7f8a0e80 : 0x18215f8
00000000`047df2b0 00000000`0222f500 : 00000000`00000000 00000000`047df660 00000001`7f8a0e80 00000000`00000002 : 0x100
00000000`047df2b8 00000000`00000000 : 00000000`047df660 00000001`7f8a0e80 00000000`00000002 00000000`01a61bf8 : 0x222f500
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: clr+1fde45
FOLLOWUP_NAME: wintriag
MODULE_NAME: clr //here is a hyperlink, so I click then got an output.
IMAGE_NAME: clr.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 517a18ff
STACK_COMMAND: ~20s; .ecxr ; kb
FAILURE_BUCKET_ID: INVALID_POINTER_READ_c0000005_clr.dll!Unknown
BUCKET_ID: X64_APPLICATION_FAULT_INVALID_POINTER_READ_clr+1fde45
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/w3wp_exe/7_5_7601_17514/4ce7afa2/clr_dll/4_0_30319_1008/517a18ff/c0000005/001fde45.htm?Retriage=1
Followup: wintriag
---------
0:020> lmvm clr
start end module name
000007fe`f9b00000 000007fe`fa465000 clr T (no symbols)
Loaded symbol image file: clr.dll
Image path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Image name: clr.dll
Timestamp: Fri Apr 26 14:04:47 2013 (517A18FF)
CheckSum: 0095CE19
ImageSize: 00965000
File version: 4.0.30319.1008
Product version: 4.0.30319.1008
File flags: 8 (Mask 3F) Private
File OS: 4 Unknown Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: Microsoft Corporation
ProductName: Microsoft® .NET Framework
InternalName: clr.dll
OriginalFilename: clr.dll
ProductVersion: 4.0.30319.1008
FileVersion: 4.0.30319.1008 (RTMGDR.030319-1000)
PrivateBuild: DDBLD208
FileDescription: Microsoft .NET Runtime Common Language Runtime - WorkStation
LegalCopyright: © Microsoft Corporation. All rights reserved.
Comments: Flavor=Retail
Questions: 1) It seems that I didn't load the sos or clr.dll correctly, What should I do next? Should I copy the clr.dll of version 1008 to the symbol path? It seems that I already have the correct version of mscordacwks.dll.
2) the exception seems caused by unmanaged code, How to find the root cause?
3) I also run '.ecxr', but it's hard for me to understand the output.
Update: Yesterday I post the output of ' !analyze -v '. Today, I retried to analyze the dump file, 1) load clr1008\sos.dll, 2) load the clr.dll by exe '.cordll -u -ve -lp clr1008' 3) run the !analyze, the output is list above. The error of mismatch sos is gone.
The version of SOS does not match the version of CLR you are debugging. Please
load the matching version of SOS for the version of CLR you are debugging.
CLR Version: 4.0.30319.1008
SOS Version: 4.0.30319.1022
The above message is gone. The below is still there.
doesn't match desired version 4.0.30319.1008
The output of the '.chain' is below:
0:020> .chain
Extension DLL search Path:
D:\Users\ABC\Downloads\windbg\WINXP;.....
Extension DLL chain:
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos: image 4.0.30319.1022, API 1.0.0, built Wed Jan 08 14:45:19 2014
[path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos.dll]
clr1008\sos.dll: image 4.0.30319.1008, API 1.0.0, built Fri Apr 26 14:21:22 2013
[path: D:\Users\ABC\Downloads\windbg\clr1008\sos.dll]
dbghelp: image 6.13.0001.776, API 6.1.6, built Sat May 15 04:57:22 2010
[path: D:\Users\ABC\Downloads\windbg\dbghelp.dll]
ext: image 6.13.0001.776, API 1.0.0, built Sat May 15 04:57:40 2010
[path: D:\Users\ABC\Downloads\windbg\winext\ext.dll]
exts: image 6.13.0001.776, API 1.0.0, built Sat May 15 04:57:13 2010
[path: D:\Users\ABC\Downloads\windbg\WINXP\exts.dll]
uext: image 6.13.0001.776, API 1.0.0, built Sat May 15 04:57:09 2010
[path: D:\Users\ABC\Downloads\windbg\winext\uext.dll]
ntsdexts: image 6.1.7752.0, API 1.0.0, built Sat May 15 04:52:37 2010
[path: D:\Users\ABC\Downloads\windbg\WINXP\ntsdexts.dll]
Is the mismatch of CLR 1008 not important or important, will it impacts the analysing? Because of the mismatch, I can't see the exception details? Or It's a native code exception, I can't see it very literally?
Update 2: I use DebugDiag 1.2 to analyze the dump file, I got the report, the first row of the report says:
WARNING - DebugDiag was not able to locate debug symbols for clr.dll, so the information below may be incomplete.
In w3wp.exe.2336.dmp the assembly instruction at clr+1fdde3 in C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll has caused an access violation exception (0xC0000005) when trying to read from memory location 0x7e710001 on thread 16
Update 3: print the output of '.ecxr' here, thread 16:
0:016> .ecxr
rax=00000000023d7a40 rbx=00000000028df550 rcx=000000015f850000
rdx=0000000000000000 rsi=0000000000000000 rdi=00000000028df060
rip=000007fef9cfdde3 rsp=00000000028df620 rbp=516db4ea5f156833
r8=0000000000000011 r9=000000000000000e r10=000000017e710001
r11=0000000000000020 r12=0000000140950378 r13=00001a7d1e7feffe
r14=0000000000000000 r15=0000000140950300
iopl=0 nv up ei ng nz na pe cy
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010281
clr+0x1fdde3:
000007fe`f9cfdde3 498b0a mov rcx,qword ptr [r10] ds:00000001`7e710001=????????????????
Update 4: the stack output get from the DebugDiag report:
Thread 16 - System ID 2308 This thread is not fully resolved and may or may not be a problem. Further analysis of these threads may be required.
Function Source
clr+1fdde3
0x4eaa1180
0x00000100
clr+8cc74
clr+8f0e0
0x00000002
0x067e8420
0x00000001`40950300
0x00000001`40950368
0x067e8520
0x067e9d20
0x067e8528
clr+8cc74
0x00000010
0x023cd2f0
0x023d5000
0x00000002
0x00000002
0x00000001`4093e0f0
0x00000001`4093e100
clr+1fba57
0x023d5000
0x00000001`00000020
0x028049e0
0x067e8530
0x00000002
0x00000001`4093e0e0
0x00000030
0x00000001
0x028df7e8
0x0000018a
System_Data_ni!IID_IRowsetBookmark+43598
Update 5:
0:016> ~#s;
clr+0x1fdde3:
000007fe`f9cfdde3 498b0a mov rcx,qword ptr [r10] ds:00000001`7e710001=????????????????
0:016> .exr -1;
ExceptionAddress: 000007fef9cfdde3 (clr+0x00000000001fdde3)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000001
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: 000000017e710001
Attempt to read from address 000000017e710001
0:016> kb;
RetAddr : Args to Child : Call Site
00000000`4eaa1180 : 00000000`00000100 000007fe`f9b8cc74 000007fe`f9b8f0e0 00000000`00000002 : clr+0x1fdde3
00000000`00000100 : 000007fe`f9b8cc74 000007fe`f9b8f0e0 00000000`00000002 00000000`067e8420 : 0x4eaa1180
000007fe`f9b8cc74 : 000007fe`f9b8f0e0 00000000`00000002 00000000`067e8420 00000001`40950300 : 0x100
000007fe`f9b8f0e0 : 00000000`00000002 00000000`067e8420 00000001`40950300 00000001`40950368 : clr+0x8cc74
00000000`00000002 : 00000000`067e8420 00000001`40950300 00000001`40950368 00000000`067e8520 : clr+0x8f0e0
00000000`067e8420 : 00000001`40950300 00000001`40950368 00000000`067e8520 00000000`067e9d20 : 0x2
00000001`40950300 : 00000001`40950368 00000000`067e8520 00000000`067e9d20 00000000`067e8528 : 0x67e8420
00000001`40950368 : 00000000`067e8520 00000000`067e9d20 00000000`067e8528 000007fe`f9b8cc74 : 0x1`40950300
00000000`067e8520 : 00000000`067e9d20 00000000`067e8528 000007fe`f9b8cc74 00000000`00000010 : 0x1`40950368
00000000`067e9d20 : 00000000`067e8528 000007fe`f9b8cc74 00000000`00000010 00000000`023cd2f0 : 0x67e8520
00000000`067e8528 : 000007fe`f9b8cc74 00000000`00000010 00000000`023cd2f0 00000000`023d5000 : 0x67e9d20
000007fe`f9b8cc74 : 00000000`00000010 00000000`023cd2f0 00000000`023d5000 00000000`00000002 : 0x67e8528
00000000`00000010 : 00000000`023cd2f0 00000000`023d5000 00000000`00000002 00000000`00000002 : clr+0x8cc74
00000000`023cd2f0 : 00000000`023d5000 00000000`00000002 00000000`00000002 00000001`4093e0f0 : 0x10
00000000`023d5000 : 00000000`00000002 00000000`00000002 00000001`4093e0f0 00000001`4093e100 : 0x23cd2f0
00000000`00000002 : 00000000`00000002 00000001`4093e0f0 00000001`4093e100 000007fe`f9cfba57 : 0x23d5000
00000000`00000002 : 00000001`4093e0f0 00000001`4093e100 000007fe`f9cfba57 00000000`023d5000 : 0x2
00000001`4093e0f0 : 00000001`4093e100 000007fe`f9cfba57 00000000`023d5000 00000001`00000020 : 0x2
00000001`4093e100 : 000007fe`f9cfba57 00000000`023d5000 00000001`00000020 00000000`028049e0 : 0x1`4093e0f0
000007fe`f9cfba57 : 00000000`023d5000 00000001`00000020 00000000`028049e0 00000000`067e8530 : 0x1`4093e100
00000000`023d5000 : 00000001`00000020 00000000`028049e0 00000000`067e8530 00000000`00000002 : clr+0x1fba57
00000001`00000020 : 00000000`028049e0 00000000`067e8530 00000000`00000002 00000001`4093e0e0 : 0x23d5000
00000000`028049e0 : 00000000`067e8530 00000000`00000002 00000001`4093e0e0 00000000`00000030 : 0x1`00000020
00000000`067e8530 : 00000000`00000002 00000001`4093e0e0 00000000`00000030 00000000`00000001 : 0x28049e0
00000000`00000002 : 00000001`4093e0e0 00000000`00000030 00000000`00000001 00000000`028df7e8 : 0x67e8530
00000001`4093e0e0 : 00000000`00000030 00000000`00000001 00000000`028df7e8 00000000`0000018a : 0x2
00000000`00000030 : 00000000`00000001 00000000`028df7e8 00000000`0000018a 000007fe`e97e0d10 : 0x1`4093e0e0
00000000`00000001 : 00000000`028df7e8 00000000`0000018a 000007fe`e97e0d10 00000000`00000002 : 0x30
00000000`028df7e8 : 00000000`0000018a 000007fe`e97e0d10 00000000`00000002 00000001`4024f070 : 0x1
00000000`0000018a : 000007fe`e97e0d10 00000000`00000002 00000001`4024f070 00000000`028049e0 : 0x28df7e8
000007fe`e97e0d10 : 00000000`00000002 00000001`4024f070 00000000`028049e0 00000001`4024f098 : 0x18a
00000000`00000002 : 00000001`4024f070 00000000`028049e0 00000001`4024f098 00000000`00000002 : System_Data_ni+0x190d10
Update last time: Before, I didn't get the correct stack displayed. I search on the Internet. then I got it from here
0:020> !clrstack
PDB symbol for clr.dll not loaded
0:020> .exepath+ clr1008
Executable image search path is: clr1008
Expanded Executable image search path is: clr1008
0:020> .reload //necessary
.............
0:020> kb
RetAddr : Args to Child : Call Site
000007fe`f9cfba57 : 00000000`0222a8c0 00000001`00000020 00000001`7f8a0e80 00000000`0222b7f0 : clr!SVR::gc_heap::mark_object_simple1+0xbd6
000007fe`f9cf9aa9 : 00000000`0222a8c0 000007fe`ecee2018 00000000`00000002 000007fe`ecee2018 : clr!SVR::gc_heap::mark_object_simple+0x4d7
000007fe`f9c190df : 00000001`7f8a0e80 00000000`0222a8c0 00000000`01a61bf8 00006246`ffffffff : clr!SVR::GCHeap::Promote+0x161
000007fe`f9c193c3 : 00000000`01a61bf8 000007fe`f9c19098 ffffffff`fffffe00 00000000`01a61c00 : clr!CalculateSizedRefSize+0x47
000007fe`f9c1932d : 00000000`00000140 00000000`00000001 00000000`00000005 00000000`047df428 : clr!ScanConsecutiveHandlesWithUserData+0x67
000007fe`f9b8d43e : 00000000`01a60000 000007fe`f9c192d4 00000000`00000005 00000000`022b7880 : clr!BlockScanBlocksWithUserData+0x59
000007fe`f9b8ca05 : 00000000`047df5a0 00000000`047df610 000007fe`f9c192d4 00000000`047df610 : clr!TableScanHandles+0x219
000007fe`f9c19211 : 00000000`00000002 00000000`00000004 00000000`00000008 00000005`ff5cd000 : clr!HndScanHandlesForGC+0x1ad
000007fe`f9bfff7a : 00000000`0222a8c0 000007fe`f9cf9890 00000000`00000002 00000000`00000008 : clr!ScanSizedRefByAD+0xf8
000007fe`f9cf7da3 : 00000000`0222a8c0 00000000`00000002 00000000`0222a8c0 000007fe`00000001 : clr!SVR::gc_heap::mark_phase+0x19c
000007fe`f9cf8992 : 00000ba3`e8dc002d 00000000`00000000 00000000`0222b098 00000000`0222a8c0 : clr!SVR::gc_heap::gc1+0x54
000007fe`f9cf7ac8 : 00000000`00000000 00000000`0222a8c0 00000000`047dfa40 00000000`00002000 : clr!SVR::gc_heap::garbage_collect+0x372
000007fe`f9c1eaeb : 00000000`047df900 00000000`0222a8c0 00000000`047dfa40 00000000`00000000 : clr!SVR::gc_heap::gc_thread_function+0x78
00000000`7722652d : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : clr!SVR::gc_heap::gc_thread_stub+0x82
00000000`7735c541 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0xd
00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d
0:020> lmv m clr
start end module name
000007fe`f9b00000 000007fe`fa465000 clr (pdb symbols) c:\symbols\clr.pdb\118B3CCCBA4A4A709B496CFB294BF5162\clr.pdb
Loaded symbol image file: clr.dll
Mapped memory image file: clr1008\clr.dll ***here is different, please compare it with previous output.
Image path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Image name: clr.dll
Dammit, I finally done with it. ^_^
If you are debugging a minidump, you need to make sure that your executable path is pointing to clr.dll as well.
There are mutiple questions in one.
Answer to question 1)
You have loaded two versions of SOS:
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos: image 4.0.30319.1022, API 1.0.0, built Wed Jan 08 14:45:19 2014
[path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos.dll]
clr1008\sos.dll: image 4.0.30319.1008, API 1.0.0, built Fri Apr 26 14:21:22 2013
[path: D:\Users\lint\Downloads\20131201\windbg\clr1008\sos.dll]
You have now the following choices:
The commands are
.unload C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos
.setdll clr1008\sos.dll
clr1008\sos!<command>
respectively
Answer to question 2)
Just forget about the managed part and debug the native way. For live debugging you would do
sxe 0xc0000005; *** Break on the exception code causing the trouble
g; *** Go and wait for the exception to happen
and at the time the exception happens
.exr -1; *** Just to see whether it's not something else
kb; *** Get the callstack
For the dump
~#s; *** Select the thread with the exception
.exr -1; *** See if we really have the C0000005 exception
kb; *** Get the callstack
should also work.
That usually helps for 3rd party native code. I'm not sure if that helps much in your case. Your problem sounds much like the StackOverflow question What are the causes and solutions of exception code c0000005 in mscorwks.dll?
If I'm not sure whether the symbols are correct, I always use
.symfix d:\symbols; *** Be sure to download symbols
.reload /f; *** Force reloading symbols
ld*; *** Just another type of reload since I don't know which one's better
If the heap was destroyed, the information in the dump may be misleading. In that case I'd try to reproduce the issue with GFlags settings for w3wp.exe: Enable heap tail checking, Enable heap free checking, Enable heap parameter checking, Enable heap validation on call
Answer to question 3)
I seldom use .ecxr, sorry I can't help on that.
Analysis of the dump you provided
The dump is only 24 MB, so it's unlikely a dump with full heap information. SOS would need a full dump (/ma) to fully work correctly. But since we expect a native issue, that doesn't really matter. And SOS might still output other useful information, e.g. about managed threads.
Doing a complete symbol load (ld*), it cannot find all symbols, but that's normal as well. At least I get much better results on the CLR stuff (in WinDbg 6.2.9200.16384 AMD64):
0:020> k
Child-SP RetAddr Call Site
00000000`047df1c0 000007fe`f9cfba57 clr!SVR::gc_heap::mark_object_simple1+0xbd6
00000000`047df260 000007fe`f9cf9aa9 clr!SVR::gc_heap::mark_object_simple+0x4d7
00000000`047df2f0 000007fe`f9c190df clr!SVR::GCHeap::Promote+0x161
00000000`047df370 000007fe`f9c193c3 clr!CalculateSizedRefSize+0x47
00000000`047df3a0 000007fe`f9c1932d clr!ScanConsecutiveHandlesWithUserData+0x67
00000000`047df3e0 000007fe`f9b8d43e clr!BlockScanBlocksWithUserData+0x59
00000000`047df420 000007fe`f9b8ca05 clr!TableScanHandles+0x219
00000000`047df4e0 000007fe`f9c19211 clr!HndScanHandlesForGC+0x1ad
00000000`047df590 000007fe`f9bfff7a clr!ScanSizedRefByAD+0xf8
00000000`047df630 000007fe`f9cf7da3 clr!SVR::gc_heap::mark_phase+0x19c
00000000`047df6c0 000007fe`f9cf8992 clr!SVR::gc_heap::gc1+0x54
00000000`047df730 000007fe`f9cf7ac8 clr!SVR::gc_heap::garbage_collect+0x372
00000000`047df7c0 000007fe`f9c1eaeb clr!SVR::gc_heap::gc_thread_function+0x78
00000000`047df810 00000000`7722652d clr!SVR::gc_heap::gc_thread_stub+0x82
00000000`047dfa60 00000000`7735c541 kernel32!BaseThreadInitThunk+0xd
00000000`047dfa90 00000000`00000000 ntdll!RtlUserThreadStart+0x1d
So that looks much like a garbage collector thing.