Before reading this article, it’s good to have some knowledge about internals of windows registry. Here is a very interesting article about it.
The machine where the BSOD occured is x64 running windows 8.1. First, we activate full memory dumps and we wait for the crash.
When the crash occurs, we get the memory dump. We open it with windbg, load symbols for kernel and user mode, and we analyze -v.
3: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
REGISTRY_ERROR (51)
Something has gone badly wrong with the registry. If a kernel debugger
is available, get a stack trace. It can also indicate that the registry got
an I/O error while trying to read one of its files, so it can be caused by
hardware problems or filesystem corruption.
It may occur due to a failure in a refresh operation, which is used only
in by the security system, and then only when resource limits are encountered.
Arguments:
Arg1: 0000000000000001, (reserved)
Arg2: ffffc000a7277000, (reserved)
Arg3: 00000000ccbe055b, depends on where Windows bugchecked, may be pointer to hive
Arg4: 00000000000001ea, depends on where Windows bugchecked, may be return code of
HvCheckHive if the hive is corrupt.
Debugging Details:
------------------
DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT
BUGCHECK_STR: 0x51
PROCESS_NAME: svchost.exe
CURRENT_IRQL: 0
ANALYSIS_VERSION: 6.3.9600.16384 (debuggers(dbg).130821-1623) amd64fre
LAST_CONTROL_TRANSFER: from fffff8011914ce46 to fffff80118d529a0
STACK_TEXT:
ffffd000`8b3137b8 fffff801`1914ce46 : 00000000`00000051 00000000`00000001 ffffc000`a7277000 00000000`ccbe055b : nt!KeBugCheckEx
ffffd000`8b3137c0 fffff801`18fe3463 : ffffc000`b211da78 00000000`04223a78 00000000`00000000 ffffc000`a7277000 : nt! ?? ::NNGAKEGL::`string'+0x1db76
ffffd000`8b313820 fffff801`18fe434a : ffffc000`b2375e64 00000000`03d01a50 ffffc000`37643535 ffffc000`ccbe055b : nt!HvFreeCell+0xf7
ffffd000`8b3138a0 fffff801`18fe2836 : ffffc000`a7277000 ffffc000`af1574b4 ffffc000`a7277000 00000000`03d01a90 : nt!CmpFreeValueData+0x6a
ffffd000`8b3138d0 fffff801`18fe10e5 : 00000000`00000000 ffffe000`28d8b4c0 ffffc000`a7277000 00000000`00000000 : nt!CmpFreeValue+0x26
ffffd000`8b313900 fffff801`19034075 : 01d063e1`4dd2cd24 ffffd000`8b3139a9 ffffc000`af1574b4 ffffc000`a995b948 : nt!CmpFreeKeyByCell+0xf9
ffffd000`8b313940 fffff801`19033d26 : ffffc000`a8e89af0 ffffd000`8b313a01 00000000`00000101 00000000`00000000 : nt!CmDeleteKey+0x1fd
ffffd000`8b313a10 fffff801`18d5e1b3 : 00000000`00000008 ffffe000`28d8b080 00000000`000001b0 00000000`00000000 : nt!NtDeleteKey+0x25e
ffffd000`8b313b00 00007ff8`15ea19ba : 00007ff8`1308e7eb 00000096`9953ccb0 00000000`00000000 00000000`0000ff00 : nt!KiSystemServiceCopyEnd+0x13
00000096`9953cbd8 00007ff8`1308e7eb : 00000096`9953ccb0 00000000`00000000 00000000`0000ff00 00000000`00010000 : ntdll!NtDeleteKey+0xa
00000096`9953cbe0 00007ff8`1308e6c5 : 00000000`000001a8 00000000`000001b0 00000000`00000000 00000000`00000000 : KERNELBASE!LocalBaseRegDeleteKeyEx+0xf7
00000096`9953cc70 00007ff8`1305efa1 : 00000000`000001a8 00000000`00000000 00000000`00000000 00000000`00000000 : KERNELBASE!RegDeleteKeyExInternalW+0xa1
00000096`9953ccd0 00007fff`fc90195c : 00000000`000001a8 00000096`98b2de20 00007ff8`00000101 00000000`000001a4 : KERNELBASE!RegDeleteTreeW+0x14d
00000096`9953cd50 00007fff`fc8fabc5 : 00000000`00000001 00000000`00000004 00000000`00000004 00000000`00000000 : defragsvc!SxRegDeleteKey+0x60
00000096`9953cdc0 00007fff`fc8ee0ec : 00007fff`fc8eb860 00007fff`fc8eb860 00000096`991965b0 00000096`98b2d710 : defragsvc!CDefragAsyncWorker::_LoadStatisticsFromRegistry+0x6f5
00000096`9953d360 00007fff`fc8f2956 : 00000000`00000000 00000096`9953d9e0 00000096`98b2e060 00000096`98b2e060 : defragsvc!CDefragAsyncWorker::_Initialize+0x19c
00000096`9953d8d0 00007fff`fc8ef347 : 00000000`00000000 00000096`9953dbf0 00000000`00000000 00000000`00000000 : defragsvc!CDefragAsyncWorker::CreateInstance+0xe6
00000096`9953d990 00007fff`fc8edca8 : 00000000`00000000 00000000`00000000 00000000`00000000 00007ff8`00000001 : defragsvc!CDefragEngine::_CreateAsyncForVolume+0x1f5
00000096`9953dab0 00007fff`fc8f3428 : 00000000`00000000 00000000`00000198 00000000`00000198 00000096`99196440 : defragsvc!CDefragEngine::_RefreshVolumeList+0x51b
00000096`9953e060 00007fff`fc8f32a5 : 00000000`00000000 00000096`991964f0 00000096`99196530 00000000`00000158 : defragsvc!CDefragEngine::FinalConstruct+0x12c
00000096`9953e0f0 00007fff`fc8ea51b : 00000000`00000000 00000000`00000001 00000096`991962f0 00000096`991962a0 : defragsvc!CSxComObjectCached<CDefragEngine>::CreateInstance+0xa1
00000096`9953e120 00007ff8`143e7cb4 : 00000096`991962a0 00007fff`fc8ea4a0 00000000`00000000 00000000`00000000 : defragsvc!CSxComClassFactorySingleton<CDefragEngine>::CreateInstance+0x7b
00000096`9953e170 00007ff8`143e7ad0 : 00000000`00000000 00007ff8`143cd330 00000096`9953e240 00000000`00000000 : combase!GetInstanceHelperMulti+0x4c [d:\9147\com\combase\objact\objact.cxx @ 2821]
00000096`9953e1c0 00007ff8`14012053 : 00000000`00000000 00000096`9953e840 00007ff8`143e7550 00000096`98b1eb40 : combase!CObjServer::CreateInstance+0x57e [d:\9147\com\combase\objact\defcxact.cxx @ 662]
00000096`9953e3a0 00007ff8`1401c70d : 00000096`9953e820 00007ff8`1444a722 00000096`98b26418 00000000`00000000 : RPCRT4!Invoke+0x73
00000096`9953e400 00007ff8`14029cb3 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : RPCRT4!NdrStubCall2+0x35e
00000096`9953ea70 00007ff8`145287e3 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : RPCRT4!NdrStubCall3+0x156
00000096`9953ed90 00007ff8`145297bd : 00000000`00000001 00007ff8`14528770 00000000`00000000 00000000`00000000 : combase!CStdStubBuffer_Invoke+0x6b [d:\9147\com\combase\ndr\ndrole\stub.cxx @ 1586]
00000096`9953edd0 00007ff8`143b4f5a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : combase!SyncStubInvoke+0x21d [d:\9147\com\combase\dcomrem\channelb.cxx @ 1664]
00000096`9953ef70 00007ff8`1452951f : 00000000`00000000 00000096`98b11f70 00000096`9953f240 00000000`00000000 : combase!CCtxComChnl::ContextInvoke+0x27a [d:\9147\com\combase\dcomrem\ctxchnl.cxx @ 1377]
00000096`9953f180 00007ff8`14528fb0 : 00000096`98b25f30 00000096`98b25f30 00000096`98b12600 00007ff8`14528dc9 : combase!AppInvoke+0x1af [d:\9147\com\combase\dcomrem\channelb.cxx @ 1481]
00000096`9953f270 00007ff8`14529b35 : 00000000`00000000 00000000`00070005 00000096`98b157d0 00000000`00000000 : combase!ComInvokeWithLockAndIPID+0x676 [d:\9147\com\combase\dcomrem\channelb.cxx @ 2314]
00000096`9953f4b0 00007ff8`140123c7 : 00000000`00000158 00007ff8`14529cf0 00000000`00000000 00000096`98b23eb0 : combase!ThreadInvoke+0x48a [d:\9147\com\combase\dcomrem\channelb.cxx @ 5488]
00000096`9953f580 00007ff8`14012220 : 00000096`98b1f360 00000000`00000000 00000096`9953f740 00000000`000618c8 : RPCRT4!DispatchToStubInCNoAvrf+0x33
00000096`9953f5d0 00007ff8`14029a78 : 00000096`98b1f360 00000000`00000000 00000096`00000000 00000096`98b25c90 : RPCRT4!RPC_INTERFACE::DispatchToStubWorker+0x190
00000096`9953f6d0 00007ff8`14012c86 : 00000000`000618c8 00000096`98b25c90 00000000`00000000 00000096`98b25c90 : RPCRT4!LRPC_SCALL::DispatchRequest+0x4c9
00000096`9953f7e0 00007ff8`14012ad8 : 00000096`00000000 00000096`98b189f0 00000096`98b25c90 00000096`99140000 : RPCRT4!LRPC_SCALL::HandleRequest+0x291
00000096`9953f890 00007ff8`140118bd : 00000096`98b13020 00000096`00000000 00007ff8`1406edbc 00000096`98b13128 : RPCRT4!LRPC_SASSOCIATION::HandleRequest+0x238
00000096`9953f920 00007ff8`140116be : 00000096`98b01088 00000096`98b13128 00000000`00000000 00007ff8`1406edbc : RPCRT4!LRPC_ADDRESS::ProcessIO+0x444
00000096`9953fa60 00007ff8`15e46b30 : 00000096`98af4010 00000096`98b01040 00007ff8`14011600 00000096`9953fb18 : RPCRT4!LrpcIoComplete+0x144
00000096`9953fb00 00007ff8`15e44e68 : 00000000`00000000 00007ff8`15e46900 00000000`00000000 00000000`00000000 : ntdll!TppAlpcpExecuteCallback+0x210
00000096`9953fb70 00007ff8`13c213d2 : 00000000`00000000 00007ff8`15e445e0 00000096`98af4010 00000000`00000000 : ntdll!TppWorkerThread+0x888
00000096`9953ff50 00007ff8`15e7e954 : 00007ff8`13c213b0 00000000`00000000 00000000`00000000 00000000`00000000 : KERNEL32!BaseThreadInitThunk+0x22
00000096`9953ff80 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x34
STACK_COMMAND: kb
FOLLOWUP_IP:
nt! ?? ::NNGAKEGL::`string'+1db76
fffff801`1914ce46 cc int 3
SYMBOL_STACK_INDEX: 1
SYMBOL_NAME: nt! ?? ::NNGAKEGL::`string'+1db76
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: nt
IMAGE_NAME: ntkrnlmp.exe
DEBUG_FLR_IMAGE_TIMESTAMP: 54c832b2
BUCKET_ID_FUNC_OFFSET: 1db76
FAILURE_BUCKET_ID: 0x51_nt!_??_::NNGAKEGL::_string_
BUCKET_ID: 0x51_nt!_??_::NNGAKEGL::_string_
ANALYSIS_SOURCE: KM
FAILURE_ID_HASH_STRING: km:0x51_nt!_??_::nngakegl::_string_
FAILURE_ID_HASH: {e5bde1d4-9584-25e7-f283-f0bca450261c}
We can see the stack:nt!KeBugCheckEx nt! ?? ::NNGAKEGL::`string'+0x1db76 nt!HvFreeCell+0xf7 nt!CmpFreeValueData+0x6a nt!CmpFreeValue+0x26 nt!CmpFreeKeyByCell+0xf9 nt!CmDeleteKey+0x1fd nt!NtDeleteKey+0x25e nt!KiSystemServiceCopyEnd+0x13 ntdll!NtDeleteKey+0xa KERNELBASE!LocalBaseRegDeleteKeyEx+0xf7 KERNELBASE!RegDeleteKeyExInternalW+0xa1 KERNELBASE!RegDeleteTreeW+0x14d defragsvc!SxRegDeleteKey+0x60 defragsvc!CDefragAsyncWorker::_LoadStatisticsFromRegistry+0x6f5 defragsvc!CDefragAsyncWorker::_Initialize+0x19c defragsvc!CDefragAsyncWorker::CreateInstance+0xe6 defragsvc!CDefragEngine::_CreateAsyncForVolume+0x1f5 defragsvc!CDefragEngine::_RefreshVolumeList+0x51b defragsvc!CDefragEngine::FinalConstruct+0x12c defragsvc!CSxComObjectCached<CDefragEngine>::CreateInstance+0xa1 defragsvc!CSxComClassFactorySingleton<CDefragEngine>::CreateInstance+0x7b combase!GetInstanceHelperMulti+0x4c [d:\9147\com\combase\objact\objact.cxx @ 2821] combase!CObjServer::CreateInstance+0x57e [d:\9147\com\combase\objact\defcxact.cxx @ 662] RPCRT4!Invoke+0x73 RPCRT4!NdrStubCall2+0x35e RPCRT4!NdrStubCall3+0x156 combase!CStdStubBuffer_Invoke+0x6b [d:\9147\com\combase\ndr\ndrole\stub.cxx @ 1586] combase!SyncStubInvoke+0x21d [d:\9147\com\combase\dcomrem\channelb.cxx @ 1664] combase!CCtxComChnl::ContextInvoke+0x27a [d:\9147\com\combase\dcomrem\ctxchnl.cxx @ 1377] combase!AppInvoke+0x1af [d:\9147\com\combase\dcomrem\channelb.cxx @ 1481] combase!ComInvokeWithLockAndIPID+0x676 [d:\9147\com\combase\dcomrem\channelb.cxx @ 2314] combase!ThreadInvoke+0x48a [d:\9147\com\combase\dcomrem\channelb.cxx @ 5488] RPCRT4!DispatchToStubInCNoAvrf+0x33 RPCRT4!RPC_INTERFACE::DispatchToStubWorker+0x190 RPCRT4!LRPC_SCALL::DispatchRequest+0x4c9 RPCRT4!LRPC_SCALL::HandleRequest+0x291 RPCRT4!LRPC_SASSOCIATION::HandleRequest+0x238 RPCRT4!LRPC_ADDRESS::ProcessIO+0x444 RPCRT4!LrpcIoComplete+0x144 ntdll!TppAlpcpExecuteCallback+0x210 ntdll!TppWorkerThread+0x888 KERNEL32!BaseThreadInitThunk+0x22 ntdll!RtlUserThreadStart+0x34As we can see the problem occurs when the service defragsvc tries to delete a key. If we see the stack, at fffff801`18fe3463 is the ret addr of the call to the function that calls KeBugCheck, HvpRemoveFreeCellHint:
fffff800`dd7f9451 8b943498000000 mov edx,dword ptr [rsp+rsi+98h] fffff800`dd7f9458 458bcc mov r9d,r12d fffff800`dd7f945b 498bcf mov rcx,r15 fffff800`dd7f945e e895010000 call nt!HvpRemoveFreeCellHint (fffff800`dd7f95f8) fffff800`dd7f9463 4883c604 add rsi,4
We disassemble the function with IDA and we see the exact point where KeBugCheck is called:
HvpRemoveFreeCellHint has these parameters:
VOID
HvpRemoveFreeCellHint(
PHHIVE Hive,
HCELL_INDEX Cell,
ULONG Index,
HSTORAGE_TYPE Type
)
In the stack we see the pointer to the HHIVE is ffffc000`a7277000, and we can explore it:3: kd> dt _HHIVE ffffc000`a7277000
nt!_HHIVE
+0x000 Signature : 0xbee0bee0
+0x008 GetCellRoutine : 0xfffff801`18fb7490 _CELL_DATA* nt!HvpGetCellPaged+0
+0x010 Allocate : 0xfffff801`18fe1d08 void* nt!CmpAllocate+0
+0x018 Free : 0xfffff801`18fe2a44 void nt!CmpFree+0
+0x020 FileWrite : 0xfffff801`190146a4 unsigned char nt!CmpFileWrite+0
+0x028 FileRead : 0xfffff801`19083128 unsigned char nt!CmpFileRead+0
+0x030 HiveLoadFailure : (null)
+0x038 BaseBlock : 0xffffc000`a7279000 _HBASE_BLOCK
+0x040 DirtyVector : _RTL_BITMAP
+0x050 DirtyCount : 0x520
+0x054 DirtyAlloc : 0x52d8
+0x058 UnreconciledVector : _RTL_BITMAP
+0x068 UnreconciledCount : 0
+0x06c BaseBlockAlloc : 0x1000
+0x070 Cluster : 1
+0x074 Flat : 0y0
+0x074 ReadOnly : 0y0
+0x074 Reserved : 0y000000 (0)
+0x075 DirtyFlag : 0x1 ''
+0x078 HvBinHeadersUse : 0x97000
+0x07c HvFreeCellsUse : 0x542620
+0x080 HvUsedCellsUse : 0x4cfe9e0
+0x084 CmUsedCellsUse : 0
+0x088 HiveFlags : 0x10200
+0x08c CurrentLog : 5
+0x090 CurrentLogSequence : 0xb53
+0x094 CurrentLogOffset : 0
+0x098 MinimumLogSequence : 0xb53
+0x09c LogFileSizeCap : 0x2000000
+0x0a0 LogDataPresent : [2] ""
+0x0a2 PrimaryFileValid : 0x1 ''
+0x0a3 BaseBlockDirty : 0 ''
+0x0a4 FirstLogFile : 0y101
+0x0a4 SecondLogFile : 0y000
+0x0a4 HeaderRecovered : 0y0
+0x0a4 LegacyRecoveryIndicated : 0y0
+0x0a4 RecoveryInformationReserved : 0y00000000 (0)
+0x0a4 RecoveryInformation : 5
+0x0a6 LogEntriesRecovered : [2] "."
+0x0a8 RefreshCount : 0
+0x0ac StorageTypeCount : 2
+0x0b0 Version : 5
+0x0b8 Storage : [2] _DUAL
BaseBlock field of the HHIVE points to the regf (the start of the hive). FirstLogFile field is the page of the first hbin block. So BaseBlock + FirstLogFile*0x1000 points to the first hbin block (read the article about registry internals to know about regf and hbin blocks).
We can see in the call to RegDeleteTreeW a pointer as parameter, 00000096`98b2de20, pointing to:
V.o.l.u.m.e.{.9.c.2.f.1.1.c.0.-.1.3.f.6.-.4.2.4.b.-.8.a.1.2.-.8.d.7.0.1.1.e.a.7.6.1.5.}
Now we will search this string from the start of first hbin, and we find it at:
V.o.l.u.m.e.{.9.c.2.f.1.1.c.0.-.1.3.f.6.-.4.2.4.b.-.8.a.1.2.-.8.d.7.0.1.1.e.a.7.6.1.5.}
Now we will search this string from the start of first hbin, and we find it at:
3: kd> s -a ffffc000`a737a000 L10000000 "Volume{9c2f11c0-13f6-424b-8a12-8d7011ea7615}"
ffffc000`a9164590 56 6f 6c 75 6d 65 7b 39-63 32 66 31 31 63 30 2d Volume{9c2f11c0-
ffffc000`af157500 56 6f 6c 75 6d 65 7b 39-63 32 66 31 31 63 30 2d Volume{9c2f11c0-
Some bytes before of the second one string found, we can see the nk cell, exactly at ffffc000`af1574b4 (we see this pointer in the stack near the call to CmpFreeKeyByCell too):ffffc000`af1574b4 nk .l.onXa.......8".....................`.
ffffc000`af1574de G.(...............8...........,...Volume{9
ffffc000`af157508 c2f11c0-13f6-424b-8a12-8d7011ea7615}...0..
It seems when this cell is going to be deleted, BSOD occurs.
nt!KeBugCheckEx nt! ?? ::NNGAKEGL::`string'+0x1db76 nt!HvFreeCell+0xf7 nt!CmpFreeValueData+0x6a nt!CmpFreeValue+0x26 nt!CmpFreeKeyByCell+0xf9 nt!CmDeleteKey+0x1fd nt!NtDeleteKey+0x25e nt!KiSystemServiceCopyEnd+0x13 ntdll!NtDeleteKey+0xa ADVAPI32!LocalBaseRegDeleteKeyEx+0xf2 ADVAPI32!RegDeleteKeyW+0x7f regedit!RegDeleteKeyRecursive+0x13a regedit!RegDeleteKeyRecursive+0x108 regedit!RegDeleteKeyRecursive+0x108 regedit!RegEdit_OnKeyTreeDelete+0xcb regedit!RegEdit_OnKeyTreeCommand+0x115 regedit!RegEdit_OnCommand+0x47 regedit!RegEditWndProc+0x298 USER32!UserCallWinProcCheckWow+0x149 USER32!DispatchClientMessage+0xa2 USER32!_fnDWORD+0x3e ntdll!KiUserCallbackDispatcherContinue USER32!NtUserTranslateAccelerator+0xa regedit!WinMain+0x265 regedit!__mainCRTStartup+0x1ab KERNEL32!BaseThreadInitThunk+0x22 ntdll!RtlUserThreadStart+0x34The registry key seems to be malformed. You can see in the capture some duplicated values: BytesPerCluster, LastRunFullDefragTime, … I guess the registry key became malformed with some sudden reboot (due to power failure). Since the registry became malformed, i had been getting BSOD randomly (between 10 to 30 minutes, each time that defragsvc started to defrag).
We can get the software hive in disk: %systemroot%\system32\config\SOFTWARE (the file is locked but you can copy it with fget.exe tool, of Hbgary) and see the malformed entry there:
I used the SOFTWARE file that we extracted, and i tried to load this hive (with regedit you can do it) under HKEY_LOCAL_MACHINE or HKEY_USERS, on XP, Win7 and Win8 operating systems. But, when you try to delete the malformed key from the new loaded hive, the system doesn’t crash.
I decided to publish the analysis because i found some interesting points, and, maybe, this problem should be fixed because, in spite of the fact this is not a security problem, a system could be totally disabled because of a registry malformation like this or similar.
No comments:
Post a Comment