Synthetic symbols and modules (WinDbg / DbgEng)

In this publication, we discuss the synthetic modules and symbols of the Windows debugging engine (debugger engine). That is, about entities that can be artificially added to the debugger for coloring memory addresses.

Synthetic modules

In general, a module within the debugging engine (debugger engine) is a certain continuous area of ​​virtual memory: the base address of the module (the address of the first byte of the projected file) and its size. For each type of debugging (live debugging / dump analysis, user mode / kernel mode, etc.), the debugger has its own mechanism for reading and keeping up to date the list of loaded modules. The easiest way to force the updating of the list of loaded modules is with the .reload command with the / s parameter .

If we talk, for example, about live debugging of the user-mode process, the list of loaded modules is based on three main loader lists : the module loading order list (InLoadOrderModuleList), the list of modules in memory (InMemoryOrderModuleList) and the initialization order list (InInitializationOrderModuleList). On the one hand, we ( almost ) do not interfere with taking arbitrary data (from a PE file on a disk, for example) and manually marking it in memory for execution. On the other hand, the removal techniques of a DLL loaded with regular tools from the above-mentioned three bootloader lists (hiding a module) have long been known.

In both cases, when debugging such a situation, it will be useful to be able to mark the area of ​​the hidden module. For this fit synthetic modules. As a practical experiment, you can simply force WinDbg to discard the loaded module from its internal list with the same .reload command , but with the / u parameter .

Next, as the listed listings, I will use the usual debugging process of the user mode (notepad.exe). First, let's remember the base address of the ntdll module (0x07ffa8a160000) and calculate its size (0x01e1000):

0:007> lm m ntdll Browse full module list start end module name 00007ffa`8a160000 00007ffa`8a341000 ntdll 0:007> ? 00007ffa`8a341000-00007ffa`8a160000 Evaluate expression: 1970176 = 00000000`001e1000 

Consider the list of the simple function RtlFreeSid from the ntdll module, which calls the function RtlFreeHeap from this module, simultaneously remembering the addresses of the characters RtlFreeSid and RtlFreeHeap (0x7ffa8a1cbc20 and 0x7ffa8a176df0):

 0:007> uf ntdll!RtlFreeSid ntdll!RtlFreeSid: 00007ffa`8a1cbc20 4053 push rbx 00007ffa`8a1cbc22 4883ec20 sub rsp,20h 00007ffa`8a1cbc26 488bd9 mov rbx,rcx 00007ffa`8a1cbc29 33d2 xor edx,edx 00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h] 00007ffa`8a1cbc34 4c8bc3 mov r8,rbx 00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h] 00007ffa`8a1cbc3b e8b0b1faff call ntdll!RtlFreeHeap (00007ffa`8a176df0) 00007ffa`8a1cbc40 33c9 xor ecx,ecx 00007ffa`8a1cbc42 85c0 test eax,eax 00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx 00007ffa`8a1cbc48 488bc3 mov rax,rbx 00007ffa`8a1cbc4b 4883c420 add rsp,20h 00007ffa`8a1cbc4f 5b pop rbx 00007ffa`8a1cbc50 c3 ret 

Also, according to the above listing, it is easy to calculate the size of the function ntdll! RtlFreeSid:

 0:007> ? 00007ffa`8a1cbc51 - 0x07ffa8a1cbc20 Evaluate expression: 49 = 00000000`00000031 

And the size of the function ntdll! RtlFreeHeap is not important for our experiment, so for simplicity we can take it equal to one byte.

Now simulate hiding the ntdll module:

 0:007> .reload /u ntdll Unloaded ntdll 

The field of this work with the address of the function ntdll! RtlFreeSid in the debugger is not so informative (and you can already access the beginning of the function only at the virtual address):

 0:007> uf 00007ffa`8a1cbc20 00007ffa`8a1cbc20 4053 push rbx 00007ffa`8a1cbc22 4883ec20 sub rsp,20h 00007ffa`8a1cbc26 488bd9 mov rbx,rcx 00007ffa`8a1cbc29 33d2 xor edx,edx 00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h] 00007ffa`8a1cbc34 4c8bc3 mov r8,rbx 00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h] 00007ffa`8a1cbc3b e8b0b1faff call 00007ffa`8a176df0 00007ffa`8a1cbc40 33c9 xor ecx,ecx 00007ffa`8a1cbc42 85c0 test eax,eax 00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx 00007ffa`8a1cbc48 488bc3 mov rax,rbx 00007ffa`8a1cbc4b 4883c420 add rsp,20h 00007ffa`8a1cbc4f 5b pop rbx 00007ffa`8a1cbc50 c3 ret 

In order to add a synthetic module, you need to call the IDebugSymbols3 :: AddSyntheticModule program interface. There are simply no built-in commands that would allow you to make this call (to my knowledge). Mikhail I. Izmestev suggested that there is a similar mechanism - the same .reload , but with the name, address and size parameters: "Module = Address, Size" (and, perhaps, it is implemented on synthetic modules).

 0:007> .reload ntdll=00007ff8`470e1000,001e1000 *** WARNING: Unable to verify timestamp for ntdll *** ERROR: Module load completed but symbols could not be loaded for ntdll 0:007> uf 00007ff8`4714bc20 ntdll+0x6ac20: 00007ff8`4714bc20 4053 push rbx 00007ff8`4714bc22 4883ec20 sub rsp,20h 00007ff8`4714bc26 488bd9 mov rbx,rcx 00007ff8`4714bc29 33d2 xor edx,edx 00007ff8`4714bc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h] 00007ff8`4714bc34 4c8bc3 mov r8,rbx 00007ff8`4714bc37 488b4930 mov rcx,qword ptr [rcx+30h] 00007ff8`4714bc3b e8b0b1faff call ntdll+0x15df0 (00007ff8`470f6df0) 00007ff8`4714bc40 33c9 xor ecx,ecx 00007ff8`4714bc42 85c0 test eax,eax 00007ff8`4714bc44 480f45d9 cmovne rbx,rcx 00007ff8`4714bc48 488bc3 mov rax,rbx 00007ff8`4714bc4b 4883c420 add rsp,20h 00007ff8`4714bc4f 5b pop rbx 00007ff8`4714bc50 c3 ret 

Well, we'll use the debugger extension, and then pykd will come to the rescue . In the most recent (at the time of writing) release 0.3.4.3 , functions for managing synthetic modules were added: addSyntheticModule (...) (which is needed in our case) and removeSyntheticModule (...).

According to the previously memorized base address and module size, we add information about the hidden ntdll module to the debugger (if an error occurs, an exception will be thrown, so silent execution is a sign of success, printing warnings can be ignored):

 0:007> !py Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>> addSyntheticModule(0x07ffa8a160000, 0x01e1000, 'ntdll') *** WARNING: Unable to verify timestamp for ntdll >>> exit() 

Now the disassembler listing has become a bit more informative:

 0:007> uf 00007ffa`8a1cbc20 ntdll+0x6bc20: 00007ffa`8a1cbc20 4053 push rbx 00007ffa`8a1cbc22 4883ec20 sub rsp,20h 00007ffa`8a1cbc26 488bd9 mov rbx,rcx 00007ffa`8a1cbc29 33d2 xor edx,edx 00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h] 00007ffa`8a1cbc34 4c8bc3 mov r8,rbx 00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h] 00007ffa`8a1cbc3b e8b0b1faff call ntdll+0x16df0 (00007ffa`8a176df0) 00007ffa`8a1cbc40 33c9 xor ecx,ecx 00007ffa`8a1cbc42 85c0 test eax,eax 00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx 00007ffa`8a1cbc48 488bc3 mov rax,rbx 00007ffa`8a1cbc4b 4883c420 add rsp,20h 00007ffa`8a1cbc4f 5b pop rbx 00007ffa`8a1cbc50 c3 ret 

Or rather, we see that a function inside the ntdll module calls another function inside the same module.

Synthetic characters

Listing, after adding a synthetic module, has become more informative, but still not as eloquent as the original one. It lacks characters (in our case, the names of the functions RtlFreeSid and RtlFreeHeap). To fix this again, a call is required, but another software interface - IDebugSymbols3 :: AddSyntheticSymbol . Pykd again ready to help us with this:

 0:007> !py Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>> addSyntheticSymbol(0x07ffa8a1cbc20, 0x31, 'RtlFreeSid') <pykd.syntheticSymbol object at 0x0000014D47699518> >>> addSyntheticSymbol(0x07ffa8a176df0, 1, 'RtlFreeHeap') <pykd.syntheticSymbol object at 0x0000014D476994A8> >>> exit() 

The result is very similar to that when using symbols from the pdb file ntdll:

 0:007> uf 00007ffa`8a1cbc20 ntdll!RtlFreeSid: 00007ffa`8a1cbc20 4053 push rbx 00007ffa`8a1cbc22 4883ec20 sub rsp,20h 00007ffa`8a1cbc26 488bd9 mov rbx,rcx 00007ffa`8a1cbc29 33d2 xor edx,edx 00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h] 00007ffa`8a1cbc34 4c8bc3 mov r8,rbx 00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h] 00007ffa`8a1cbc3b e8b0b1faff call ntdll!RtlFreeHeap (00007ffa`8a176df0) 00007ffa`8a1cbc40 33c9 xor ecx,ecx 00007ffa`8a1cbc42 85c0 test eax,eax 00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx 00007ffa`8a1cbc48 488bc3 mov rax,rbx 00007ffa`8a1cbc4b 4883c420 add rsp,20h 00007ffa`8a1cbc4f 5b pop rbx 00007ffa`8a1cbc50 c3 ret 

Features of synthetic entities

It should be noted that the creation of synthetic symbols is allowed in any existing modules, and not only in synthetic ones. That is, you can add symbols both to modules with accessible pdb files , and to modules for which we do not have debug information files. But characters outside modules cannot be created. That is, to mark an arbitrary memory address outside the boundaries of existing modules, you need to create an embedding synthetic module at the beginning, and then (if necessary, the module name may be enough for coloring ) to create a synthetic symbol in it.

It is also useful to mention that synthetic modules and symbols can be removed by careless hand movement:

• reloading the module debugging symbols removes all the synthetic symbols of this module;
• rebooting all modules removes all synthetic modules.

! synexts

Although the use of pykd is convenient in the overwhelming majority of cases (especially given the presence of a bootstarpper to it), sometimes you can get into a situation where you need to spend considerable power to use pykd. For example, I once needed to debug with the host machine, which was running Windows XP. As it turned out, for a long time now, pykd does not support XP, and I needed synthetic symbols and modules. It seemed to me that for this task it is easier to assemble a separate small extension that will solve a narrow circle of necessary tasks than to restore full XP support for pykd. As a result, a separate ! Synexts project was created.

This is a simple extension that has two exports available to the user:

• ! synexts.addsymbol - creates a synthetic symbol in any existing module;
• ! synexts.addmodule - creates a synthetic module in the internal list of the debug engine.

To demonstrate, we again imitate the hiding of the ntdll module:

 0:007> .reload /u ntdll Unloaded ntdll 0:007> uf 00007ffa`8a1cbc20 00007ffa`8a1cbc20 4053 push rbx 00007ffa`8a1cbc22 4883ec20 sub rsp,20h 00007ffa`8a1cbc26 488bd9 mov rbx,rcx 00007ffa`8a1cbc29 33d2 xor edx,edx 00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h] 00007ffa`8a1cbc34 4c8bc3 mov r8,rbx 00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h] 00007ffa`8a1cbc3b e8b0b1faff call 00007ffa`8a176df0 00007ffa`8a1cbc40 33c9 xor ecx,ecx 00007ffa`8a1cbc42 85c0 test eax,eax 00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx 00007ffa`8a1cbc48 488bc3 mov rax,rbx 00007ffa`8a1cbc4b 4883c420 add rsp,20h 00007ffa`8a1cbc4f 5b pop rbx 00007ffa`8a1cbc50 c3 ret 

Again, we restore knowledge about the module and the characters in the form of synthetic entities, but using the! Synexts extension:

 0:007> !synexts.addmodule ntdll C:\Windows\System32\ntdll.dll 00007ffa`8a160000 0x01e1000 *** WARNING: Unable to verify timestamp for C:\Windows\System32\ntdll.dll Synthetic module successfully added 0:007> !synexts.addsymbol RtlFreeSid 00007ffa`8a1cbc20 31 Synthetic symbol successfully added 0:007> !synexts.addsymbol RtlFreeHeap 00007ffa`8a176df0 1 Synthetic symbol successfully added 

It is assumed that the compiled synexts.dll library (corresponding to the bit width used by WinDbg) you have already copied to the winext directory of the debugger:

 0:007> .chain Extension DLL search Path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\WINXP;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\arcade;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\pri;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64;<…> Extension DLL chain: pykd.pyd: image 0.3.4.3, API 1.0.0, built Thu Jan 10 19:56:25 2019 [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\pykd.pyd] synexts: API 1.0.0, built Fri Jan 18 17:38:17 2019 [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\synexts.dll] <…> 

And again we see the result of adding synthetic symbols to the synthetic module:

 0:007> uf 00007ffa`8a1cbc20 ntdll!RtlFreeSid: 00007ffa`8a1cbc20 4053 push rbx 00007ffa`8a1cbc22 4883ec20 sub rsp,20h 00007ffa`8a1cbc26 488bd9 mov rbx,rcx 00007ffa`8a1cbc29 33d2 xor edx,edx 00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h] 00007ffa`8a1cbc34 4c8bc3 mov r8,rbx 00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h] 00007ffa`8a1cbc3b e8b0b1faff call ntdll!RtlFreeHeap (00007ffa`8a176df0) 00007ffa`8a1cbc40 33c9 xor ecx,ecx 00007ffa`8a1cbc42 85c0 test eax,eax 00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx 00007ffa`8a1cbc48 488bc3 mov rax,rbx 00007ffa`8a1cbc4b 4883c420 add rsp,20h 00007ffa`8a1cbc4f 5b pop rbx 00007ffa`8a1cbc50 c3 ret