Go back

Contents

Hooks detection methods
1. Check whether hooks are set within system functions
2. Check user clicks via mouse hooks
3. Check for incorrectly hooked functions
Signature recommendations
Countermeasures
Credits


Hooks detection methods

Techniques described here make use of hooks either to detect user presence or as means to be checked whether some unusual-for-host-OS hooks installed.


1. Check whether hooks are set within system functions

Malware reads memory at specific addresses to check if Windows API functions are hooked.
This method is based on the fact, that emulation environments are most likely to hook these functions to be able to gather data and statistics during an emulation.


Popular functions to be checked:

  • ReadFile
  • DeleteFile
  • CreateProcessA/W

Reading memory is accomplished via the following functions:

  • ReadProcessMemory
  • NtReadVirtualMemory

Then different algorithms may be used for checking:

  • Comparing first two bytes with \x8B\xFF (mov edi, edi) — typical prologue start for kernel32 functions.
  • Comparing first N bytes with \xCC - software breakpoint (int 3), not connected with hooks directly but still a suspicious behavior.
  • Comparing first N bytes with \xE9 (call) or with \xEB (jmp instruction) — typical instructions for redirecting execution.
  • Checking for push/ret combo for execution redirection.

and so on.


It’s pretty tricky to count for every possible comparison so general indication of something unusual in application’s behavior is reading memory where OS libraries reside. If to be more precise: reading memory where “interesting” functions are situated.


This atricle explains how to detect user-mode hooks and remove them. The following code samples are taken from the article.


Example of hook detection

HOOK_TYPE IsHooked(LPCVOID lpFuncAddress, DWORD_PTR *dwAddressOffset) {
    LPCBYTE lpBytePtr = (LPCBYTE)lpFuncAddress;

    if (lpBytePtr[0] == 0xE9) {
        *dwAddressOffset = 1;
        return HOOK_RELATIVE;    // E9 jmp is relative.
    } else if (lpBytePtr[0] == 0x68 &&  lpBytePtr[5] == 0xC3) {
        *dwAddressOffset = 1;
        return HOOK_ABOLSUTE;    // push/ret is absolute.
    }

    return HOOK_NONE;            // No hook.
}

LPVOID lpFunction = ...;
DWORD_PTR dwOffset = 0;
LPVOID dwHookAddress = 0;

HOOK_TYPE ht = IsHooked(lpFunction, &dwOffset);
if (ht == HOOK_ABSOLUTE) {
    // 1. Get the pointer to the address (lpFunction + dwOffset)
    // 2. Cast it to a DWORD pointer
    // 3. Dereference it to get the DWORD value
    // 4. Cast it to a pointer
    dwHookAddress = (LPVOID)(*(LPDWORD)((LPBYTE)lpFunction + dwOffset));
} else if (ht == HOOK_RELATIVE) {
    // 1. Get the pointer to the address (lpFunction + dwOffset)
    // 2. Cast it to an INT pointer
    // 3. Dereference it to get the INT value (this can be negative)
    INT nJumpSize = (*(PINT)((LPBYTE)lpFunction  + dwOffset);
    // 4. E9 jmp starts from the address AFTER the jmp instruction
    DWORD_PTR dwRelativeAddress = (DWORD_PTR)((LPBYTE)lpFunction + dwOffset + 4));
    // 5. Add the relative address and jump size
    dwHookAddress = (LPVOID)(dwRelativeAddress + nJumpSize);
}

Example of unhooking functions

// Parse the PE headers.
PIMAGE_DOS_HEADER pidh = (PIMAGE_DOS_HEADER)lpMapping;
PIMAGE_NT_HEADERS pinh = (PIMAGE_NT_HEADERS)((DWORD_PTR)lpMapping + pidh->e_lfanew);

// Walk the section headers and find the .text section.
for (WORD i = 0; i < pinh->FileHeader.NumberOfSections; i++) {
    PIMAGE_SECTION_HEADER pish = (PIMAGE_SECTION_HEADER)((DWORD_PTR)IMAGE_FIRST_SECTION(pinh) + 
                                 ((DWORD_PTR)IMAGE_SIZEOF_SECTION_HEADER * i));
    if (!strcmp(pish->Name, ".text")) {
        // Deprotect the module's memory region for write permissions.
        DWORD flProtect = ProtectMemory(
            (LPVOID)((DWORD_PTR)hModule + (DWORD_PTR)pish->VirtualAddress),    // Address to protect.
            pish->Misc.VirtualSize,                        // Size to protect.
            PAGE_EXECUTE_READWRITE                         // Desired protection.
        );

        // Replace the hooked module's .text section with the newly mapped module's.
        memcpy(
            (LPVOID)((DWORD_PTR)hModule + (DWORD_PTR)pish->VirtualAddress),
            (LPVOID)((DWORD_PTR)lpMapping + (DWORD_PTR)pish->VirtualAddress),
            pish->Misc.VirtualSize
        );

        // Reprotect the module's memory region.
        flProtect = ProtectMemory(
            (LPVOID)((DWORD_PTR)hModule + (DWORD_PTR)pish->VirtualAddress),    // Address to protect.
            pish->Misc.VirtualSize,                        // Size to protect.
            flProtect                                      // Revert to old protection.
        );
    }
}


2. Check user clicks via mouse hooks

This technique is described by this link (p.4, p.7).


Malware sets mouse hook to detect a click (or more) if it occurs. If it’s the case malware treats the host a usual one, i.e., with end user behind the screen - not a virtual environment. If no mouse click is detected then it’s very likely a virtual environment.


Functions used:

  • SetWindowsHookExA/W (WH_MOUSE_LL, ...)
  • GetAsyncKeyState

Code sample (SetWindowsHookExA)

HHOOK g_hhkMouseHook = NULL;

LRESULT CALLBACK mouseHookProc(int nCode, WPARAM wParam, LPARAM lParam)
{
  switch (wParam)
  {
  case WM_MOUSEMOVE:
    // ...
    break;
  case WM_NCLBUTTONDOWN:
    // ...
    break;
  case WM_LBUTTONUP:
    UnhookWindowsHookEx(g_hhkMouseHook);
    CallMaliciousCode();
    ExitProcess(0);
  }
  return CallNextHookEx(g_hhkMouseHook, nCode, wParam, lParam);
}

g_hhkMouseHook = SetWindowsHookEx(WH_MOUSE_LL, mouseHookProc, GetModuleHandleA(NULL), NULL);

Code sample (GetAsyncKeyState)

std::thread t([]()
{
  int count = 0;
  while (true)
  {
    if (GetAsyncKeyState(VK_LBUTTON) || GetAsyncKeyState(VK_RBUTTON) || GetAsyncKeyState(VK_MBUTTON))
    {
      if (++count == 2)
        break;
    }
    Sleep(100);
  }
  CallMaliciousCode();
});
t.join();


3. Check for incorrectly hooked functions

There are more than 400 Native API functions (or Nt-functions) in ntdll.dll that are usually hooked in sandboxes. In such a large list, there is enough space for different kinds of mistakes. We checked the hooked Nt-functions in popular sandboxes and found several issues. On of them is a lack of necessary checks for arguments in a hooked function. This case is described our article “Timing: Call a potentially hooked delay function with invalid arguments evasions
Another issue we found is a discrepancy in the number of arguments in a hooked and an original function. If a function is hooked incorrectly, in kernel mode this may lead an operating system to crash. Incorrect user-mode hooks are not as critical. However, they may lead an analyzed application to crash or can be easily detected. For example, let’s look at the NtLoadKeyEx function. It was first introduced in Windows Server 2003 and had only 4 arguments. Starting from Windows Vista up to the latest version of Windows 10, it has 8 arguments:

; Exported entry 318. NtLoadKeyEx
; Exported entry 1450. ZwLoadKeyEx
; __stdcall NtLoadKeyEx(x, x, x, x, x, x, x, x)
public _NtLoadKeyEx@32

However, in the Cuckoo monitor, the NtLoadKeyEx declaration still has only 4 arguments:

*  POBJECT_ATTRIBUTES TargetKey
*  POBJECT_ATTRIBUTES SourceFile
** ULONG Flags flags
** HANDLE TrustClassKey trust_class_key

We found this legacy prototype used in other sources as well. For example, CAPE monitor has the same issue:

extern HOOKDEF(NTSTATUS, WINAPI, NtLoadKeyEx,
    __in      POBJECT_ATTRIBUTES TargetKey,
    __in      POBJECT_ATTRIBUTES SourceFile,
    __in      ULONG Flags,
    __in_opt  HANDLE TrustClassKey
);

Therefore, if a sandbox uses any recent Windows OS, this function is hooked incorrectly. After the call to the incorrectly hooked function, the stack pointer value becomes invalid. Therefore, a totally “legitimate” call to the RegLoadAppKeyW function, which calls NtLoadKeyEx, leads to an exception. This fact can be used to evade Cuckoo and CAPE sandbox with just a single call to the RegLoadAppKeyW function.

Code sample

RegLoadAppKeyW(L"storage.dat", &hKey, KEY_ALL_ACCESS, 0, 0);
// If the application is running in a sandbox an exception will occur
// and the code below will not be executed.

// Some legitimate code that works with hKey to distract attention goes here
// ...
RegCloseKey(hKey);
// Malicious code goes here
// ...

Instead of using RegLoadAppKeyW, we can call the NtLoadKeyEx function directly and check the ESP value after the call.

Code sample

__try
{
    _asm mov old_esp, esp
    NtLoadKeyEx(&TargetKey, &SourceFile, 0, 0, 0, KEY_ALL_ACCESS, &hKey, &ioStatus);
    _asm mov new_esp, esp
    _asm mov esp, old_esp
    if (old_esp != new_esp)
        printf("Sandbox detected!");
}
__except (EXCEPTION_EXECUTE_HANDLER)
{
    printf("Sandbox detected!");
}


Signature recommendations

No signature recommendations are provided for this evasion group as it’s hard to make a difference between the code which aims for some evasion technique and the one which is “legally used”.


Countermeasures

  • versus function hook checks: set kernel mode hooks; second solution is to use stack routing to implement function hooking;
  • versus mouse click checks via hooks: use mouse movement emulation module.
  • versus incorrect function hooks: ensure all the hooked function have the same number of arguments as the original functions


Credits

Credits go to user dtm from 0x00sec.org forum.

Due to modular code structure of the Check Point’s tool called InviZzzible it would require more space to show a code sample from this tool for the same purposes. That’s why we’ve decided to use other great open-source projects for examples throughout the encyclopedia.


Go back