Back to main page

Contents

Hooks detection methods
1. Check whether hooks are set within system functions
2. Check user clicks via mouse hooks
Signature recommendations
Countermeasures
Credits


Hooks detection methods

Techniques described here make use of hooks either to detect user presence or as means to be checked whether some unusual-for-host-OS hooks installed.


1. Check whether hooks are set within system functions

Malware reads memory at specific addresses to check if the Windows API functions are hooked.
This method is based on the fact, that emulation environments are most likely to hook these functions to be able to gather data and statistics during emulation.


Popular functions to be checked:

  • ReadFile
  • DeleteFile
  • CreateProcessA/W

Reading memory is accomplished via the following functions:

  • ReadProcessMemory
  • NtReadVirtualMemory

Then different algorithms may be used for checking:

  • Comparing first two bytes with \x8B\xFF (mov edi, edi) — typical prologue start for kernel32 functions.
  • Comparing first N bytes with \xCC - software breakpoint (int 3), not connected with hooks directly but still a suspicious behavior.
  • Comparing first N bytes with \xE9 (call) or with \xEB (jmp instruction) — typical instructions for redirecting execution.
  • Checking for push/ret combo for execution redirection.

and so on.


It’s pretty tricky to count for every possible comparison so general indication of something unusual in application’s behavior is reading memory where OS libraries reside. If to be more precise: reading memory where “interesting” functions are situated.


This atricle explains how to detect user-mode hooks and remove them. The following code samples are taken from the article.


Example of hook detection

HOOK_TYPE IsHooked(LPCVOID lpFuncAddress, DWORD_PTR *dwAddressOffset) {
    LPCBYTE lpBytePtr = (LPCBYTE)lpFuncAddress;

    if (lpBytePtr[0] == 0xE9) {
        *dwAddressOffset = 1;
        return HOOK_RELATIVE;    // E9 jmp is relative.
    } else if (lpBytePtr[0] == 0x68 &&  lpBytePtr[5] == 0xC3) {
        *dwAddressOffset = 1;
        return HOOK_ABOLSUTE;    // push/ret is absolute.
    }

    return HOOK_NONE;            // No hook.
}

LPVOID lpFunction = ...;
DWORD_PTR dwOffset = 0;
LPVOID dwHookAddress = 0;

HOOK_TYPE ht = IsHooked(lpFunction, &dwOffset);
if (ht == HOOK_ABSOLUTE) {
    // 1. Get the pointer to the address (lpFunction + dwOffset)
    // 2. Cast it to a DWORD pointer
    // 3. Dereference it to get the DWORD value
    // 4. Cast it to a pointer
    dwHookAddress = (LPVOID)(*(LPDWORD)((LPBYTE)lpFunction + dwOffset));
} else if (ht == HOOK_RELATIVE) {
    // 1. Get the pointer to the address (lpFunction + dwOffset)
    // 2. Cast it to an INT pointer
    // 3. Dereference it to get the INT value (this can be negative)
    INT nJumpSize = (*(PINT)((LPBYTE)lpFunction  + dwOffset);
    // 4. E9 jmp starts from the address AFTER the jmp instruction
    DWORD_PTR dwRelativeAddress = (DWORD_PTR)((LPBYTE)lpFunction + dwOffset + 4));
    // 5. Add the relative address and jump size
    dwHookAddress = (LPVOID)(dwRelativeAddress + nJumpSize);
}

Example of unhooking functions

// Parse the PE headers.
PIMAGE_DOS_HEADER pidh = (PIMAGE_DOS_HEADER)lpMapping;
PIMAGE_NT_HEADERS pinh = (PIMAGE_NT_HEADERS)((DWORD_PTR)lpMapping + pidh->e_lfanew);

// Walk the section headers and find the .text section.
for (WORD i = 0; i < pinh->FileHeader.NumberOfSections; i++) {
    PIMAGE_SECTION_HEADER pish = (PIMAGE_SECTION_HEADER)((DWORD_PTR)IMAGE_FIRST_SECTION(pinh) + 
                                 ((DWORD_PTR)IMAGE_SIZEOF_SECTION_HEADER * i));
    if (!strcmp(pish->Name, ".text")) {
        // Deprotect the module's memory region for write permissions.
        DWORD flProtect = ProtectMemory(
            (LPVOID)((DWORD_PTR)hModule + (DWORD_PTR)pish->VirtualAddress),    // Address to protect.
            pish->Misc.VirtualSize,                        // Size to protect.
            PAGE_EXECUTE_READWRITE                         // Desired protection.
        );

        // Replace the hooked module's .text section with the newly mapped module's.
        memcpy(
            (LPVOID)((DWORD_PTR)hModule + (DWORD_PTR)pish->VirtualAddress),
            (LPVOID)((DWORD_PTR)lpMapping + (DWORD_PTR)pish->VirtualAddress),
            pish->Misc.VirtualSize
        );

        // Reprotect the module's memory region.
        flProtect = ProtectMemory(
            (LPVOID)((DWORD_PTR)hModule + (DWORD_PTR)pish->VirtualAddress),    // Address to protect.
            pish->Misc.VirtualSize,                        // Size to protect.
            flProtect                                      // Revert to old protection.
        );
    }
}


2. Check user clicks via mouse hooks

This technique is described by this link (p.4, p.7).


Malware sets mouse hook to detect a click (or more) if it occurs. If it’s the case malware treats the host a usual one, i.e., with end user behind the screen - not a virtual environment. If no mouse click is detected then it’s very likely a virtual environment.


Functions used:

  • SetWindowsHookExA/W (WH_MOUSE_LL, ...)
  • GetAsyncKeyState

Code sample (SetWindowsHookExA)

HHOOK g_hhkMouseHook = NULL;

LRESULT CALLBACK mouseHookProc(int nCode, WPARAM wParam, LPARAM lParam)
{
  switch (wParam)
  {
  case WM_MOUSEMOVE:
    // ...
    break;
  case WM_NCLBUTTONDOWN:
    // ...
    break;
  case WM_LBUTTONUP:
    UnhookWindowsHookEx(g_hhkMouseHook);
    CallMaliciousCode();
    ExitProcess(0);
  }
  return CallNextHookEx(g_hhkMouseHook, nCode, wParam, lParam);
}

g_hhkMouseHook = SetWindowsHookEx(WH_MOUSE_LL, mouseHookProc, GetModuleHandleA(NULL), NULL);

Code sample (GetAsyncKeyState)

std::thread t([]()
{
  int count = 0;
  while (true)
  {
    if (GetAsyncKeyState(VK_LBUTTON) || GetAsyncKeyState(VK_RBUTTON) || GetAsyncKeyState(VK_MBUTTON))
    {
      if (++count == 2)
        break;
    }
    Sleep(100);
  }
  CallMaliciousCode();
});
t.join();


Signature recommendations

No signature recommendations are provided for this evasion group as it’s hard to make a difference between the code which aims for some evasion technique and the one which is “legally used”.


Countermeasures

  • versus function hook checks: set kernel mode hooks; second solution is to use stack routing to implement function hooking;
  • versus mouse click checks via hooks: use mouse movement emulation module.


Credits

Credits go to user dtm from 0x00sec.org forum.

Though Check Point tool InviZzzible has them all implemented, due to modular structure of the code it would require more space to show a code sample from this tool for the same purposes. That’s why we’ve decided to use other great open-source projects for examples throughout the encyclopedia.


Back to main page