64-bit windows VMware detection

Question

I am trying to develop an application which detects if program is running inside a virtual machine.

For 32-bit Windows, there are already methods explained in the following link: http://www.codeproject.com/Articles/9823/Detect-if-your-program-is-running-inside-a-Virtual

I am trying to adapt the code regarding Virtual PC and VMware detection in an 64-bit Windows operating system. For VMware, the code can detect successfully in an Windows XP 64-bit OS. But the program crashes when I run it in a native system (Windows 7 64-bit OS).

I put the code in an .asm file and define custom build step with ml64.exe file. The asm code for 64-bit Windows is:

IsInsideVM proc

      push   rdx
      push   rcx
      push   rbx

      mov    rax, 'VMXh'
      mov    rbx, 0     ; any value but not the MAGIC VALUE
      mov    rcx, 10    ; get VMWare version
      mov    rdx, 'VX'  ; port number

      in     rax, dx    ; read port
                        ; on return EAX returns the VERSION
      cmp    rbx, 'VMXh'; is it a reply from VMWare?
      setz   al         ; set return value
      movzx rax,al

      pop    rbx
      pop    rcx
      pop    rdx

      ret
IsInsideVM endp

I call this part in a cpp file like:

__try
{
returnValue = IsInsideVM();
}
__except(1)
{
    returnValue = false;
}

Thanks in advance.

Answer 1

The old red pill from Joanna may work: random backup page of invisiblethings.org blog:

Swallowing the Red Pill is more or less equivalent to the following code (returns non zero when in Matrix):

 int swallow_redpill () {
   unsigned char m[2+4], rpill[] = "\x0f\x01\x0d\x00\x00\x00\x00\xc3";
   *((unsigned*)&rpill[3]) = (unsigned)m;
   ((void(*)())&rpill)();
   return (m[5]>0xd0) ? 1 : 0;
 }

The heart of this code is actually the SIDT instruction (encoded as 0F010D[addr]), which stores the contents of the interrupt descriptor table register (IDTR) in the destination operand, which is actually a memory location. What is special and interesting about SIDT instruction is that, it can be executed in non privileged mode (ring3) but it returns the contents of the sensitive register, used internally by operating system.

Because there is only one IDTR register, but there are at least two OS running concurrently (i.e. the host and the guest OS), VMM needs to relocate the guest's IDTR in a safe place, so that it will not conflict with a host's one. Unfortunately, VMM cannot know if (and when) the process running in guest OS executes SIDT instruction, since it is not privileged (and it doesn't generate exception). Thus the process gets the relocated address of IDT table. It was observed that on VMWare, the relocated address of IDT is at address 0xffXXXXXX, whereas on Virtual PC it is 0xe8XXXXXX. This was tested on VMWare Workstation 4 and Virtual PC 2004, both running on Windows XP host OS.

Note: I haven't tested it myself but look that it uses an unprivileged approach. If it does not work at first for x64, some tweaking may help.

Also, just found out a question with content that may help you: Detecting VMM on linux