Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Reverse-engineering a virtual method that only XORs the low byte of EAX before returning

I need help with reverse engineering a virtual method from disassembly. The code was originally compiled with Microsoft's Visual C++. The method in question is as follows:

sub_92D110    proc near
xor al, al
retn
sub_92d110    endp

This method is referenced between a lot of classes, even multiple times inside of a vtable of one class. I am not sure what it does; does it mean the method got inlined but the call remains so that the vtables retain their size?

And if so, what does xor al, al do? Am I misunderstanding a calling convention or something?

like image 759
Artemoire Avatar asked Dec 19 '22 05:12

Artemoire


1 Answers

This is most probably something like:

bool someclass::somemethod() {
    return false;
} 

Explanation

  • xor al,al sets the low byte of eax to zero.
  • All x86 calling conventions use eax as the "return value" register for register-sized integer values.
  • This cannot be a function returning an int (as in return 0;), given that it cleans only the low byte (and no x86 calling convention uses eax as an input parameter, so it's not some bizarre function that takes an integer argument, zeroes its low byte and returns it).
  • This leaves us with a function that returns a byte-sized value, set to zero, so it could either return a char/unsigned char (0) or a bool (false); I'm way more inclined to think that it's the second option, given that in practice it arises way more often (especially in "empty" base class implementations of methods possibly redefined by derived classes).
  • It either takes no parameters or is a variadic function that doesn't look at any of its parameters. C++ methods on VC++/x86 employ the __thiscall calling convention, which, besides putting the this pointer in ecx, is the same as __stdcall for "regular" methods, and same as __cdecl for variadics; now, __stdcall is callee cleanup, and here there's no cleanup to speak of, which would mean no arguments; on the other hand, there's no cleanup in the called method even in a __cdecl function, so we cannot rule out this possibility a priori. That being said, I don't think that this last option is likely.

This method is referenced between a lot of classes, even multiple times inside of a vtable of one class.

It's perfectly normal; VC++'s linker regularly merges unrelated functions that compile to the same machine code (and confusingly calls this "identical COMDAT folding").

Given that this process is very low level (it essentially looks at the bytes generated for the various functions and sees if they can be de-duplicated), in theory all the hypotheses above may hold together—it may be a method taking no arguments and returning a bool false in one vtable slot and a varargs method returning a char zero in another one.

like image 52
Matteo Italia Avatar answered Dec 24 '22 00:12

Matteo Italia