The ultimate goal is comparing 2 binaries built from exact same source in exact same environment and being able to tell that they indeed are functionally equivalent.
One application for this would be focusing QA time on things that were actually changed between releases, as well as change monitoring in general.
MSVC in tandem with PE format naturally makes this very hard to do.
So far I found and neutralized those things:
I parse PE, find offsets and sizes for all those things and ignore byte ranges when comparing binaries. Works like charm (well, for the few tests I've run it). I can tell that signed executable with version 1.0.2.0 built on Win Server 2008 is equal to unsigned one, of version 10.6.6.6, build on my Win XP dev box, as long as compiler version and all sources and headers are the same. This seems to work for VC 7.1 -- 9.0. (For release builds)
With one caveat.
Absolute paths for both builds must be the same must have the same length.
cl.exe converts relative paths to absolute ones, and puts them right into objects along with compiler flags and so on. This has unproportional effects on whole binary. One character change in path will result in one byte changed here and there several times over whole .text section (however many objects were linked I suspect). Changing length of the path results in significantly more differences. Both in obj files and in linked binary.
Feels like file path with compile flags is used as some kind of hash, which makes it into linked binary or even affects placement order of unrelated pieces of compiled code.
So here is the 3-part question (summarized as "what now?"):
Should I abandon the whole project and go home because what I am trying to do breaks laws of physics and corporate policy of MS?
Assuming I handle absolute path issue (on policy level or by finding a magical compiler flag), are there any other things I should look out for? (things like __TIME__ do mean changed code, so I don't mind those not being ignored)
Is there a way to either force compiler to use relative paths, or to fool it into thinking the path is not what it is?
Reason for the last one is beautifully annoying Windows file system. You just never know when deleting several gigs worth of sources and objects and svn metadata will fail because of a rogue file lock. At least creating new root always succeeds while there is space left. Running multiple builds at once is an issue too. Running bunch of VMs, while a solution, is a rather heavy one.
I wonder if there is a way to setup a virtual file system for a process and its children so that several process trees will see different "C:\build" dirs, private to them only, all at the same time... A light-weight virtualization of sorts...
UPDATE: we recently opensourced the tool on GitHub. See Compare section in documentation.
Although the definitive answer is "it depends", it is reasonable to expect that most compilers will be deterministic most of the time, and that the binaries produced should be identical. Indeed, some version control systems depend on this.
Open a developer command prompt If you have installed Microsoft Visual C++ Build Tools 2015 on Windows 10 or later, open the Start menu and choose All apps. Scroll down and open the Visual C++ Build Tools folder. Choose Visual C++ 2015 x86 Native Tools Command Prompt to open the command prompt window.
C++ compiler and linker options are located under the C/C++ and Linker nodes in the left pane under Configuration Properties. These options translate directly to command-line options that will be passed to the compiler. To read documentation about a specific option, select the option in the center pane and press F1.
Check your Microsoft Visual C++ installation# To open the Developer Command Prompt for VS, start typing 'developer' in the Windows Start menu, and you should see it appear in the list of suggestions. The exact name depends on which version of Visual Studio or the Visual Studio Build Tools you have installed.
I solved this to an extent.
Currently we have build system that makes sure all new builds are on the path of constant length (builds/001, builds/002, etc), thus avoiding shifts in the PE layout. After build a tool compares old and new binaries ignoring relevant PE fields and other locations with known superficial changes. It also runs some simple heuristics to detect dynamic ignorable changes. Here is full list of things to ignore:
Once in a while linker would make some PE sections bigger without throwing anything else out of alignment. Looks like it moves section boundary inside the padding -- it is zeros all around anyway, but because of it I'll get binaries with 1 byte difference.
UPDATE: we recently opensourced the tool on GitHub. See Compare section in documentation.
A simple solution would be to standardise on your build paths, so they are always of the form, for example:
c:\buildXXXX
Then, when you compare, say, build0434 to build0398, just preprocess the binary to change all occurrences of build0434 to build0398. Choose a pattern you know is unlikely to show up in your actual source/data, except in those strings the compiler/linker embed into the PE.
Then you can just do your normal difference analysis. By using the same length pathnames, you won't shift any data around and cause false positives.
Another tip is to use dumpbin.exe (ships with MSVC). Use dumpbin /all to dump all details of a binary to a text/hex dump. This can make it more obvious to see what/where is changing.
For example:
dumpbin /all program1.exe > program1.txt
dumpbin /all program2.exe > program2.txt
windiff program1.txt program2.txt
Or use your favourite text diffing tool, instead of Windiff.
You may find Microsoft's bindiff.exe tool useful, which can be obtained here:
Windows XP Service Pack 2 Support Tools
It has a /v option, to instruct it to ignore certain binary fields, such as timestamps, checksums, etc.:
"BinDiff uses a special compare routine for Win32 executable files that masks out various build time stamp fields in both files when performing the compare. This allows two executable files to be marked as "Near Identical" when the files are truely identical, except for the time they were built."
However, it sounds like you may be already doing a superset of what bindiff.exe does.
Have you tried disassembling the executable and comparing the disassembly? That should remove a lot of the distracting details you mention, and make removing others a lot easier.
Is there a way to either force compiler to use relative paths, or to fool it into thinking the path is not what it is?
You have two ways to do this:
In either case, you're going to map and reuse the same drive letter for a folder before you start a particular build, so that the path appears identical to the compiler.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With