Separate 'debug' and 'release' builds?

Question

Having separate debug and release builds is a good idea, because it does make development easier.

But debug builds should be for development only, not for testing. You test release builds only. And you don't use developers to test those builds, you use testers.

It's a simple policy that gives the best of both worlds, IMO.

Edit: In response to a comment, I think it's obvious that debug and release builds (can) generate different code. Think "-DDEBUG" vs. "-DNDEBUG", and "#if defined(DEBUG)", etc.

So it's vital that you test the code that you end up shipping. If you do generate different code in debug and release builds, that means testing twice - regardless of whether or not it's tested by the same person.

Debug symbols are not that big an issue, however. Always build with debugging symbols, keep a copy of the unstripped binary, but release a stripped binary. As long as you tag each binary with a build number somehow, you should always be able to identify which unstripped binary corresponds to the stripped binary that you have to debug...

How to strip binaries and load symbols in your debugger from an external source is platform-dependent.

This might be minor, but it adds up to what others have said here. One of the advantages of having QA test release builds is that over time the built in debugging and logging capabilities of your software will advance due to the needs of developers who need to figure out why things are going wrong in QA.

The more the developers need to debug release builds, the better tools you'll have later when customers start having issues. Of course, no reason for developers to work on release builds as part of the development cycle.

Also, I don't know any software company that has long enough cycles to afford the overhead of switching QA from debug to release builds halfway through a version's testing period. Having to do a full QA cycle is something that all too often happens pretty rarely.

Our policy is to have developers work on Debug builds, but EVERYONE else (QA, BAs, sales etc) runs the release version. Yesterday I had to fix a bug that only showed up in the release build it, was obvious what was happening simply BECAUSE it only showed up in release

It's first one here in this shop, and I've been here 18 months or so.

Where things get hairy is when the Release build does different things to the debug build - Yes, I have been to Hell and seen this in some very old, very ropy production code.

I see no reason why not to have both if the only difference between the configurations are debug symbols and optimisations.

so you need to build a release which you can if necessary debug ... this may mean enabling debug symbols, and disabling some optimizations, even in the 'release' build.

Ummm... it sounds like you're doing a debug build to me... right?

The part where you went wrong is this statement:

I think it's better to release the version of the software which your developers actually tested

Developers don't test code. Tests test code.

Your unit tests should test ALL build configurations. Do not make your developers work with one hand tied behind their back - let them use all the debugging tools they have at there disposal. A Debug build is one of these.

Regarding asserts: the use of assertions greatly depends on whether or not you program by contract. If you do, then assertions merely check the contract in a debug build.

As per my answer in the linked thread, we also use the same build for debug and release for very similar reasons. The 10%-20% performance gains from the optimiser tend to be very minor when compared to manual optimisations at algorithm level. A single build removes many potential bugs. Specifically;

• Uninitialised variables and small buffer overflows may end up with very different results in debug and optimised release builds.

• Even with the symbolic information available, debugging an optimised release can be difficult as the object doesn't match the source, e.g. variables may have been optimised out and code may have been re-arranged. Thus bugs reported in tested release builds can be more difficult, and hence time-consuming, to track down.

Having compared unoptimised and optimised builds under automated regression tests, the performance gains provided by the optimisation don't provide enough extra value to have two builds in my case. It is may be worth noting that the software that I develop is very CPU hungry (e.g. creating and manipulating large surface models).

When developing with Java, I hate non-debug versions. When an exception is thrown, you get no line information which makes it hard or even impossible to track bugs down. Also, the runtime difference between debug and non-debug is around 5% with Java 5 or later, so this is really no issue and with todays hard disks, size doesn't matter anymore.

On the plus side using debug versions:

• Stack traces contain all the information you need
• Variables can be examined
• If you have a problem in production, you can simply attach to the running process without having to stop the server first to install a debug version.
• You won't get caught by clever optimization bugs
• The build is more simple (just one artifact)

Developers work with debug builds, QA and everyone else uses the release version, which we call "production". The main advantage to this is that in the debug build, we can add lots of extra code and assertions. Some objects contain extra pieces of information that have no use except when viewing code in the debugger. Some objects validate themselves periodically to make sure that all the state information is consistent. These things make the debug version much slower, but they have helped us find no end of bugs that would have been hell to find in the production build.

As I said, all of our QA and performance testing uses production builds, and we do occasionally run into problems that show up in production but not in debug. But they're relatively rare, and as a developer, the advantages of debugging a debug build rather than a production build far outweigh that problem.