How to approach something complex?

Question

You know that particular part of your code that is essential for the project but will probably take a lot of time to get it done? Do you ever get the feeling that you'd rather work on something else (probably less important) or not code at all instead of working on that part? That beast you try so hard to avoid and use every lazy trick you know to delay its inevitable implementation?

Now, I'm probably just being lazy, but I've always had to deal with code like that. Write something I don't feel like writing (and it's worse if you're doing it for fun and not getting paid!). A large system that will take a lot of time to get it into a stage where you get back any useful results or indication of it working. How do you start coding something like that? Most people would probably suggest divide and conquer and similar architectural techniques, but this isn't about how you do it; it's about how you get yourself started on doing it. What's the very first steps you'd take?

Answer 1

I'll tell a story of a case in which this happened to me.

I wanted to implement a new frametype decision algorithm for x264 that used forward dynamic programming (the Viterbi algorithm). But it was going to be complicated, messy, ugly, and so forth. And I really didn't want to do it. I tried to pawn off the project onto Google Summer of Code, but out of some sort of terrible bad luck, the one student that we had that simply bailed on his project... was the student that chose that project.

So after two months of complaining about it and dodging it, I finally got to work on the algorithm. And here's how I did it.

First, I talked to the other developer, who apparently already had some ideas on how to do it. We talked it over and he explained it to me until I fully understood the process from an algorithmic standpoint. This is the first step of any such project: understand the algorithm behind it so well that you can pseudocode the entire thing.

Then, I talked to another coworker of mine. We went up to a whiteboard and I sketched it out until he understood it too. By explaining it to someone else, I gained understanding myself. This is the second step: explain the algorithm to someone else so well that they can pseudocode it. This is an emulation of the programming process, since programming is a form of "explaining" an algorithm to the computer.

Then, I wrote a simple Java prototype that used arbitrary fake values for the cost function and was solely being used to test the Viterbi search. I finished it, and checked it against an exhaustive search--it matched perfectly. My dynamic programming was correct. This is the third step: write the simplest possible form of the algorithm in the simplest possible environment.

Then I ported it to C, x264's native language. It worked again. This is the fourth step: port that simple form of the algorithm to the full environment.

Then, finally, I replaced the fake cost function with the real one. After some bughunting and fixing, it worked. This is the final step: integrate the algorithm completely with the environment.

This process took barely a week, but from the perspective of me at the start of the project, it was completely daunting and I couldn't get myself to even get started--yet by breaking it down into such a step by step process, I was able to not only get it done, but get it done much faster than I expected.

And the benefits went far beyond x264; I now understand Viterbi so thoroughly that I now can explain it to others... and those others can benefit greatly from it. For example, one of the ffmpeg developers is using an adaptation of my algorithm and code to optimally solve a somewhat different problem: optimal header placement in audio files.

Answer 2

Generally I love the large, complex part. They are the parts that actually extend a challenge and force me to carefully consider what I'm doing. It's all the small, tedious bits that I dislike. However, when it comes to doing anything I've been putting off I find one simple piece of advice important:

JUST DO IT!!!

Seriously, once it's started it's much easier to finish. I always find I put things off until I start them, then suddenly I find that, now that I've started, it's not as bad as I had imagined, and look, it's almost done already!

Answer 3

Divide and conquer is not just about structuring code, it also works as an approach to make a project conceptually manageable. If I have a hard time getting started on a project it almost always because it's to big and scary. By dividing into conceptually manageable pieces, it becomes less scary.

I also believe in "tracer bullets" as described by the pragmatic programmers. Reduce the project to the absolutely simplest possible "proof of concept" of the core parts, e.g. without UI, special cases, error handling and so on. Perhaps its just a few core routines with associated unit-tests. With this you have conquered the scary parts, and can build from the core.

Basically the trick to getting started (for me at least) is: Don't start on the whole project. Start on one small (preferably core) part and build from there. If I still have a hard time getting started, It's because the small part I decided on is still to big, so I have to divide and reduce it further.