What approach(es) have you used for lightweight Python unit-tests on App Engine?

Question

I'm about to embark on some large Python-based App Engine projects, and I think I should check with Stack Overflow's "wisdom of crowds" before committing to a unit-testing strategy. I have an existing unit-testing framework (based on unittest with custom runners and extensions) that I want to use, so anything "heavy-weight"/"intrusive" such as nose, webtest, or gaeunit doesn't seem appropriate. The crucial unit tests in my worldview are extremely lightweight and fast ones, ones that run in an extremely short time, so I can keep running them over and over all the time without breaking my development rhythm (e.g., for a different project, I get 97% or so coverage for a 20K-lines project with several dozens of super-fast tests that take 5-7 seconds, elapsed time, for a typical run, overall -- that's what I consider a decent suite of small, fast unit-tests). I'll have richer/heavier tests as well of course, all the way to integration tests with selenium or windmill, that's not what I'm asking about;-) -- my focus in this question (and in most of my development endeavors;-) is on the small, lightweight unit-tests that lightly and super-rapidly cover my code, not on the deeper ones.

So I think what I need is essentially a set of small, very lightweight simulations of the various key App Engine subsystems -- data store, memcache, request/response objects and calls to webapp handlers, user handling, mail, &c, roughly in this order of priority. I haven't found exactly what I'm looking for, so it seems to me that I should either rely on mox, as I've done often in the past, which basically means mocking each subsystem used in a given test and setting up all expectations &c (strong, but lots of work each time, and very sensitive to the tested-code's internals, i.e. very "white-box"y), or rolling my own simulation of each subsystem (and doing asserts on the simulated subsystems' states as part of the unit tests). The latters seems feasible, given GAE's Python-side strong "stubs" architecture... but I can't believe I need to roll my own, i.e., that nobody's already written such simple-minded simulators!-) E.g., for the datastore, it looks like what I need is more or less the "datastore on file" stub that's already part of the SDK, plus a way to mark it readonly and easy-to-use accessors for assertions about the datastore's state; and so forth, subsystem by subsystem -- each seems to need "just a bit more" than what's already in the SDK, "perched on top" of the existing "stubs" architecture.

So, before diving in and spending a day or two of precious development time "rolling my own" simulations of GAE subsystems for unit testing purposes, I thought I'd double check with the SO crowd and see what y'all think of this... or, if there's already some existing open source set of such simulators that I can simply reuse (or minimally tweak!-), and which I've just failed to spot in my searching!-)

Edit: to clarify, if I do roll my own, I do plan to leverage the SDK-supplied stubs where feasible; but for example there's no stub for a datastore that gets initially read in from a file but then not saved at the end, so I need to subclass and tweak the existing one (which also doesn't offer particularly convenient ways to do asserts on its state -- same for the mail service stub, etc). That's what I mean by "rolling my own" -- not "rewriting from scratch"!-)

Edit: "why not GAEUnit" -- GAEUnit is nice for its own use cases, but running dev_appserver and seeing results in my browser (or even via urllib.urlopen) is definitely not what I'm after -- I want to use a fully automated setup, suitable for running within an existing test-running framework which is based on extending unittest, and no HTTP in the way (said framework defines a "fast" test as one that among other thing does no sockets and minimal disk I/O -- we simulate or mock these -- so via gaeunit I could do no better than "medium" tests) + no convenient way to prepopulate datastore for each test (and no OO structure to help customize things).

Answer 1

You don't need to write your own stubs - the SDK includes them, since they're what it uses to emulate the production APIs. Not all of them are suitable for use in unit-tests, but most are. Check out this code for an example of the setup/teardown code you need to make use of the built in stubs.

Answer 2

NoseGAE is a nose plugin that support unittests by automatically setting up the development environment and a test datastore for you. Very useful when developing on dev_appserver.