How to do scalar testing with android?

Question

I'm not sure "scalar testing" is the correct term for it but I mean tests that aren't boolean "fail or succeed". The problem I'm working on is a chromatic tuner for android:

http://code.google.com/p/androidtuner/

And I want to test the algorithm by running a few wav files and processing the resulting pitch graph. The goal is to define the scalar test result as a normalized x-minus-y-squared-sum where x is the detected pitch and y is the expected pitch. So a perfect test run would be 0 but more realistically I'd like to tweak the algorithm and see if/how it improved on all the test cases.

Generally speaking - can a unit test result in a number and not a boolean? Does the android testing framework allow it? How should I integrate whichever solution with Eclipse?

My current idea is to just circumvent everything and use adb to fetch files generated after running each test. Though that's not too awesome.

Answer 1

It strikes me that a unit test is the wrong tool for the task you're tackling here. By their very nature, unit testing frameworks are designed to produce true/false results.

If all you're seeking to do is produce some metrics by test invoking your algorithm with different input data you might want to consider writing your own Instrumentation subclass. This is what Android itself uses to run your unit tests on the device.

An Instrumentation will allow you to invoke your services/activities with whatever datasets you need and you can build up a results Bundle to summarise the results.

Plus you can invoke an instrumentation from the ADB, just like a set of test cases.