Buildbot parsing Python Unit test results

Question

I have a test suite that outputs test results in the Python Unit Test format: http://docs.python.org/library/unittest.html

Is there an existing Buildbot module/plugin that can parse this form?

Example:

DigitalReadWrite_02                                         ... ok
DigitalReadWrite_03                                         ... ok
DigitalReadWrite_04                                         ... ok
PWMoutput_02  (PWM=128 50% LOW 49% HIGH)                    ... ok
PWMoutput_03  (PWM=128 50% LOW 49% HIGH)                    ... ok
PWMoutput_04  (PWM=128 50% LOW 49% HIGH)                    ... ok
--------------------------
Ran 6 tests in 1.652s

OK

I've written a custom parser, but it's only got the basic cases. Is it worth the effort to make it comprehensive for all flavors of Python Unit test format.

Answer 1

No, it makes no sense to develop a parser. You can obtain the equivalent information from classes in runner.py module.

Consider extending both classes TextTestRunner and TextTestResult with your custom logic (python 2.7). The output you have listed is produced by TextTestResult.

Alternatively you can extend only TextTestResult and alter the class attribute TextTestRunner.resultclass setting it to your new extension class name.

The data you can extract from TextTestResult and put into some list of dictionaries is greater or equivalent to the data your parser is able to extract.

The unittest framework allows such tricks due to its flexible design. Hope this was helpful.

[EDIT]

I would find publishing your results so far (e.g. as an Open Source code on github) potentially very useful for people that find your question!

Having said that I doubt it would be easy to improve the actual parser beyond basic regexp parsing.

If you still would like to pursuit the text parsing approach - you might need to enumerate and describe "all flavors of Python Unit test format" that you want to cover/support. If you are lucky to put such a description in form of a context-free grammar, then maybe you would be able to develop a parser for it, that would cover "those" cases as a form of a language.

Please take my word of caution: if text parsing is not covered by simple regexp'ing and there is a chance that you are trying to parse some irregular (context-sensitive) language - most likely you would find it extremely difficult to accomplish.