Running python scripts within a subpackage of my package

Question

Having some trouble figuring out the correct, python 2.x preferred way to do relative imports so that I can keep test scripts together in one subpackage, and have those test scripts be able to test my library.

$ farm\testpad\testpad.py
Traceback (most recent call last):
  File "C:\farm\testpad\testpad.py", line 4, in <module>
    from ..animals.dog import dog
ValueError: Attempted relative import in non-package

$ python -m farm\testpad\testpad
C:\Python27\python.exe: No module named farm\testpad\testpad

In the following example, what do I need to modify to do what I want? Please, and thanks for any help given.

e.g.

Package structure:

farm/
    __init__.py # empty
    animals/
        __init__.py # empty
        dog.py
    testpad/
        __init__.py # empty
        testpad.py

dog.py

import sys
import os

class dog():
  def __init__(self, name):
    self.name = name

  def speak(self):
    print "woof"

testpad.py

import os
import sys

from ..animals.dog import dog

# create a dog and test its speak action
def testpad():
  d = dog("Allen")
  d.speak()

if __name__ == "__main__":
  testpad()

Answer 1

Three things:

• always run your tests scripts from root of your project. This simple rule does not harm anything and will simplify your scenario
• prefer absolute imports
• -m option for python expects module in dotted form

Applying this to your code

Modify testpad.py

import os
import sys

from farm.animals.dog import dog

# Create a dog and test its speak action

    def testpad():
      d = dog("Allen")
      d.speak()

    if __name__ == "__main__":
      testpad()

Call it from python -m <module>

$ python -m farm.testpad.testpad
woof

Bonus - testing from nose

For my testing I use following pattern

• keep all testing code in project subdir called test or better tests
• use nose testing framework

Being in root of your project

Install nose:

$ pip install nose

What creates a command nosetests

Reorganize your code

$ mkdir tests
$ mv farm/testpad/testpad.py tests/test_animals.py
$ rm -rf farm/testpad/

Run the tests (currently just one)

The simples way, wich does the work, but tries to keep the output minimal:

$ nosetests
.
----------------------------------------------------------------------
Ran 1 test in 0.003s

OK

As you see, nose is discovering test cases on his own (searching for whatever starts with test)

Make it a bit more verbose:

$ nosetests -v
test_animals.testpad ... ok

----------------------------------------------------------------------
Ran 1 test in 0.003s

OK

Now you know, what tests were run.

To see captured outut, use -s switch:

$ nosetests -vs
test_animals.testpad ... woof
ok

----------------------------------------------------------------------
Ran 1 test in 0.003s

OK

Add more tests to test_animals.py

import os
import sys

# create a dog and test its speak action
def test_dog():
    from farm.animals.dog import dog
    d = dog("Allen")
    d.speak()

def test_cat():
    from farm.animals.dog import cat
    c = cat("Micy")
    d.speak()

def test_rabbit():
    from farm.animals.dog import rabbit
    r = rabbit()
    r.speak()

Test it:

$ nosetests -vs
test_animals.test_dog ... woof
ok
test_animals.test_cat ... ERROR
test_animals.test_rabbit ... ERROR

======================================================================
ERROR: test_animals.test_cat
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/javl/Envs/so/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/javl/sandbox/so/testpad/tests/test_animals.py", line 12, in test_cat
    from farm.animals.dog import cat
ImportError: cannot import name cat

======================================================================
ERROR: test_animals.test_rabbit
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/javl/Envs/so/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/javl/sandbox/so/testpad/tests/test_animals.py", line 17, in test_rabbit
    from farm.animals.dog import rabbit
ImportError: cannot import name rabbit

----------------------------------------------------------------------
Ran 3 tests in 0.004s

FAILED (errors=2)

Add --pdb switch to jump to debugger (if you install ipdbplugin, you might use --ipdb):

$ nosetests -vs --pdb
test_animals.test_dog ... woof
ok
test_animals.test_cat ... > /home/javl/sandbox/so/testpad/tests/test_animals.py(12)test_cat()
-> from farm.animals.dog import cat
(Pdb) l
  7         from farm.animals.dog import dog
  8         d = dog("Allen")
  9         d.speak()
 10  
 11     def test_cat():
 12  ->     from farm.animals.dog import cat
 13         c = cat("Micy")
 14         d.speak()
 15  
 16     def test_rabbit():
 17         from farm.animals.dog import rabbit
(Pdb) 

Use h or find some tutorial for debugger, it is great tool

Real test cases shall assert that behaviour is as expected.

Assert printed values are as expected

As you are printing to stdout, we can capture the output using mocking (in Python 2.x you have to install it, in Python 3.x you from unittest import mock):

$ pip install mock

And modify your test_animals.py:

from mock import patch
from StringIO import StringIO

# create a dog and test its speak action
@patch("sys.stdout", new_callable=StringIO)
def test_dog(mock_stdout):
    from farm.animals.dog import Dog
    d = Dog("Allen")
    d.speak()
    assert mock_stdout.getvalue() == "woof\n"

@patch("sys.stdout", new_callable=StringIO)
def test_cat(mock_stdout):
    from farm.animals.cat import Cat
    c = Cat("Micy")
    c.speak()
    assert mock_stdout.getvalue() == "mnau\n"

@patch("sys.stdout", new_callable=StringIO)
def test_rabbit(mock_stdout):
    from farm.animals.rabbit import Rabbit
    r = Rabbit("BB")
    r.speak()
    assert mock_stdout.getvalue() == "Playboy, Playboy\n"

Final test case of my taste test_mytaste.py

def test_dog():
    from farm.animals.dog import Dog
    d = Dog("Allen")
    sound = d.speak()
    assert sound == "woof", "A dog shall say `woof`"

def test_cat():
    from farm.animals.cat import Cat
    c = Cat("Micy")
    sound = c.speak()
    assert sound == "mnau", "A cat shall say `mnau`"

def test_rabbit():
    from farm.animals.rabbit import Rabbit
    r = Rabbit("BB")
    sound = r.speak()
    assert sound == "Playboy, Playboy", "A Rabbit shall say ..."

This requires you refactor your code (class names starting uppercase, speak method not printing but returning the sound).

In fact, you may start writing your code from test cases and adding tested modules later on, this often leads to nicer design as you start thinking of real use from the very beginning.

Selective call of test cases

nose is great in searching for test cases (generally whatever starts with test), but sometime you might focus on particular test or use differently named python file. You can tell nose to use just one test file:

$ nosetests -vs tests/test_mytaste.py 
test_mytaste.test_dog ... ok
test_mytaste.test_cat ... ok
test_mytaste.test_rabbit ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.002s

OK

or even target nose to run specific test from it:

$ nosetests -vs tests/test_mytaste.py:test_dog
test_mytaste.test_dog ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK