How to run unitests of the form test/a.py?

Question

Is it possible to implement a Python project with a file structure like the following?:

myproj
├── a.py
├── b.py
├── c.py
└── test/
    ├── a.py
    ├── b.py
    └── c.py

Note, in particular, that the test scripts under test/ have the same basenames as the module files they are testing  ¹. (In other words, test/a.py contains the unit tests for a.py; test/b.py contains those for b.py, etc.)

The tests under test/ all import unittest and define subclasses of unittest.TestCase.

I want to know how to run the tests under test/, both individually, and all together.

I've tried many variations of python -m unittest ..., but they all either fail (examples below), or end up running zero tests.

For example,

% python -m unittest test.a
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.7/unittest/__main__.py", line 12, in <module>
    main(module=None)
  File "/usr/lib/python2.7/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
  File "/usr/lib/python2.7/unittest/main.py", line 149, in parseArgs
    self.createTests()
  File "/usr/lib/python2.7/unittest/main.py", line 158, in createTests
    self.module)
  File "/usr/lib/python2.7/unittest/loader.py", line 130, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python2.7/unittest/loader.py", line 100, in loadTestsFromName
    parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'a'

If I change the name of the test/ directory to t/, then the error becomes:

% python -m unittest t.a
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.7/unittest/__main__.py", line 12, in <module>
    main(module=None)
  File "/usr/lib/python2.7/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
  File "/usr/lib/python2.7/unittest/main.py", line 149, in parseArgs
    self.createTests()
  File "/usr/lib/python2.7/unittest/main.py", line 158, in createTests
    self.module)
  File "/usr/lib/python2.7/unittest/loader.py", line 130, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
    module = __import__('.'.join(parts_copy))
ImportError: No module named t

Or

% python -m unittest t/a.py
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.7/unittest/__main__.py", line 12, in <module>
    main(module=None)
  File "/usr/lib/python2.7/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
  File "/usr/lib/python2.7/unittest/main.py", line 149, in parseArgs
    self.createTests()
  File "/usr/lib/python2.7/unittest/main.py", line 158, in createTests
    self.module)
  File "/usr/lib/python2.7/unittest/loader.py", line 130, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
    module = __import__('.'.join(parts_copy))
ImportError: Import by filename is not supported.

(I am using Python 2.7.9.)

---

UPDATE

Since I put a bounty on this question, I'll be very explicit about what would constitute an acceptable answer.

Either one of the following would be acceptable:

• How to invoke unittest from the command line to run either individual tests, or all the tests under the test/ directory; it is acceptable for solution to entail small changes to the code in the test scripts (e.g. modifications to the import statements).

• If the file structure shown above is not possible for some reason, a detailed explanation would be an acceptable solution.

As a base case, start with the following minimal case with the following file structure:

myproj
├── a.py
├── b.py
└── test/
    ├── a.py
    └── b.py

...and the following contents

# a.py
def hello():
    print 'hello world'

# b.py
def bye():
    print 'good-bye world'

# test/a.py

import unittest
import a

class TestA(unittest.TestCase):
    def test_hello(self):
        self.assertEqual(a.hello(), None)

# test/b.py

import unittest
import b

class TestB(unittest.TestCase):
    def test_bye(self):
        self.assertEqual(b.bye(), None)

Show how one tells unittest to run the test test/a.py, and how to run "all the tests under test". (The latter should continue to work, even if the new test scripts are added to test, or some of the current test scripts are removed.)

---

Minimal tests of the proposals offered so far show that they don't work. For example:

% python -m unittest discover -s test -p '*.py'
EE
======================================================================
ERROR: test_hello (a.TestA)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/SHIVAMJINDAL/myproj/test/a.py", line 6, in test_hello
    self.assertEqual(a.hello(), None)
AttributeError: 'module' object has no attribute 'hello'

======================================================================
ERROR: test_bye (b.TestB)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/SHIVAMJINDAL/myproj/test/b.py", line 6, in test_bye
    self.assertEqual(b.bye, None)
AttributeError: 'module' object has no attribute 'bye'

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (errors=2)
% tree .
.
├── a.py
├── b.py
├── __init__.py
└── test/
    ├── a.py
    ├── b.py
    └── __init__.py

1 directory, 6 files

---

^{1 This constraint is very much intentional, and it is an integral part of the problem presented here. (IOW, a "solution" that entails relaxing this constraint is in fact not a solution.)}

Answer 1

I was able to make your approach work, but there are few changes that are needed and mandatory.

1. The test should be run with a parent directory context
2. Both main folder and test folder should have a __init__.py
3. The imports in the tests should be a relative import and not a direct import

So below is my tree structure

root@5db7ad85dafd:/project# tree
.
 __init__.py
 a.py
 test
     __init__.py
     a.py

1 directory, 4 files

root@5db7ad85dafd:/project# python --version
Python 2.7.9

project/a.py

hello = 'tarun'

project/test/a.py

import unittest
from .. import a

class TestStringMethods(unittest.TestCase):
   def test_abc(self):
       assert a.hello == "tarun"

Notice the from .. import a which is import for this to work

Next we run the test being in the root folder of the project like below

root@5db7ad85dafd:/project# python -m unittest discover -t .. -s test -p "*.py"
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

-t here sets the top level import directory, so our relative imports can work -s tells which directory our tests are in -p tells what patter the test should be discovered in

When you want to run a individual test you will do something like below

python -m unittest discover -t .. -s test -p "a.py"

or

python -m unittest discover -t .. -s test -p "*.py" a

A picture is always worth more than words

Edit-1

Wanted to update my answer after seeing Peter's answer. The reason I didn't mention the import from a fixed named package was that it would mean that you need to know the name of the folder where the code is cloned and it is enforced to remain the same. But if you still want to go with that approach, then one approach is to move the actual into a sub-folder

So it would be repo/project/test/a.py and then in your tests you will use

from project import a

and then run it like below from the repo folder

root@5db7ad85dafd:/repo# python -m unittest discover -v -t project -s project.test -p "*.py"
test_abc (test.a.TestStringMethods) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

or like below from the test folder

root@5db7ad85dafd:/repo/project# python -m unittest discover -v -t .. -s test -p "*.py"
test_abc (project.test.a.TestStringMethods) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

In this case moving your project folder one level from root, will make sure that the project name is not dependent on the folder where your project is cloned

Answer 2

Tarun's answer is already pretty complete... In short:

1. You must make your modules importable (by creating __init__.py files)
2. You must tell the interpreter exactly where to look to prevent the naming clash between your modules.

The import doesn't have to be relative, though. Assuming that your project is going to be packaged up the import could (arguably, should) be precisely as you would expect users of your package to use - e.g. from myproj import a.

At this point I have also got python -m unittest discover -t .. -s test -p '*.py' to work. But this is the point where I get fed up with the extra hoops that the basic unittest package places on the user. I'd also recommend that once you have made the 2 changes, you also install nosetests (strictly speaking the python nose package) as it generally makes life easier for you to find and run the tests.

For example:

$ tree
.
├── a.py
├── a.pyc
├── b.py
├── b.pyc
├── __init__.py
├── __init__.pyc
└── test
    ├── a.py
    ├── a.pyc
    ├── b.py
    ├── b.pyc
    ├── __init__.py
    └── __init__.pyc

1 directory, 12 files
$ cat test/a.py
# test/a.py

import unittest
from myproj import a

class TestA(unittest.TestCase):
    def test_hello(self):
        self.assertEqual(a.hello(), None)
$ nosetests test/*.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK