Py.test - Applying variables to decorators from a csv?

Question

Please bear with me while I try to explain my predicament, I'm still a Python novice and so my terminology may not be correct. Also I'm sorry for the inevitable long-windedness of this post, but I'll try to expalin in as much relevant detail as possible.

A quick rundown:

I'm currently developing a suite of Selenium tests for a set of websites that are essentially the same in functionality, using py.test

• Tests results are uploaded to TestRail, using the pytest plugin pytest-testrail.

• Tests are tagged with the decorator @pytestrail.case(id) with a unique case ID

A typical test of mine looks like this:

@pytestrail.case('C100123')  # associates the function with the relevant TR case
@pytest.mark.usefixtures()
def test_login():
   # test code goes here

As I mentioned before, I'm aiming to create one set of code that handles a number of our websites with (virtually) identical functionality, so a hardcoded decorator in the example above won't work.

I tried a data driven approach with a csv and a list of the tests and their case IDs in TestRail.

Example:

website1.csv:
Case ID | Test name
C100123 | test_login


website2.csv:
Case ID | Test name
C222123 | test_login

The code I wrote would use the inspect module to find the name of the test running, find the relevant test ID and put that into a variable called test_id:

import csv
import inspect
class trp(object):
def __init__(self):
    pass


with open(testcsv) as f:  # testcsv could be website1.csv or website2.csv
    reader = csv.reader(f)
    next(reader)  # skip header
    tests = [r for r in reader]


def gettestcase(self):
    self.current_test = inspect.stack()[3][3]
    for row in trp.tests:
        if self.current_test == row[2]:
            self.test_id = (row[0])
            print(self.test_id)
            return self.test_id, self.current_test


def gettestid(self):
    self.gettestcase()

The idea was that the decorator would change dynamically based on the csv that I was using at the time.

@pytestrail.case(test_id)  # now a variable
@pytest.mark.usefixtures()
def test_login():
   trp.gettestid()
   # test code goes here

So if I ran test_login for website1, the decorator would look like:

@pytestrail.case('C100123')

and if I ran test_login for website2 the decorator would be:

@pytestrail.case('C222123')

I felt mighty proud of coming up with this solution by myself and tried it out...it didn't work. While the code does work by itself, I would get an exception because test_id is undefined (I understand why - gettestcase is executed after the decorator, so of course it would crash.

The only other way I can handle this is to apply the csv and testIDs before any test code is executed. My question is - how would I know how to associate the tests with their test IDs? What would an elegant, minimal solution to this be?

Sorry for the long winded question. I'll be watching closely to answer any questions if you need more explanation.

Answer 1

pytest is very good at doing all kind of metaprogramming stuff for the tests. If I understand your question correctly, the code below will do the dynamic tests marking with pytestrail.case marker. In the project root dir, create a file named conftest.py and place this code in it:

import csv
from pytest_testrail.plugin import pytestrail


with open('website1.csv') as f:
    reader = csv.reader(f)
    next(reader)
    tests = [r for r in reader]


def pytest_collection_modifyitems(items):
    for item in items:
        for testid, testname in tests:
            if item.name == testname:
                item.add_marker(pytestrail.case(testid))

Now you don't need to mark the test with @pytestrail.case()at all - just write the rest of code and pytest will take care of the marking:

def test_login():
    assert True

When pytest starts, the code above will read website1.csv and store the test IDs and names just as you did in your code. Before the test run starts, pytest_collection_modifyitems hook will execute, analyzing the collected tests - if a test has the same name as in csv file, pytest will add the pytestrail.case marker with the test ID to it.