Pytest Generate Tests Based on Arguments

Question

New to pytest...

I have the following in conftest.py to collect a team argument from the command line, and read in a yaml config file:

import pytest
import yaml


def pytest_addoption(parser):
    parser.addoption(
        '--team',
        action='store',
        )


@pytest.fixture
def team(request):
    return request.config.getoption('--team')


@pytest.fixture
def conf(request):
    with open('config.yml', 'r') as f:
        conf = yaml.load(f.read())
    return conf

I want to run a test on each player inside conf[team]['players'] (a list). I can do so as follows in test_players.py:

def test_players(team, conf):
    players = conf[team]['players']
    for p in players:
        assert p == something

This sort of works, in that it iterates through the players, but the whole thing gets treated as a single test. If anything fails the whole test is treated as failed. I'd like each player to be tested separately.

If I put in the players manually I can get this to work:

import pytest

class Test_Player():
    @pytest.mark.parametrize(
        'player', [
            'player1',
            'player2',
            'player3',
        ],
    )
    def test_player(self, player):
        assert player == something

So my problem is that I don't know how to get conf[team] passed into pytest.mark.parametrize. I've tried these, but in both cases it complains that conf isn't defined.

import pytest

class Test_Player():
    @pytest.mark.parametrize(
        'player', conf[team]['players'],
    )
    def test_player(self, player):
        assert player == something

and

import pytest

class Test_Player(team, conf):
    @pytest.mark.parametrize(
        'player', conf[team]['players'],
    )
    def test_player(self, player):
        assert player == something

What am I missing here?

Answer 1

The problem with your setup is that you want to parameterize on conf[team], but conf needs to be defined at import time, because that's when the decorator executes.

So, you'll have to go about this parameterization differently, using pytest's metafunc parametrization features.

.
├── conftest.py
├── teams.yml
└── test_bobs.py

In the yaml file:

# teams.yml
bobs: [bob1, bob2, potato]
pauls: [paultato]

In the test module:

# test_bobs.py
def test_player(player):
    assert 'bob' in player

In the pytest conf:

import pytest
import yaml


def pytest_addoption(parser):
    parser.addoption('--team', action='store')


def pytest_generate_tests(metafunc):
    if 'player' in metafunc.fixturenames:
        team_name = metafunc.config.getoption('team')

        # you can move this part out to module scope if you want
        with open('./teams.yml') as f:
            teams = yaml.load(f)

        metafunc.parametrize("player", teams.get(team_name, []))

Now execute:

pytest --team bobs

You should see three tests executed: two passing tests (bob1, bob2) and one failing test (potato). Using pytest --team pauls will make one failing test. Using pytest --team bogus will result in a skipped test. If you want a different behaviour there, change the teams.get(team_name, []) to, for example, teams[team_name].