How to test a dockerized application in an Azure DevOps (Server) pipeline?

Question

I have a simple python dockerized application whose structure is

/src
 - server.py
 - test_server.py
Dockerfile
requirements.txt

in which the docker base image is Linux-based, and server.py exposes a FastAPI endpoint.
For completeness, server.py looks like this:

from fastapi import FastAPI
from pydantic import BaseModel

class Item(BaseModel):
    number: int

app = FastAPI(title="Sum one", description="Get a number, add one to it", version="0.1.0")

@app.post("/compute")
async def compute(input: Item):
    return {'result': input.number + 1}

Tests are meant to be done with pytest (following https://fastapi.tiangolo.com/tutorial/testing/) with a test_server.py:

from fastapi.testclient import TestClient
from server import app
import json

client = TestClient(app)

def test_endpoint():
    """test endpoint"""
    response = client.post("/compute", json={"number": 1})
    values = json.loads(response.text)
    assert values["result"] == 2

Dockerfile looks like this:

FROM tiangolo/uvicorn-gunicorn:python3.7

COPY . /app

RUN pip install -r requirements.txt

WORKDIR /app/src

EXPOSE 8000

CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]

At the moment, if I want to run the tests on my local machine within the container, one way to do this is

1. Build the Docker container
2. Run the container, get its name via docker ps
3. Run docker exec -it <mycontainer> bash and execute pytest to see the tests passing.

Now, I would like to run tests in Azure DevOps (Server) before pushing the image to my Docker registry and triggering a release pipeline. If this sounds an OK thing to do, what's the proper way to do it?

So far, I hoped that something along the lines of adding a "PyTest" step in the build pipeline would magically work:

I am currently using a Linux agent, and the step fails with

The failure is not surprising, as (I think) the container is not run after being built, and therefore pytest can't run within it either :(

Another way to solve the solve this is to include pytest commands in the Dockerfile and deal with the tests in a release pipeline. However I would like to decouple the testing from the container that is ultimately pushed to the registry and deployed.

Is there a standard way to run pytest within a Docker container in Azure DevOps, and get a graphical report?

Answer 1

Update your azure-pipelines.yml file as follows to run the tests in Azure Pipelines

Method-1 (using docker)

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'
steps:
- task: Docker@2
  inputs:
    command: 'build'
    Dockerfile: '**/Dockerfile'
    arguments: '-t fast-api:$(Build.BuildId)'
- script: |
    docker run fast-api:$(Build.BuildId) python -m pytest
  displayName: 'Run PyTest'

Successfull pipeline screenshot

Method-2 (without docker)

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'
strategy:
  matrix:
    Python37:
      python.version: '3.7'

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '$(python.version)'
  displayName: 'Use Python $(python.version)'

- script: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
  displayName: 'Install dependencies'

- script: |
    pip install pytest pytest-azurepipelines
    python -m pytest
  displayName: 'pytest'

BTW, I have one simple FastAPI project, you can reference if your want.