Running pre-commit hooks (e.g. pylint) when developing with docker

Question

pre-commit hooks are run in a separate virtual environment (or Docker container). However our code is running on Docker and we're also developing using Docker.

Up until now we didn't have to install any dependencies on our host systems, but when running mypy, isort and pylint they run into problems because they can't access the dependencies installed.

Our first idea was to install the dependencies in a virtual environment on the host system, but that also seems like a clumsy workaround.

Is there a good way to run pre-commit with full access to the container?

Answer 1

We had the same thought and that's what we ended up with in our team:

• Pre-commit Docker image. Simple, minimal Docker image based on docker image containing just pre-commit itself (implicitly containing python, pip and so on). Note: docker image is really needed here because we do use Docker-based hooks.
• Set of separated Docker images containing "complicated" hooks/checkers. By "complicated" I mean ones which require some build-time dependencies, compilers, extra libraries and so on, stuff which cannot be installed with simple "pip install" on clean machine. One example would be clang-format which we build from sources so the final clang-format Docker image contains just clang-format binary. That's where mypy would go probably as well as it has a lot of extra non-Python dependencies.

Note: pre-commit is a bit bad when working in "docker in docker" mode, so we had to apply a workaround, see https://github.com/pre-commit/pre-commit/issues/1387

In the end, our .pre-commit-config.yaml file has entries like:

# Normal, "simple" hooks which can be just installed as is
- repo: ...pre-commit-hooks/pre-commit-hooks
  rev: v3.3.0
  hooks:
   - ...

# Docker hooks
- repo: local
  hooks:
  - id: docker-clang-format
    name: Docker Clang Format
    language: docker_image
    types:
    - c++
    entry: <our-registry.com>/clang_format:11

# Local workarounds for devs who cannot or don't want to use Docker, but still would like to benefit from running pre-commit locally
- repo: <...>/pre-commit-clang-format
    rev: ...
    hooks:
      - id: clang-format
        stages: [manual]  # Mind this line, only for manual run
        types:
          - c++