Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python Poetry: how to specify platform-specific dependency alternatives?

Some background: The project I'm working on uses python-ldap library. Since we are a mixed-OS development team (some use Linux, some macOS and some Windows), I'm trying to make the project build on all environments. Unfortunately, python-ldap is not officially supported for Windows, but there are unofficial wheels maintained by Christoph Gohlke. I've tested the wheel file and it works fine.

The problem: how do I tell Poetry to use the wheel on Windows and the official python-ldap package on Linux and macOS?

I've tried multiple things, including using multiple constraint dependencies and markers:

python-ldap = [
    { markers = "sys_platform == 'linux'", version = "*" },
    { markers = "sys_platform == 'win32'", path="lib/python_ldap-3.2.0-cp36-cp36m-win_amd64.whl" }

... but, judging from the poetry.lock file, it seems markers are then merged and just determine whether the library should be installed at all:

[[package]]
category = "main"
description = "Python modules for implementing LDAP clients"
marker = "sys_platform == \"linux\" or sys_platform == \"win32\""
name = "python-ldap"

Is there another way of dealing with platform-specific dependencies in Poetry?

like image 825
Igor Brejc Avatar asked Apr 06 '20 04:04

Igor Brejc


People also ask

Does poetry manage Python versions?

Since version 1.2, Poetry no longer supports managing environments for Python 2.7.

Where are poetry dependencies installed?

By default, Poetry is installed into a platform and user-specific directory: ~/Library/Application Support/pypoetry on MacOS.

What is poetic Lock Python?

There is a specific option for the lock command: poetry lock --no-update. This makes it possible to remove a dependency from pyproject. toml and update the lock file without upgrading dependencies. Note that this is only available since 1.1.


1 Answers

The best way to do this is the use the --platform option with the poetry add command. For installing faiss on both Mac (faiss-cpu with no CUDA GPU support) and Linux (faiss-gpu with GPU/CUDA support available) you run the following commands:

# Add each package to your project
poetry add faiss-gpu --platform linux
poetry add faiss-cpu --platform darwin

# Thereafter just install
poetry install

As mentioned above, you can do this in the pyproject.toml file as described in the other answer, but the CLI is best. Be sure to poetry update if you edit pyproject.toml directly:

[tool.poetry.dependencies]
faiss-cpu = {version = "^1.7.1", platform = "darwin"}
faiss-gpu = {version = "^1.7.1", platform = "linux"}

Holla!

like image 136
rjurney Avatar answered Oct 05 '22 04:10

rjurney