How to specify platforms-specific extras or version in Pipfile?

Question

The "Advanced Usage of Pipenv" docs, in the "Specifying Basically Anything" section, explains how to include a package requirement, conditionally on the os platform, in the Pipfile, eg:

pywinusb = {version = "*", sys_platform = "== 'win32'"}

It's not clear what to do when one wants to specify different version or options for a package, depending on the platform.

Specifically, I have this requirement:

faust = {version=">=1.10.1", extras=["aiodns", "ciso8601", "cython"]}

and I want to exclude the optional extra ciso8601 in Windows.

If I write

faust = {version=">=1.10.1", extras=["aiodns", "ciso8601", "cython"], platform_system = "!= 'Windows'"}
faust = {version=">=1.10.1", extras=["aiodns", "cython"], platform_system = "== 'Windows'"}

I get the error: tomlkit.exceptions.KeyAlreadyPresent: Key "faust " already exists.

Additionally, I want the same thing in the setup.py (install_requires section).

Answer 1

WORKING ANSWER - 06-30-2021 @15:43 UTC

I review the faust repository many times and found that aiodns, cython and ciso8601 aren't dependencies per se, but are bundles, which are installed with pip this way according to the ReadMe:

pip install "faust[aiodns, ciso8601, cython]"

I created a setup.py with your requirements. I was able to install faust with the bundles aiodns, ciso8601, cython on a non-windows system. I don't have a windows system to test that install, but I did flip the system platform parameters, which worked.

import setuptools

setuptools.setup(

    name="faust_pip_test", # change for your package
    version="1.0.0", # change for your package
    python_requires='>=3.6',
    install_requires=["faust[aiodns, ciso8601,cython]>=1.10.1;sys_platform != 'win32'",
                  "faust[aiodns, cython]>=1.10.1;sys_platform == 'win32'",]

)

Here is the faust_pip_test package being installed with faust and the additional packages aiodns, ciso8601 and cython on my MacBook.

After looking at the PipeEnv documentation, which is not intuitive it seem that you can do the following in your pipfile. Every reference I reviewed required you to specify a file path for the secondary key, which in this cause is faust_win. The file being called in faust_win is from faust PyPi downloads.

[packages]
faust = {version = ">=1.10.1",extras = ["aiodns", "ciso8601", "cython"],sys_platform = "!='windows'"}
faust_win = {file = 'https://files.pythonhosted.org/packages/79/f8/3fec4f5c3e5bf1ce8bb557ae507525253fa30a5cfc5984f342b931143f75/faust-1.10.4-py2.py3-none-any.whl',version = ">=1.10.1",extras = ["aiodns", "cython"],sys_platform = "=='windows'"}

I'm using PyCharm for my development environment. If I create a __init__.py in my project faust_pipenv_test it automatically triggers a dependencies install for my project. Faust standard dependencies requirements were installed plus the additional packages aiodns, ciso8601 and cython.

I also received a message that the faust_win requirement wasn't installed, because it didn't meet this requirement sys_platform =='win32'.

UPDATE 06-28-2021 @12:23 UTC

When I looked at the source code of faust I noted that the packages aiodns, cython and ciso8601 are dependencies in the faust's setup.py file.

So based on your question it seems that you don't want to install the faust default dependency of ciso8601 on a non-windows system. And based on the extras list it also seems that you are trying to limit other dependencies.

Is this what you are trying to accomplish?

The Python Developer's Guide states that the "extras" parameter is used to installed extra dependencies, which are in addition to the regular dependencies of a Python package being installed.

Reference: PEP 426 -- Metadata for Python Software Packages 2.0

The code below is from the PipeEnv Python package and it shows that the "extras" parameter is meshed with a package's core dependencies.

def dependencies(self):
        # type: () -> Tuple[Dict[S, PackagingRequirement], List[Union[S, PackagingRequirement]], List[S]]
        build_deps = []  # type: List[Union[S, PackagingRequirement]]
        setup_deps = []  # type: List[S]
        deps = {}  # type: Dict[S, PackagingRequirement]
        if self.setup_info:
            setup_info = self.setup_info.as_dict()
            deps.update(setup_info.get("requires", {}))
            setup_deps.extend(setup_info.get("setup_requires", []))
            build_deps.extend(setup_info.get("build_requires", []))
            if self.extras and self.setup_info.extras:
                for dep in self.extras:
                    if dep not in self.setup_info.extras:
                        continue
                    extras_list = self.setup_info.extras.get(dep, [])  # type: ignore
                    for req_instance in extras_list:  # type: ignore
                        deps[req_instance.key] = req_instance
        if self.pyproject_requires:
            build_deps.extend(list(self.pyproject_requires))
        setup_deps = list(set(setup_deps))
        build_deps = list(set(build_deps))
        return deps, setup_deps, build_deps

So the examples below needs testing, because it not clear in the Advanced Usage of Pipenv document if you can override the dependencies requirements from a third-party Python package.

[packages]
faust = {version= ">=1.10.1", extras=["aiodns", "cython"]}
ciso8601 = {version = ">=2.1.0", sys_platform = "!= 'Windows'", index="pypi"}

or 

[packages]
faust = {version=">=1.10.1", extras=["aiodns", "ciso8601", "cython"], sys_platform= "!= 'Windows'"}
"faust_win" = {version=">=1.10.1", extras=["aiodns", "cython"], sys_platform = "== 'Windows'"}

ORIGINAL POST 06-26-2021

---

Duplicate keys are not allowed in a TOML document.

For example this:

faust = {version=">=1.10.1", extras=["aiodns", "ciso8601", "cython"], platform_system = "!= 'Windows'"}

faust = {version=">=1.10.1", extras=["aiodns", "cython"], platform_system = "== 'Windows'"}

Reference the source code of pipenv

# file: /pipenv/vendor/tomlkit/exceptions.py
# which is referenced from here /pipenv/vendor/tomlkit/container.py

class KeyAlreadyPresent(TOMLKitError):
    """
    An already present key was used.
    """

    def __init__(self, key):
        message = 'Key "{}" already exists.'.format(key)

        super(KeyAlreadyPresent, self).__init__(message)

Additionally the duplicate key issue has been discussed on TOML Github repository.

• Allow duplicate keys
• How to handle duplicate keys