poetry change python version to 3.x

Question

According to poetry's docs, the proper way to setup a new project is with poetry new poetry-demo, however this creates a project based on the now deprecated python2.7 by creating the following toml file:

[tool.poetry]
name = "poetry-demo"
version = "0.1.0"
description = ""
authors = ["Harsha Goli <[email protected]>"]

[tool.poetry.dependencies]
python = "^2.7"

[tool.poetry.dev-dependencies]
pytest = "^4.6"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

How can I update this to 3.7? Simply changing python = "^2.7" to python = "^3.7" results in the following error when poetry install is run:

[SolverProblemError]
The current project's Python requirement (2.7.17) is not compatible with some of the required packages Python requirement:
  - zipp requires Python >=3.6

Because no versions of pytest match >=4.6,<4.6.9 || >4.6.9,<5.0
 and pytest (4.6.9) depends on importlib-metadata (>=0.12), pytest (>=4.6,<5.0) requires importlib-metadata (>=0.12).
And because no versions of importlib-metadata match >=0.12,<1.5.0 || >1.5.0
 and importlib-metadata (1.5.0) depends on zipp (>=0.5), pytest (>=4.6,<5.0) requires zipp (>=0.5).
Because zipp (3.1.0) requires Python >=3.6
 and no versions of zipp match >=0.5,<3.1.0 || >3.1.0, zipp is forbidden.
Thus, pytest is forbidden.
So, because poetry-demo depends on pytest (^4.6), version solving failed.

Answer 1

Whenever you change dependencies by hand in your pyproject.toml you have to take care of these points:

1. Run poetry lock --no-update afterwards. The reasons for this is, that poetry install takes the poetry.lock as input if can find one and not the pyproject.toml.

2. If you change the python version and uses in-project virtualenv, remove the .venv before running poetry install. poetry doesn't change the python version of a venv once it is created, because it uses the python version itself to create the virtualenv.

Answer 2

Poetry makes it super easy to work with different Python versions or virtual environments. The recommended way to specify your Python version according to Poetry docs is

poetry env use /path/to/preferred/python/version

You can get the path to your Python version by running

which python3.7

Answer 3

Use:

poetry env use 3.9

or

poetry env use $(which python3.9)

doesn't work for some reason for python 3.9. Though having explicitly stated version requirement in pyproject.toml file, it's sad that one needs to manually enforce this.

Answer 4

I had the same problem. I solve it by fixing the first line in the file /home/nordman/.poetry/bin/poetry (nordman is my local name).

Just change #!/usr/bin/env python to #!/usr/bin/env python3

Answer 5

Interestingly, poetry is silently failing due to a missing package the tool itself relies on and continues to install a broken venv. Here's how you fix it.

sudo apt install python3-venv
poetry env remove python3
poetry install

I had to remove pytest, and then reinstall with poetry add pytest.

EDIT: I ran into this issue again when upgrading a project from python3.7 to python3.8 - for this instead of installing python3-venv, you'd want to install python3.8-venv instead

Answer 6

You can change in pyproject.toml and execute de this command "poetry env use 3.x" that works for me.

Answer 7

I've experienced the exact same behavior on my MacBook! So to make this practical, first, let's see, what is the default Python for the MacBook. I'm running macOS Big Sur 11.2.1 (20D74). In terminal:

python --version                                                                                                                  
Python 2.7.16

With that, let's install poetry. I've used poetry installation script from the GitHub. Using this script is a recommended method to install poetry (though I'd argue this is NOT the best idea to pipe scripts from the Internet into the python interpreter) The installation script is pretty smart: it attempts to detect which python is available on the system:

def _which_python(self):
    """Decides which python executable we'll embed in the launcher script."""
    allowed_executables = ["python", "python3"]
    if WINDOWS:
        allowed_executables += ["py.exe -3", "py.exe -2"]
...

So as the default Python is 2.7.16, and that is what returned by calling python, the entry in ~/.poetry/bin will look like:

#!/usr/bin/env python

And here you go! The default python will end up in the pyproject.toml file and you'll need to do an extra dance to a) make sure python3 is a dependency for your project; and b) that the virtual environment will use python3 as python interpreter.

As @mfalade mentioned, you can set environment with poetry env:

poetry env use /path/to/python3

And also modifying the pyproject.toml with:

...
[tool.poetry.dependencies]
python = "^3.9"
...

From this point you are good to go, poetry will use python3 to create virtual environment for you project, dependency is specified in the file and ... the grass is green again.

But can we use python3 by default? There is an insightful thread on GitHub and another one. And if you look back at the code snippet from the installation script above, you might wonder why not to check for python3 first and then for python, especially taking into account that this is the exact order for Windows installation. Well, you are not alone, I also wonder ;)

I would not suggest editing the ~/.poetry/bin/poetry file directly as this file is generated by the installation script (so what will happen if you run installation script again? And again?).

Really, this is a minor annoyance, and knowing the tool it is easy to work around it. I'd expect that to be mentioned in installation guide though...

Anyway, hope it helps!

Answer 8

You can simply use pyenv for that. Create .python-version file inside your project and poetry will match the exact python version.

# check current python version (set up globally)
❯ pyenv version
3.9.0 (set by /Users/[email protected]/.python-version)

# create .python-version file for project
❯ pyenv local 3.9.0

# check python version again (now it's set up locally)
❯ pyenv version
3.9.0 (set by /Users/[email protected]/Documents/myproject/.python-version)

❯ poetry lock
(...)

❯ poetry run python --version
Python 3.9.0

Answer 9

I tried every solution I could find, but could not get poetry to pick up the upgraded version of Python. Finally found something that worked.

tldr;

1. Run poetry config virtualenvs.in-project true to use local venvs
2. Run deactivate to exit any existing venv
3. Run poetry shell to activate a new, local venv

So here's what I was seeing:

(my-project) ~/I/my-project ❯❯❯ poetry env info

Virtualenv
Python:         3.9.6
Implementation: CPython
Path:           /Users/my-user/.local/share/virtualenvs/my-project-some-hash
Valid:          True

System
Platform: darwin
OS:       posix
Python:   /usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9

I tried running poetry env remove python and got an error:

(my-project) ~/I/my-project ❯❯❯ poetry env remove python

  ValueError

  Environment "my-project-some-hash-py3.9" does not exist.

Meanwhile, I saw somewhere that it's recommend to use local virtual environments by setting this property, so I did it:

poetry config virtualenvs.in-project true

This did not solve my problem, but then I realized that changing this setting would not take affect automatically because I was already in another non-local virtual environment (see poetry env info output above).

So based on the docs, I ran deactivate to deactivate the current virtual environment.

Now, I saw this:

~/I/my-project ❯❯❯ poetry env info

Virtualenv
Python:         3.10.1
Implementation: CPython
Path:           NA

System
Platform: darwin
OS:       posix
Python:   /Users/my-user/.pyenv/versions/3.10.1

Now I ran poetry shell to create a new virtual environment and it picked up the new local venvs setting:

~/I/my-project ❯❯❯ poetry shell
Creating virtualenv my-project in /Users/my-user/projects/my-project/.venv
Spawning shell within /Users/my-user/projects/my-project/.venv

And finally, I saw the upgraded Python version I was expecting!

(.venv) ~/I/my-project ❯❯❯ poetry run python -V
Python 3.10.1

Update: After I did all of the above, I noticed that poetry env use works fine!

(.venv) ~/I/my-project ❯❯❯ python -V
Python 3.10.1
(.venv) ~/I/my-project ❯❯❯ poetry env use 3.9.6
Recreating virtualenv my-project in /Users/my-user/projects/my-project/.venv
Using virtualenv: /Users/my-user/projects/my-project/.venv
(.venv) ~/I/my-project ❯❯❯ python -V
Python 3.9.6