Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Keeping the same, shared virtualenvs when switching from pyenv-virtualenv to pipenv

I started looking at pipenv and it seems to be pretty good. My only concern is, that most of my projects involve numpy, scipy and some other not-so-small libraries.

The current way manage my projects: I have pyenv and pyenv-virtualenv installed. I have a few (currently 4) specific virtualenvs that each cater to a type of project. The projects themselves have .pyenv-version set, I have the autoload virtualenv feature of pyenv enabled. If I need to share a project, I generate a requirements.txt with pip freeze -l from the virtualenv.

So in my current setup, I have X number of projects and Y, Y << X number of virtualenvs, all amounting to a few GB of harddisk space. Note that because of large libraries like numpy each of the virtualenvs are pretty big, around 700-900 MB.

My question:

As far as I understand, pipenv will, by default create a virtualenv for all of my projects, so the harddisk space taken up by my virtualenvs would increase considerably. What I'm interested in is:

  • is it possible to share pipenv environments across several projects, that use exactly the same dependencies? i.e. multiple pipenv configs that load the same virtualenv?
  • if not, is it possible to generate pipenv config files from a virtualenv I set up with pyenv? i.e. I would not use pipenv to actually run my projects, I would not create any virtualenvs with pipenv, but I would create pipenv config files for sharing the project (in this case, probably along side a requirements.txt as well).

edit: I rewrote most of the question to make it clearer.

like image 505
fbence Avatar asked Apr 28 '19 16:04

fbence


People also ask

How do I make Pipenv ignore Virtualenvs 1?

You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning. No virtualenv has been created for this project(/Library/Frameworks/Python. framework/Versions/3.7/lib/python3.

Is Pipenv better than virtualenv?

Pipenv adds a number of higher level features than virtualenv, namely dependency management. Virtualenv doesn't manage dependencies, all it does is provide isolated environment to install dependencies.

What is the difference between Pyenv and Pipenv?

Pyenv is to manage Python versions, and Pipenv is to create virtual environments required for each project and manage python packages and their dependencies for each project. This is a great way of working on different projects locally without any hassles. We will use Homebrew to install Pyenv and Pipenv.


1 Answers

pipenv doesn't seem to be a good fit for your specific workflow because it's project-centric rather than environment-centric. pipenv treats a virtual environment as volatile and reserves the right to alter it freely if circumstances call for it. You can use it but in the case of alterations to your environments, it will be a pain to keep all projects synchronized due to pipenv's stricter scrunity.

You can explicitly specify a virtual environment for pipenv to use for a project by creating a .venv file in the project root with a path to it (normally, virtualenvs are created in a specific location with autogenerated names that include a hash of the path to the project). This seems to be undocumented.

However, pipenv, unlike virtualenv, checks and enforces that the virtual environment has the exact set of modules satisfying conditions in Pipfile and the exact "last tested configuration" specified in a generated Pipfile.lock.

  • Besides, if you generate Pipfile from requirements.txt, it will specify exact package versions and include all dependencies -- while by design, it's supposed to contain more intelligent information.

So, if you change any package version in an environment, you'll need to:

  • update all Pipfile.locks in affected projects (e.g. copy the changed one). With a generated Pipfile, you may get away with deleting them instead.
  • update all Pipfiles in affected projects to the new package versions (e.g. copy the changed one) if there was a change
like image 192
ivan_pozdeev Avatar answered Oct 23 '22 03:10

ivan_pozdeev