Linking pyenv python to homebrew in order to avoid homebrew python@3.8 installation

Question

Some packages from brew require python@3.8 as a dependency. For example:

Click to copy

$ brew deps vim
gdbm
gettext
libyaml
lua
openssl@1.1
perl
python@3.8
readline
ruby
sqlite
xz

However, I want to manage all my python installations with pyenv and I would not like to download python@3.8 with brew. This would imply having to exactly same versions of python installed in 2 different locations, which I would like to avoid. Reading up on brew, pyenv and python I´ve come to understand that having python installed in different parts of the system may cause some trouble in the future, which I'd also like to avoid. Unfortunately I cannot seem to resolve the python dependency in brew packages through pyenv. Below follow the steps I've tried to overcome this.

I have installed pyenv with brew, and necessary python versions from there.

Click to copy

$ pyenv versions
  system
* 3.8.2 (set by PYENV_VERSION environment variable)

I have tried solving this according to this Github discussion by setting a brew alias such as:

Click to copy

alias brew='env PATH=${PATH//$(pyenv root)\/shims:/} brew'

As that did not resolve the dependency issue, I tried create a link for python@3.8 in /usr/local/Cellar which would point to the pyenv python, somehow similar to this issue with:

Click to copy

ln -s  ~/.pyenv/versions/3.8.2 $(brew --cellar python)@3.8

This did not work, so I have also tried adding the link to ´/usr/local/bin´:

Click to copy

ln -s  ~/.pyenv/versions/3.8.2 /usr/local/bin/python@3.8

However, running brew info vim still shows that the python@3.8 dependency is not satisfied.

Click to copy

$ brew info vim
vim: stable 8.2.0900 (bottled), HEAD
Vi 'workalike' with many additional features
https://www.vim.org/
Conflicts with:
  ex-vi (because vim and ex-vi both install bin/ex and bin/view)
  macvim (because vim and macvim both install vi* binaries)
Not installed
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/vim.rb
==> Dependencies
Required: gettext ✘, lua ✘, perl ✘, python@3.8 ✘, ruby ✘

Any ideas how can I link/connect my pyenv python installation to homebrew so that the additional python@3.8 is not installed? Or maybe solve the issue in another manner using pyenv global/local/shell? I am currently using macOs Catalina.

Any help is greatly appreciated!

Answer 1

Unfortunately "unlinking" python from all formulae it is somewhat unnecessary as with each update you must "relink" them.

Take black, as an example:

Click to copy

$ ls -al $(brew --prefix black)/libexec/bin
total 104
drwxr-xr-x  16 thecesrom  staff   512 Jun 11 08:32 .
drwxr-xr-x   6 thecesrom  staff   192 Jun 11 08:32 ..
-rw-r--r--   1 thecesrom  staff  8834 Jun 10 15:27 Activate.ps1
-rw-r--r--   1 thecesrom  staff  1916 Jun 11 08:32 activate
-rw-r--r--   1 thecesrom  staff   865 Jun 11 08:32 activate.csh
-rw-r--r--   1 thecesrom  staff  2005 Jun 11 08:32 activate.fish
-rwxr-xr-x   1 thecesrom  staff   256 Jun 11 08:32 black
-rwxr-xr-x   1 thecesrom  staff   251 Jun 11 08:32 black-primer
-rwxr-xr-x   1 thecesrom  staff   257 Jun 11 08:32 blackd
-rwxr-xr-x   1 thecesrom  staff  1000 Jun 11 08:32 chardetect
-rwxr-xr-x   1 thecesrom  staff   257 Jun 11 08:32 pip
-rwxr-xr-x   1 thecesrom  staff   257 Jun 11 08:32 pip3
-rwxr-xr-x   1 thecesrom  staff   257 Jun 11 08:32 pip3.9
lrwxr-xr-x   1 thecesrom  staff    84 Jun 10 15:27 python -> ../../../../../opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin/python3.9
lrwxr-xr-x   1 thecesrom  staff    84 Jun 10 15:27 python3 -> ../../../../../opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin/python3.9
lrwxr-xr-x   1 thecesrom  staff    84 Jun 10 15:27 python3.9 -> ../../../../../opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin/python3.9

Notice that python, python3 and python3.9 all point to Homebrew's python dependency. So every time a formula depending on python is updated you must unlink all symlinks and recreate them pointing to your pyenv-installed version. But this still would not remove python downloaded via brew, and you would potentially end up with two installations of the same Python version in your machine.

A different way to see this problem is to add brew's python to pyenv.

As you can see, in my case python@3.9 was installed as a dependency to black and other formulae, so this is what I did.

First, create a symlink at ~/.pyenv/versions.

Click to copy

$ cd ~/.pyenv/versions
$ ln -sfv "$(brew --prefix python@3.9)" 3.9
$ ls -al
total 0
drwxr-xr-x  4 thecesrom  staff  128 Aug 31 07:51 .
drwxr-xr-x  6 thecesrom  staff  192 Jun  8 10:58 ..
drwxr-xr-x  6 thecesrom  staff  192 Aug 31 07:48 2.7.18
lrwxr-xr-x  1 thecesrom  staff   25 Aug 31 07:51 3.9 -> /usr/local/opt/python@3.9

2021-09-06 edit: I suggest using 3.9 to avoid having to change the symlink with every Python upgrade.

Since I wanted to include the same folders (bin, include, lib, share) as my other Python installation (2.7.18), which I installed via pyenv, then I created a symlink for include.

Click to copy

$ cd "$(brew --prefix python@3.9)"
$ ln -sfv Frameworks/Python.framework/Versions/3.9/include/python3.9 include
include -> Frameworks/Python.framework/Versions/3.9/include/python3.9

Once I did that, I created symlinks for idle, pip, python and wheel at the following location:

Click to copy

$ cd "$(brew --prefix python@3.9)/bin"
$ ln -sfv idle3 idle
idle -> idle3
$ ln -sfv pip3 pip 
pip -> pip3
$ ln -sfv python3 python
python -> python3
$ ln -sfv wheel3 wheel
wheel -> wheel3

I did that, because I got the following output when I ran brew link python@3.9:

Click to copy

$ brew link python@3.9
Warning: Already linked: /usr/local/Cellar/python@3.9/3.9.7
To relink, run:
  brew unlink python@3.9 && brew link python@3.9

Once that's done, run pyenv rehash as recommended by pyenv whenever you install new versions.

Click to copy

$ pyenv rehash

Then, verify all versions managed by pyenv.

Click to copy

$ pyenv versions
  system
  2.7.18
* 3.9 (set by /Users/thecesrom/.pyenv/version)

And finally configure pyenv as needed. In my case I set 3.9 as global by running pyenv global 3.9, and now when I run python --version I get the following output:

Click to copy

$ python --version
Python 3.9.7

Answer 2

First of all, I appreciate the desire to keep redundancy to a minimum! I was sad to see no answers, but I think that I've figured it out. You can do this, though Homebrew warns against it and you're wading into the territory of relying on your own understanding to support this use case (as Homebrew states in their post). The SO issue you linked to has the right idea with the links, but from what I can tell they don't get the links quite right. I'm all for experimenting and learning (it's the only way?), so by all means go for it, but again you won't get much support if you're not able to dig into your own $PATH and follow the links that you've set up yourself.

If you already have any traces of Homebrew's python3, first get rid of them (brew unlink python3 and brew uninstall python3). If you already had python@3.8 from Homebrew, when uninstalling it, note the packages that depended on it (for example, ffmpeg), and reinstall them after.

As of this post, Homebrew is expected Python 3.8.6 for it's python@3.8, so first install that version with pyenv following their documentation:

Click to copy

pyenv install 3.8.6

That will put (by default) the actual Python install in

Click to copy

~/.pyenv/versions/3.8.6

Now we just need to add one link, and then let brew do the rest. I'll use full paths here, so you can run this from anywhere (and remember to read ln -s ... ... in your head as Link -Symbolic Target LinkName):

Click to copy

ln -s ~/.pyenv/versions/3.8.6 $(brew --cellar python)/3.8.6

With the -f flag, you could have omitted the trailing /3.8.6, as ln will use the name of the target. To be as explicit as possible on the link, you should have

Click to copy

ln -s ~/.pyenv/versions/3.8.6 /usr/local/Cellar/python@3.8/3.8.6

Here's what the result should look like:

Click to copy

➜  ~ ls -alF $(brew --cellar python@3.8) 
total 0
lrwxr-xr-x  1 [my username]  admin    36B Oct 14 16:52 3.8.6 -> 
/Users/[my username]/.pyenv/versions/3.8.6

Finally, let Homebrew manage the rest of necessary linking:

Click to copy

brew link python3