Linking pyenv python to homebrew in order to avoid homebrew [email protected] installation

Question

Some packages from brew require [email protected] as a dependency. For example:

$ brew deps vim
gdbm
gettext
libyaml
lua
[email protected]
perl
[email protected]
readline
ruby
sqlite
xz

However, I want to manage all my python installations with pyenv and I would not like to download [email protected] with brew. This would imply having to exactly same versions of python installed in 2 different locations, which I would like to avoid. Reading up on brew, pyenv and python I´ve come to understand that having python installed in different parts of the system may cause some trouble in the future, which I'd also like to avoid. Unfortunately I cannot seem to resolve the python dependency in brew packages through pyenv. Below follow the steps I've tried to overcome this.

I have installed pyenv with brew, and necessary python versions from there.

$ pyenv versions
  system
* 3.8.2 (set by PYENV_VERSION environment variable)

I have tried solving this according to this Github discussion by setting a brew alias such as:

alias brew='env PATH=${PATH//$(pyenv root)\/shims:/} brew'

As that did not resolve the dependency issue, I tried create a link for [email protected] in /usr/local/Cellar which would point to the pyenv python, somehow similar to this issue with:

ln -s  ~/.pyenv/versions/3.8.2 $(brew --cellar python)@3.8

This did not work, so I have also tried adding the link to ´/usr/local/bin´:

ln -s  ~/.pyenv/versions/3.8.2 /usr/local/bin/[email protected]

However, running brew info vim still shows that the [email protected] dependency is not satisfied.

$ brew info vim
vim: stable 8.2.0900 (bottled), HEAD
Vi 'workalike' with many additional features
https://www.vim.org/
Conflicts with:
  ex-vi (because vim and ex-vi both install bin/ex and bin/view)
  macvim (because vim and macvim both install vi* binaries)
Not installed
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/vim.rb
==> Dependencies
Required: gettext ✘, lua ✘, perl ✘, [email protected] ✘, ruby ✘

Any ideas how can I link/connect my pyenv python installation to homebrew so that the additional [email protected] is not installed? Or maybe solve the issue in another manner using pyenv global/local/shell? I am currently using macOs Catalina.

Any help is greatly appreciated!

Answer 1

Unfortunately "unlinking" python from all formulae it is somewhat unnecessary as with each update you must "relink" them.

Take black, as an example:

$ ls -al $(brew --prefix black)/libexec/bin
total 104
drwxr-xr-x  16 thecesrom  staff   512 Jun 11 08:32 .
drwxr-xr-x   6 thecesrom  staff   192 Jun 11 08:32 ..
-rw-r--r--   1 thecesrom  staff  8834 Jun 10 15:27 Activate.ps1
-rw-r--r--   1 thecesrom  staff  1916 Jun 11 08:32 activate
-rw-r--r--   1 thecesrom  staff   865 Jun 11 08:32 activate.csh
-rw-r--r--   1 thecesrom  staff  2005 Jun 11 08:32 activate.fish
-rwxr-xr-x   1 thecesrom  staff   256 Jun 11 08:32 black
-rwxr-xr-x   1 thecesrom  staff   251 Jun 11 08:32 black-primer
-rwxr-xr-x   1 thecesrom  staff   257 Jun 11 08:32 blackd
-rwxr-xr-x   1 thecesrom  staff  1000 Jun 11 08:32 chardetect
-rwxr-xr-x   1 thecesrom  staff   257 Jun 11 08:32 pip
-rwxr-xr-x   1 thecesrom  staff   257 Jun 11 08:32 pip3
-rwxr-xr-x   1 thecesrom  staff   257 Jun 11 08:32 pip3.9
lrwxr-xr-x   1 thecesrom  staff    84 Jun 10 15:27 python -> ../../../../../opt/[email protected]/Frameworks/Python.framework/Versions/3.9/bin/python3.9
lrwxr-xr-x   1 thecesrom  staff    84 Jun 10 15:27 python3 -> ../../../../../opt/[email protected]/Frameworks/Python.framework/Versions/3.9/bin/python3.9
lrwxr-xr-x   1 thecesrom  staff    84 Jun 10 15:27 python3.9 -> ../../../../../opt/[email protected]/Frameworks/Python.framework/Versions/3.9/bin/python3.9

Notice that python, python3 and python3.9 all point to Homebrew's python dependency. So every time a formula depending on python is updated you must unlink all symlinks and recreate them pointing to your pyenv-installed version. But this still would not remove python downloaded via brew, and you would potentially end up with two installations of the same Python version in your machine.

A different way to see this problem is to add brew's python to pyenv.

As you can see, in my case [email protected] was installed as a dependency to black and other formulae, so this is what I did.

First, create a symlink at ~/.pyenv/versions.

$ cd ~/.pyenv/versions
$ ln -sfv "$(brew --prefix [email protected])" 3.9
$ ls -al
total 0
drwxr-xr-x  4 thecesrom  staff  128 Aug 31 07:51 .
drwxr-xr-x  6 thecesrom  staff  192 Jun  8 10:58 ..
drwxr-xr-x  6 thecesrom  staff  192 Aug 31 07:48 2.7.18
lrwxr-xr-x  1 thecesrom  staff   25 Aug 31 07:51 3.9 -> /usr/local/opt/[email protected]

2021-09-06 edit: I suggest using 3.9 to avoid having to change the symlink with every Python upgrade.

Since I wanted to include the same folders (bin, include, lib, share) as my other Python installation (2.7.18), which I installed via pyenv, then I created a symlink for include.

$ cd "$(brew --prefix [email protected])"
$ ln -sfv Frameworks/Python.framework/Versions/3.9/include/python3.9 include
include -> Frameworks/Python.framework/Versions/3.9/include/python3.9

Once I did that, I created symlinks for idle, pip, python and wheel at the following location:

$ cd "$(brew --prefix [email protected])/bin"
$ ln -sfv idle3 idle
idle -> idle3
$ ln -sfv pip3 pip 
pip -> pip3
$ ln -sfv python3 python
python -> python3
$ ln -sfv wheel3 wheel
wheel -> wheel3

I did that, because I got the following output when I ran brew link [email protected]:

$ brew link [email protected]
Warning: Already linked: /usr/local/Cellar/[email protected]/3.9.7
To relink, run:
  brew unlink [email protected] && brew link [email protected]

Once that's done, run pyenv rehash as recommended by pyenv whenever you install new versions.

$ pyenv rehash

Then, verify all versions managed by pyenv.

$ pyenv versions
  system
  2.7.18
* 3.9 (set by /Users/thecesrom/.pyenv/version)

And finally configure pyenv as needed. In my case I set 3.9 as global by running pyenv global 3.9, and now when I run python --version I get the following output:

$ python --version
Python 3.9.7

Answer 2

First of all, I appreciate the desire to keep redundancy to a minimum! I was sad to see no answers, but I think that I've figured it out. You can do this, though Homebrew warns against it and you're wading into the territory of relying on your own understanding to support this use case (as Homebrew states in their post). The SO issue you linked to has the right idea with the links, but from what I can tell they don't get the links quite right. I'm all for experimenting and learning (it's the only way?), so by all means go for it, but again you won't get much support if you're not able to dig into your own $PATH and follow the links that you've set up yourself.

If you already have any traces of Homebrew's python3, first get rid of them (brew unlink python3 and brew uninstall python3). If you already had [email protected] from Homebrew, when uninstalling it, note the packages that depended on it (for example, ffmpeg), and reinstall them after.

As of this post, Homebrew is expected Python 3.8.6 for it's [email protected], so first install that version with pyenv following their documentation:

pyenv install 3.8.6

That will put (by default) the actual Python install in

~/.pyenv/versions/3.8.6

Now we just need to add one link, and then let brew do the rest. I'll use full paths here, so you can run this from anywhere (and remember to read ln -s ... ... in your head as Link -Symbolic Target LinkName):

ln -s ~/.pyenv/versions/3.8.6 $(brew --cellar python)/3.8.6

With the -f flag, you could have omitted the trailing /3.8.6, as ln will use the name of the target. To be as explicit as possible on the link, you should have

ln -s ~/.pyenv/versions/3.8.6 /usr/local/Cellar/[email protected]/3.8.6

Here's what the result should look like:

➜  ~ ls -alF $(brew --cellar [email protected]) 
total 0
lrwxr-xr-x  1 [my username]  admin    36B Oct 14 16:52 3.8.6 -> 
/Users/[my username]/.pyenv/versions/3.8.6

Finally, let Homebrew manage the rest of necessary linking:

brew link python3