How to make Python package update within a code to take effect on-the-fly

Question

I have the following line of codes:

from pip import main as pipmain

# initial installation
pipmain(["install", "pyscenic==0.10.0"])
import pyscenic
pyscenic.__version__

# return 0.10.0
# Some large code here

# second installation
pipmain(["install", "install", "pyscenic==0.10.4"])
import pyscenic
pyscenic.__version__
# still return 0.10.0

# Another large chunk that required new version

There I want to upgrade the pyscenic package on-the-fly inside my code. However as I noted above, in the second installation the version still doesn't change. I expect it to change to 0.10.4. How can I do it properly?

I also tried this, still no avail:

import os
import importlib
os.system('pip install pyscenic==0.10.0')
import pyscenic
pyscenic.__version__
os.system('pip install pyscenic==0.10.4')
import pyscenic
pyscenic.__version__
importlib.reload(pyscenic)
pyscenic.__version__

All code tested on IPython (interactive). If I exit the IPython and re-enter again it will take effect. But that's not what I want.

Answer 1

As several previous answers (1, 2) and recent requests have mentioned, pip and Python weren't really designed with this in mind.

But with some clever hacking of Python's name system, and some knowledge of the package you want to work with, you could install two versions next to each other:

# Copyright © 2021 Alexander L. Hayes
# MIT License

git clone [email protected]:aertslab/pySCENIC pySCENIC100
git clone [email protected]:aertslab/pySCENIC pySCENIC104

(
  cd pySCENIC100
  git checkout 0.10.0
  sed -i "s/pyscenic/pyscenic100/g" setup.py
  sed -i "s/pyscenic/pyscenic100/g" MANIFEST.in
  sed -i "s/pyscenic/pyscenic100/g" setup.cfg
  (
    cd src
    mv pyscenic pyscenic100
    (
      cd pyscenic100
      sed -i "s/pyscenic/pyscenic100/g" binarization.py
      sed -i "s/pyscenic/pyscenic100/g" __init__.py
      sed -i "s/pyscenic/pyscenic100/g" _version.py
    )
  )
  python setup.py install
)

(
  cd pySCENIC104
  git checkout 0.10.4
  sed -i "s/pyscenic/pyscenic104/g" setup.py
  sed -i "s/pyscenic/pyscenic104/g" MANIFEST.in
  sed -i "s/pyscenic/pyscenic104/g" setup.cfg
  (
    cd src
    mv pyscenic pyscenic104
    (
      cd pyscenic104
      sed -i "s/pyscenic/pyscenic104/g" binarization.py
      sed -i "s/pyscenic/pyscenic104/g" __init__.py
      sed -i "s/pyscenic/pyscenic104/g" _version.py
      (
        cd cli
        sed -i "s/pyscenic/pyscenic104/g" *.py
      )
    )
  )
  python setup.py install
)

This bash script clones the repository twice, checks out versions 0.10.0 and 0.10.4, does some renaming via sed, and finally installs two libraries named pyscenic100 and pyscenic104:

import pyscenic100
import pyscenic104

print(pyscenic100.__version__)
print(pyscenic104.__version__)
# 0.10.0+0.g3de37cb.dirty
# 0.10.4+0.g436561f.dirty

I don't know what happens during "# Some large code here", but it looks like examples from the documentation/tests work:

from pyscenic100.featureseq import Feature as Feature100
from pyscenic104.featureseq import Feature as Feature104


f1 = Feature100.from_string('chr1 12 50 feature1 10.0 +')
f2 = Feature100.from_string('chr1 40 60 feature2 10.0 -')
print(f1.has_overlap_with(f2))
# True

f1 = Feature104.from_string('chr1 12 50 feature1 10.0 +')
f2 = Feature104.from_string('chr1 40 60 feature2 10.0 -')
print(f1.has_overlap_with(f2))
# True