Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

install conda package to google colab

I try to install packages from anaconda to google's colab.

But it doesn't work. The whole thing is voodoo magic.

The following code is in one cell.

Notebook's cell:

!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
!bash Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local/
!rm Miniconda3-latest-Linux-x86_64.sh
!conda install -y --prefix /usr/local/ ujson aiohttp tqdm
import sys
os.environ['PYTHONPATH'] = "/usr/local/miniconda3"
os.environ['PATH'] = '/usr/local/miniconda3/bin:' + os.environ['PATH']
sys.path.append('/usr/local/lib/python3.6/site-packages/')
import ujson

Result:

ModuleNotFoundError: No module named 'ujson'

If I go into a bash shell with "!bash" and then run the "bash's" python, I can import ujson in that python. But, if I directly import ujson in the "notebook's" python, it doesn't work.

The methods here doesn't seem to work anymore

  • How to build libraries via conda on colab.research?
  • What's the latest conda version compatible with Google Colab suggests the following, which doesn't work now:
!wget -c https://repo.anaconda.com/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
!chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
!bash ./Miniconda3-4.5.4-Linux-x86_64.sh -b -f -p /usr/local
!conda install -q -y --prefix /usr/local ujson
import sys
sys.path.append("/usr/local/conda/lib/python3.6/site-packages/")
print(ujson.dumps({1:2}))

What is the latest hack that would work?

like image 608
Duh Huh Avatar asked Dec 13 '19 23:12

Duh Huh


People also ask

How install conda package in Colab?

To install conda, a library has been created specifically for Google Colab, conda-colab, and fortunately for us it is very easy to use ! You just have to install it with the pip command, then install conda with the condacolab. install() function. The code should return something like : conda 4.9.

Can we use conda in Google Colab?

How does it work? Google Colab runs on Python 3.7. We install the Miniconda distribution on top of the existing one at /usr/local , add a few configuration files so we stay with Python 3.7 ( conda auto updates by default) and the newly installed packages are available.


2 Answers

There are 2 problems that must be solved:

  • ujson will normally upgrade to python 3.7, must avoid this.
  • path to conda library is changed, must update it.

For 1, you need to add python=3.6 to conda install.

For 2, you need to add path to /usr/local/lib/python3.6/site-packages

Here's the new code

# same
!wget -c https://repo.anaconda.com/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
!chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
!bash ./Miniconda3-4.5.4-Linux-x86_64.sh -b -f -p /usr/local
# update 1
!conda install -q -y --prefix /usr/local python=3.6 ujson
# update 2
import sys
sys.path.append('/usr/local/lib/python3.6/site-packages')
# test it
import ujson
print(ujson.dumps({1:2}))
like image 179
korakot Avatar answered Sep 23 '22 06:09

korakot


In the google colab, Jupyter IDE, try executing:

!pip install ujson

This worked for me before. Let me know if it works now.

like image 35
Aditya Bhattacharya Avatar answered Sep 19 '22 06:09

Aditya Bhattacharya