Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to transfer Anaconda env installed on one machine to another? [Both with Ubuntu installed]

I have been using Anaconda(4.3.23) on my GuestOS ubuntu 14.04 which is installed on Vmware on HostOS windows 8.1. I have setup an environment in anaconda and have installed many libraries, some of which were very hectic to install (not straight forward pip installs). few libraries had inner dependencies and had to be build together and from their git source.

Problem I am going to use Cloud based VM (Azure GPU instance) to use GPU. but I don't want to get into the hectic installation again as i don't want to waste money on the time it will take me to install all the packages and libraries again

Is there any way to transfer/copy my existing env (which has everything already installed) to the Cloud VM?

like image 263
Qaisar Rajput Avatar asked Aug 24 '17 14:08

Qaisar Rajput


4 Answers

From the very end of this documentation page:

Save packages for future use:

conda list --export > package-list.txt

Reinstall packages from an export file:

conda create -n myenv --file package-list.txt
like image 128
Miladiouss Avatar answered Nov 19 '22 19:11

Miladiouss



If conda list --export failes like this ...

Executing conda list --export > package-list.txt creates a file which looks like this:

# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: win-64
_tflow_1100_select=0.0.1=gpu
absl-py=0.5.0=py_0
astor=0.7.1=py_0
...

But creating a new environment by executing conda create -n myenv --file package-list.txt gives me this error:

Solving environment: ...working... failed

PackagesNotFoundError: The following packages are not available from current channels:

  - markdown==2.6.11=py_0
  ...

... then try to use conda env export

According to this discussion execute the following command on your source machine:

source activate yourEnvironment
conda env export --no-builds > file.txt

On the target machine execute:

conda env create --file /path/to/file.txt

The file generated by conda env export looks a bit different, but it contains pip packages as well:

name: yourEnvironment
channels:
  - conda-forge
  - defaults
dependencies:
  - absl-py=0.5.0
  ...
  - pip:
    - astroid==2.0.4
    ...
like image 23
Adrian Kiesthardt Avatar answered Nov 19 '22 20:11

Adrian Kiesthardt


## You can try below approach to move all the package from one machine to other :
## Note : Machine that packages are being moved should be same and python version also should be same



$ pip install conda-pack



# To package an environment:

## Pack environment my_env into my_env.tar.gz

$ conda pack -n my_env

## Pack environment my_env into out_name.tar.gz

$ conda pack -n my_env -o out_name.tar.gz


## Pack environment located at an explicit path into my_env.tar.gz

$ conda pack -p /explicit/path/to/my_env


# After following above approach, you will end up with a tar.gz file. Now to install package from this zip file follow below approach.

## To install the environment:

## Unpack environment into directory `my_env`

$ mkdir -p my_env
$ tar -xzf my_env.tar.gz -C my_env


## Use Python without activating or fixing the prefixes. Most Python
## libraries will work fine, but things that require prefix cleanups
## will fail.

$ ./my_env/bin/python


## Activate the environment. This adds `my_env/bin` to your path

$ source my_env/bin/activate


## Run Python from in the environment

(my_env) $ python


## Cleanup prefixes from in the active environment.
## Note that this command can also be run without activating the environment
## as long as some version of Python is already installed on the machine.

(my_env) $ conda-unpack
like image 2
devesh48 Avatar answered Nov 19 '22 19:11

devesh48


You can probably get away with copying the whole Anaconda installation to your cloud instance.

like image 1
Burrito Avatar answered Nov 19 '22 19:11

Burrito