PyDev import times are 10x slower than using command line

Question

I'm new to python and using PyDev with Eclipse I noticed excruciatingly slow startup time whenever I try to execute the code I'm working on. I narrowed this down to libraries import.

For example if I run the following code

import timeit
startTime = timeit.default_timer()

import numpy as np
print("loaded numpy: ", timeit.default_timer() - startTime)

import pandas as pd
print("loaded pandas: ", timeit.default_timer() - startTime)

from pandas import ExcelWriter
print("loaded sub-pandas 1: ", timeit.default_timer() - startTime)

from pandas import DataFrame
print("loaded sub-pandas 2: ", timeit.default_timer() - startTime)

import timeit
print("loaded timeit: ", timeit.default_timer() - startTime)

from sqlalchemy.sql.expression import false
print("loaded sqlalchemy: ", timeit.default_timer() - startTime)

import os
print("loaded os: ", timeit.default_timer() - startTime)

It will benchmark time for PyDev as:

    loaded numpy:  6.791420515378803
    loaded pandas:  13.315319435084618
    loaded sub-pandas 1:  13.31538835744522
    loaded sub-pandas 2:  13.315418989605488
    loaded timeit:  13.315443057731413
    loaded sqlalchemy:  13.668371856921556
    loaded os:  13.668398113058927

while with command line execution it will be:

loaded numpy:  1.6967744335238362
loaded pandas:  3.7941380255968165
loaded sub-pandas 1:  3.7944563812624534
loaded sub-pandas 2:  3.795081787867914
loaded timeit:  3.795144146194173
loaded sqlalchemy:  3.915562085554165
loaded os:  3.915884087905548

Can anybody help with getting to the bottom of this? Even with the command line option 4s for loading a couple of standard libraries seems to be an overkill.

Thank you!

Answer 1

After two days of tearing my hair, the problem was somehow related to where the project is located. If the project py files are on the local it runs fast, if they are on the office network things slow down dramatically.

Note that the anaconda 3.6 installation is on the local. Also note that our network latency is not that bad. So i'm not quite sure why would this behavior happen.

So moving the project to C drive speeded things up to acceptable load time.

Here's the output for the new project located locally:

loaded numpy:  0.14122337554736752
loaded pandas:  0.5681306651263066
loaded sub-pandas 1:  0.568159473943701
loaded sub-pandas 2:  0.5681747900238348
loaded timeit:  0.5681882827610955
loaded sqlalchemy:  0.6529934184615654
loaded os:  0.6530225919475344