How to install Python packages over SSH Port Forwarding?

Question

I am controlling a remote unit over SSH and OPENVPN.

On the remote unit I want to install some Python packages using pip but:

• the remote company firewall allows only traffic on port 22 (and not 443, needed by pip);
• DNS is not installed on the remote unit;
• I cannot modify any OPENVPN settings (or I would like to avoid this option as it means to access some remote sysadmin and try to convince him that the configuration must be changed);
• all systems are Linux (Ubuntu + Debian). Non Windows involved.

Stripping down hours of attempts (I am not a system admin and my knowledge on this subject is very limited), the idea was to open an obvious SSH port forwarding:

ssh -R 9999:pypi.python.org:443 [email protected]

and then, on the remote unit play with pip install:

pip install pymodbus==1.3.2 --proxy localhost:9999

But this command returns:

Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement pymodbus==1.3.2

/root/.pip/pip.log is:

  Getting page https://pypi.python.org/simple/pymodbus/
  Could not fetch URL https://pypi.python.org/simple/pymodbus/: connection error: ('Connection aborted.', BadStatusLine("''",))
  Will skip URL https://pypi.python.org/simple/pymodbus/ when looking for download links for pymodbus==1.3.2
  Getting page https://pypi.python.org/simple/
  Could not fetch URL https://pypi.python.org/simple/: connection error: ('Connection aborted.', BadStatusLine("''",))
  Will skip URL https://pypi.python.org/simple/ when looking for download links for pymodbus==1.3.2
  Cannot fetch index base URL https://pypi.python.org/simple/
  URLs to search for versions for pymodbus==1.3.2:
  * https://pypi.python.org/simple/pymodbus/1.3.2
  * https://pypi.python.org/simple/pymodbus/
  Getting page https://pypi.python.org/simple/pymodbus/1.3.2
  Could not fetch URL https://pypi.python.org/simple/pymodbus/1.3.2: connection error: ('Connection aborted.', BadStatusLine("''",))
  Will skip URL https://pypi.python.org/simple/pymodbus/1.3.2 when looking for download links for pymodbus==1.3.2
  Getting page https://pypi.python.org/simple/pymodbus/

It is obvious the remote unit cannot read the index page on pypi.pthon.org because the connection is refused.

What is the correct syntax for what I am trying to achieve?

Answer 1

Proxy is going to be tricky. I suggest that you scp the pip module source file and install it locally from source. Use
pip install package —download="/pth/to/downloaded/file” to get the package, scp it to the dest server and use pip install “/pth/to/scp/file”

Answer 2

It's look like my problem. after exploration, I have found a solution. And because in my region, pypi.python.org is slow, so I change my pip.conf and use pypi.douban.com/simple, as my index-url. this website use http protocol. so in my solution. I use 80 port as my target port.

Problem: I have two host. host1 could connect Pypi.douban.com. and host2 couldn't. but I can connect host2 in host1 through ssh.

so in host2, I open a tmux session and open a ssh tunnel by local port forwarding(not remote port forwarding):

ssh -L 9999:pypi.douban.com:80 username@host1

after this redirect, I can use

pip install scikit-learn --proxy localhost:9999

to install package in host2.