Python FTPS hangs on directory list in passive mode

Question

I managed to connect to a FTP server using curl and list the content of the directory out:

$ curl -v --insecure --ftp-ssl --user xxx:yyy blabla:990/out/
> AUTH SSL
< 234 Proceed with negotiation.
...
> USER xxx
< 331 Please specify the password.
> PASS yyy
< 230 Login successful.
> PBSZ 0
< 200 PBSZ set to 0.
> PROT P
< 200 PROT now Private.
> PWD
< 257 "/"
> CWD out
< 250 Directory successfully changed.
> EPSV
< 229 Entering Extended Passive Mode (|||51042|).
*   Trying aaa.bbb.ccc.ddd...
* Connecting to aaa.bbb.ccc.ddd (aaa.bbb.ccc.ddd) port 51042
* Connected to blabla (aaa.bbb.ccc.ddd) port 990 (#0)
> TYPE A
< 200 Switching to ASCII mode.
> LIST
< 150 Here comes the directory listing.
* Maxdownload = -1
* Doing the SSL/TLS handshake on the data stream
* SSL re-using session ID
* TLS 1.0 connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate: blabla
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
{ [539 bytes data]
100   539    0   539    0     0    900      0 --:--:-- --:--:-- --:--:--   899* Remembering we are in dir "out/"
< 226 Directory send OK.

I tried to do the same using python (2.7.10 on Mac OS X 10.10.5)

import ftplib

ftps = ftplib.FTP_TLS()
ftps.set_debuglevel(1)
print ftps.set_pasv(True)
print ftps.connect("blabla", 990)
print ftps.login("xxx", "yyy")
print ftps.sendcmd("PBSZ 0")
print ftps.prot_p()
print ftps.pwd()
print ftps.cwd("out")
print ftps.transfercmd("LIST")
ftps.close()

But with python the LIST command hangs indefinitely

*cmd* 'AUTH TLS'
*resp* '234 Proceed with negotiation.'
*cmd* 'USER xxx'
*resp* '331 Please specify the password.'
*cmd* 'PASS ********\n'
*resp* '230 Login successful.'
230 Login successful.
*cmd* 'PBSZ 0'
*resp* '200 PBSZ set to 0.'
200 PBSZ set to 0.
*cmd* 'PBSZ 0'
*resp* '200 PBSZ set to 0.'
*cmd* 'PROT P'
*resp* '200 PROT now Private.'
200 PROT now Private.
*cmd* 'PWD'
*resp* '257 "/"'
/
*cmd* 'CWD out'
*resp* '250 Directory successfully changed.'
250 Directory successfully changed.
*cmd* 'PASV'
*resp* '227 Entering Passive Mode (aa,bb,cc,dd,199,97).'

Why could this be happening?

UPDATE: Ok, I added the option --disable-epsv to my curl command and this fails, too. So the reason why python fails to list the directory is that is uses PASV. How can I force Python to use EPSV instead?

UPDATE2: This bug https://github.com/python/cpython/pull/28 seems to cause the hanging of Python ftplib when the IP address returned by PASV is wrong. This seems to be the problem in my case, too. But does anybody know a workaround how to force Python to use EPSV instead?

Answer 1

From reading the source code of ftplib I figured, that one can force ftplib to use EPSV by setting

ftps.af = socket.AF_INET6

This makes ftplib think that we are using an IPv6 connection (even if we are in fact using IPv4) and makes it use EPSV instead of PASV. My full working program was in the end

import ftplib
import socket

ftps = ftplib.FTP_TLS()

ftps.connect("blabla", 990)
ftps.login("xxx", "yyy")
ftps.prot_p()
ftps.cwd("out")
ftps.af = socket.AF_INET6
ftps.retrlines("LIST")
ftps.quit()