List files with pyftp - proftpd vs. pyftpdlib behavior

Question

I have a test code which uses an FTP stub with pyftpdlib, which to my surprise failed in production. The reason for this is that proftpd returns the directory name in response to NLST. Here is the response from pyftpdlib FTP stub:

In [10]: local_conn.login('user', '12345')
Out[10]: '230 Login successful.'

In [11]: import ftplib

In [12]: local_conn = ftplib.FTP()

In [13]: local_conn.connect('localhost', 2121)
Out[13]: '220 pyftpdlib 1.4.0 ready.'

In [14]: local_conn.login('user', '12345')
Out[14]: '230 Login successful.'

In [15]: local_conn.nlst('structuredata_advanced')
Out[15]: 
['Report_20150618.csv',
 'Report_20150618.fin',
 'Report_20150619.csv',
 'Report_20150619.fin',
 'Report_20150620.csv',
 'Report_20150620.fin']

Here is the response from proftpd:

In [16]: remote_conn = ftplib.FTP()

In [17]: remote_conn.connect('A1B.7Y.XX.XX', 21)
Out[17]: '220 ProFTPD 1.3.4a Server (vztd3.company.com) [A1B.7Y.XX.XX]'

In [18]: remote_conn.login('remoteuser', 'verysecret')
Out[18]: '230 User yougov logged in'

In [19]: remote_conn.nlst('structuredata_advanced')
Out[19]: 
['structuredata_advanced/Report_20150624.csv',
 'structuredata_advanced/Report_20150629.csv',
 'structuredata_advanced/Report_20150625.fin',
 'structuredata_advanced/Report_20150628.fin',
 'structuredata_advanced/Report_20150627.fin',
 'structuredata_advanced/Report_20150620.fin',
 'structuredata_advanced/Report_20150619.csv', 
  ...]

It's easy enough to remove those directory names:

    # this code works both in production and testing 
    files = conn.nlst(basedir)
    # proftd is weired it returns the basedir name too
    files = [f.split('/')[-1] for f in files]

but I would like to understand if this is something that pyftpdlib does wrong?
Is this something that can be configured in ProFTPD?
Is there something I need to know about the FTP protocol and NLST command?

update

I just tested another ftp server called uftpd it behaves like pyftpdlib when issuing NLST.

Answer 1

I'm the author of uftpd.

I just googled a bit, turns out DJB wrote aboute this a while back, and unfortunately it seems output differs between servers.

My interpretation though is; that it is recommended to not prefix each output file in a given directory with the directory name. I.e., if the client sends 'NLST dir' the server should not reply:

dir/a
dir/b

but instead simply output the files in directory dir/ like this:

a
b