Menu
Note: that I have both blocks of code (see below) in the same .rb file. The first time ftp.getbinaryfile() works then it throws the error.
Note: that file variable is a static path to the file used for debugging purposes only.
I have this code in ruby 2.0.0p481 (2014-05-08) [x64-mingw32]
file = "/Filetrack/E-mail_Gateway/_Installer/GA/E-mail Gateway_10.0_Changes_PUBLIC.pdf"
list = ftp.list('*')
list.each{|item|
counter=counter+1
counter++
ftp.getbinaryfile(file, where_to_save+File.basename(file)+counter.to_s, 1024)
puts "downloaded - .each used"
}
then in the same .rb file I got this code
ftp.list('*') { |item|
puts "downloading using .list('*') {"
counter++
ftp.getbinaryfile(file, where_to_save+File.basename(file)+counter.to_s, 1024)
puts "downloaded #{file}"
}
and that code throws me this error
Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:974:in `parse227': 200 Type set to I. (Net::FTPReplyError)
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:394:in `makepasv'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:406:in `transfercmd'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:487:in `block (2 levels) in retrbinary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:199:in `with_binary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:485:in `block in retrbinary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:484:in `retrbinary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:617:in `getbinaryfile'
ftp session is created by
ftp = Net::FTP.new('ftp.***.***.net')
ftp.passive = false
ftp.debug_mode = true
ftp.login(ftp_username, ftp_password)
could someone explain why the second version works?
Added ftp debugging log:
put: USER r.***
get: 331 Password required for r.***.
put: PASS ************
get: 230-Welcome to FTP
get: 230 User r.****logged in.
put: TYPE I
get: 200 Type set to I.
put: CWD /Filetrack/E-mail_Gateway/_Installer/GA/010_000_003_000/
get: 250 CWD command successful.
put: TYPE A
get: 200 Type set to A.
put: PASV
get: 227 Entering Passive Mode (194,212,10,23,195,92).
put: LIST *
get: 125 Data connection already open; Transfer starting.
get: 226 Transfer complete.
put: TYPE I
get: 200 Type set to I.
put: PASV
get: 227 Entering Passive Mode (194,212,10,23,195,93).
put: RETR /Filetrack/E-mail_Gateway/_Installer/GA/010_000_003_000/E-mail Gateway_10.0_Changes_PUBLIC.pdf
get: 125 Data connection already open; Transfer starting.
get: 226 Transfer complete.
downloaded - .each used
put: TYPE A
get: 200 Type set to A.
put: PASV
get: 227 Entering Passive Mode (***,***,10,23,195,97).
put: LIST *
get: 125 Data connection already open; Transfer starting.
downloading using .list('*') {
put: TYPE I
get: 226 Transfer complete.
put: PASV
get: 200 Type set to I.
put: TYPE A
get: 227 Entering Passive Mode (***,***,10,23,195,98).
put: TYPE I
get: 200 Type set to A.
d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:974:in `parse227': 200 Type set to I. (Net::FTPReplyError)
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:394:in `makepasv'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:406:in `transfercmd'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:487:in `block (2 levels) in retrbinary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:199:in `with_binary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:485:in `block in retrbinary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:484:in `retrbinary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:617:in `getbinaryfile'
from download2 - debugging.rb:41:in `block in <main>'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:518:in `block (3 levels) in retrlines'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:515:in `loop'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:515:in `block (2 levels) in retrlines'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:199:in `with_binary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:512:in `block in retrlines'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:511:in `retrlines'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:760:in `list'
from download2 - debugging.rb:38:in `<main>'
UPDATE2
log if ftp.passive = false is used
downloading using .list('*') {
put: TYPE I
get: 226 Transfer complete.
put: PORT ***,***,20,102,235,136
get: 200 Type set to I.
put: RETR /Filetrack/E-mail_Gateway/_Installer/GA/010_000_003_000/Email Gateway_10.0_Changes_PUBLIC.pdf
get: 200 PORT command successful.
put: TYPE A
put: TYPE I
d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/protocol.rb:211:in `write': An existing connection was forcibly closed by the remote host. (Errno::ECONNRESET)
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/protocol.rb:211:in `write0'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/protocol.rb:185:in `block in write'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/protocol.rb:202:in `writing'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/protocol.rb:184:in `write'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:283:in `putline'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:360:in `block in voidcmd'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:359:in `voidcmd'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:183:in `send_type_command'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:172:in `binary='
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:201:in `ensure in with_binary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:201:in `with_binary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:512:in `block in retrlines'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:511:in `retrlines'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:760:in `list'
from download2 - debugging.rb:39:in `<main>'
UPDATE3
I tried to run the same code in ftp active mode few times and actually all files are downloaded but the script finishes with an error.
downloading using .list('*') {
put: TYPE I
get: 200 Type set to I.
put: PORT **,**,20,102,197,73
get: 200 PORT command successful.
put: RETR /Filetrack/E-mail_Gateway/_Installer/GA/010_000_003_000/E-mail Gateway_10.0_Changes_PUBLIC.pdf
get: 150 Opening BINARY mode data connection for /Filetrack/E-mail_Gateway/_Installer/GA/010_000_003_000/E-mail Gateway_10.0_Changes_PUBLIC.pdf(
60911 bytes).
get: 226 Transfer complete.
put: TYPE A
get: 200 Type set to A.
downloaded /Filetrack/E-mail_Gateway/_Installer/GA/010_000_003_000/E-mail Gateway_10.0_Changes_PUBLIC.pdf
put: TYPE I
get: 200 Type set to I.
d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/protocol.rb:158:in `rescue in rbuf_fill': Net::ReadTimeout (Net::ReadTimeout)
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/protocol.rb:152:in `rbuf_fill'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/protocol.rb:134:in `readuntil'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:1108:in `readline'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:289:in `getline'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:300:in `getmultiline'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:318:in `getresp'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:338:in `voidresp'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:526:in `block (2 levels) in retrlines'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:199:in `with_binary'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:512:in `block in retrlines'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:511:in `retrlines'
from d:/prog/Ruby200-x64/lib/ruby/2.0.0/net/ftp.rb:760:in `list'
from download2 - debugging.rb:39:in `<main>'
832
Your connection uses PASSIVE mode. Since you have not shown the part of code you create FTP object, I will assume that you setting mode to passive explicitly.
ftp = Net::FTP.new('example.com')
ftp.passive = true
Based on the stack trace of exception, one can see that issue happens when the method makepasv issues PASV command, but instead of getting a response of 227 Entering Passive Mode (194,212,10,23,195,93)., it gets a response 200 Type set to I.
Implementation of makepasv and parse227 (Refer Reference 1 & Reference 2 later in the post) indicates that code specifically looks for return code of 227 and if that is not the case, it will throw an FTPError.
This is what is happening in the given scenario.
This can be attributed to the syntax shown below.
This may very well be not so well understood behavior (as I myself discovered during the course answering to this post)
ftp.list('*') { |item|
ftp.getbinaryfile(file, where_to_save+File.basename(file)+counter.to_s, 1024)
}
In the above code, a LIST * command is issued by ftp.list('*'). Typical response for this command would like below:
put: LIST *
get: 125 Data connection already open; Transfer starting.
get: 226 Transfer complete.
As can be seen, LIST * produces two lines of result. This fact is crucial to understand the issue.
The block passed to ftp.list('*') downloads a binary file using getbinaryfile method.
getbinaryfile will typically issue below commands:
TYPE I to put the connection in image (binary) modePASVto enter passive modeRETR /path/of/file/to/downloadWhen the block executes for the first result of ftp.list('*'), and starts issuing commands related to getbinaryfile, at that point of time, only first line of response of LIST * has been read - the second line is yet to be read. It is this second line that shows up as response to next command issued in the block.
Hence, when first command TYPE I is issued, the code reads the second line of LIST * as response (as evident in debug logs)
put: TYPE I
get: 226 Transfer complete.
When second command PASV is issued, the code reads the response of TYPE I (as evident from debug logs)
put: PASV
get: 200 Type set to I.
Implementation of makepasv is such that it expects that response to have response code of 227 (Refer line 394 and 973 in Reference 1 and Reference 2 respectively). An exception Net::FTPReplyError was thrown gets thrown in this case as parse227 was passed a response of TYPE I command.
In summary, when using passive mode, it seems that it is not feasible to perform other FTP operations in the block given to `ftp.list('*')
In this case, the ftp.list('*') is invoked without a block, and hence it returns the Array of strings as output. Using each on that array does not create the similar situation - and hence, there are no issues observed.
It seems that author(s) of FTP#list expected the below two variants to work in equivalent manner:
ftp.list('*') { |f| } # block given to list
ftp.list('*').each { |f| } # block given to enum returned by list
As per official documentation of list API:
list(*args) { |line| ... }
Returns an array of file information in the directory (the output is like
ls -l). If a block is given, it iterates through the listing.
If we look at the implementation of list, then, we see that when a block is given, each line read from ftp.list('*') is yielded to the block one by one. When using passive mode, if the block tries to execute any other FTP commands, this causes the above mentioned.
754 def list(*args, &block) # :yield: line
755 cmd = "LIST"
756 args.each do |arg|
757 cmd = cmd + " " + arg.to_s
758 end
759 if block
760 retrlines(cmd, &block)
761 else
762 lines = []
763 retrlines(cmd) do |line|
764 lines << line
765 end
766 return lines
767 end
768 end
We can solve this problem by changing the implementation to become equivalent to ftp.list('*').each variant by first collecting all lines from LIST * response into an array, and the passing that array to the block if a block was given. We will still stay true to the API documentation.
def list(*args, &block) # :yield: line
cmd = "LIST"
args.each do |arg|
cmd = cmd + " " + arg.to_s
end
# First lets fetch all the lines
lines = []
retrlines(cmd) do |line|
lines << line
end
if block
lines.each { |l| yield l }
else
return lines
end
end
I have reported a bug in Ruby Bug Tracker suggesting above change in implementation of FTP#list method.
Reference 1 - Implementation of makepasv
391 # sends the appropriate command to enable a passive connection
392 def makepasv # :nodoc:
393 if @sock.peeraddr[0] == "AF_INET"
394 host, port = parse227(sendcmd("PASV"))
395 else
396 host, port = parse229(sendcmd("EPSV"))
397 # host, port = parse228(sendcmd("LPSV"))
398 end
399 return host, port
400 end
Reference 2 - Implementation of parse227
968 # handler for response code 227
969 # (Entering Passive Mode (h1,h2,h3,h4,p1,p2))
970 #
971 # Returns host and port.
972 def parse227(resp) # :nodoc:
973 if resp[0, 3] != "227"
974 raise FTPReplyError, resp
975 end
976 if m = /\((?<host>\d+(,\d+){3}),(?<port>\d+,\d+)\)/.match(resp)
977 return parse_pasv_ipv4_host(m["host"]), parse_pasv_port(m["port"])
978 else
979 raise FTPProtoError, resp
980 end
981 end
Source code snippets were taken from ftp.rb.
UPDATE: 13 Sep, 2015 The proposed change has been accepted by Ruby core team for this issue.
78
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
