Menu
My code is experiencing a time out exception after exactly 30 minutes when downloading a large file over FTP. The server is FileZilla running on Windows. We have an SSL certificate configured with options Enable FTP over SSL/TLS support (FTPS) and Allow explicit FTP over TLS enabled. I have access to the server and the FileZilla configuration but can't see anything that might cause this behavior. Below is the source code which is running on .NET 4.6.2 on a Windows 2012 Server Machine. It can download files from the FTP server but will time out with the exception (listed below) after exactly 30 minutes if the file takes longer than 30 minutes to download.
As a test I have used FileZilla Client, running from the same client PC, to download multiple large files from the same server end point simultaneously so that the download for each file took over 30 minutes to complete. No errors occurred in this scenario.
I have searched on StackOverflow as well as Google but nothing promising turned up. If anyone has some tips on where to look (server side or client side) I would be most appreciative.
public class FtpFileDownloader
{
// log4net
private static readonly ILog Logger = LogManager.GetLogger(typeof(FtpFileDownloader));
public void DownloadFile()
{
// setting the SecurityProtocol did not change the outcome, both were tried. Originally it was not set at all.
// ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
ServicePointManager.ServerCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) => true;
const int timeout = 7200000;
const string file = "some-existing-file";
try
{
var request = (FtpWebRequest) WebRequest.Create("uri-path-to-file");
request.KeepAlive = false;
request.Timeout = -1;
request.ReadWriteTimeout = timeout;
request.Credentials = new NetworkCredential("userName", "password");
request.UsePassive = true;
request.EnableSsl = true;
request.Method = WebRequestMethods.Ftp.DownloadFile;
Logger.Debug($"Downloading '{file}'");
using (var response = (FtpWebResponse) request.GetResponse())
using (var sourceStream = response.GetResponseStream())
using (var targetStream = new FileStream("some-target-on-disk", FileMode.Create, FileAccess.Write))
{
try
{
sourceStream.CopyTo(targetStream);
targetStream.Flush();
Logger.Debug($"Finished download '{file}'");
}
catch (Exception exInner)
{
Logger.Error($"Error occurred trying to download file '{file}'.", exInner);
}
}
}
catch (Exception ex)
{
Logger.Error($"Error occurred trying to dispose streams when downloading file '{file}'.", ex);
}
}
}
ERROR FtpFileDownloader - Error occurred trying to download file 'some-existing-file'.
System.IO.IOException: Received an unexpected EOF or 0 bytes from the transport stream.
at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Security._SslStream.StartFrameBody(Int32 readBytes, Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.TlsStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.FtpDataStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.Stream.InternalCopyTo(Stream destination, Int32 bufferSize)
at System.IO.Stream.CopyTo(Stream destination)
at FtpFileDownloader.DownloadFile
ERROR FtpFileDownloader - Error occurred trying to dispose streams when downloading file 'some-existing-file'.
System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
at System.Net.FtpWebRequest.RequestCallback(Object obj)
at System.Net.CommandStream.Dispose(Boolean disposing)
at System.IO.Stream.Close()
at System.IO.Stream.Dispose()
at System.Net.ConnectionPool.Destroy(PooledStream pooledStream)
at System.Net.ConnectionPool.PutConnection(PooledStream pooledStream, Object owningObject, Int32 creationTimeout, Boolean canReuse)
at System.Net.FtpWebRequest.FinishRequestStage(RequestStage stage)
at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
at System.Net.FtpWebRequest.RequestCallback(Object obj)
at System.Net.CommandStream.Abort(Exception e)
at System.Net.CommandStream.CheckContinuePipeline()
at System.Net.FtpWebRequest.DataStreamClosed(CloseExState closeState)
at System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
at System.Net.FtpDataStream.Dispose(Boolean disposing)
at System.IO.Stream.Close()
at System.IO.Stream.Dispose()
at FtpFileDownloader.DownloadFile
Listen on these ports: 21
Max. number of users: 0 (infinite)
Number of threads: 2
Connection timeout: 120 (seconds)
No Transfer timeout: 9000 (seconds)
Log timeout: 60 (seconds)
23-2-2018 11:40:40 - (not logged in) (194.123.75.2)> Connected on port 21, sending welcome message...
23-2-2018 11:40:40 - (not logged in) (194.123.75.2)> 220 Welcome
23-2-2018 11:40:40 - (not logged in) (194.123.75.2)> AUTH TLS
23-2-2018 11:40:40 - (not logged in) (194.123.75.2)> 234 Using authentication type TLS
23-2-2018 11:40:40 - (not logged in) (194.123.75.2)> TLS connection established
23-2-2018 11:40:40 - (not logged in) (194.123.75.2)> USER my-user-account
23-2-2018 11:40:40 - (not logged in) (194.123.75.2)> 331 Password required for my-user-account
23-2-2018 11:40:40 - (not logged in) (194.123.75.2)> PASS **************
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> 230 Logged on
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> PBSZ 0
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> 200 PBSZ=0
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> PROT P
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> 200 Protection level set to P
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> OPTS utf8 on
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> 202 UTF8 mode is always enabled. No need to send this command.
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> PWD
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> 257 "/" is current directory.
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> TYPE I
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> 200 Type set to I
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> PASV
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> 227 Entering Passive Mode (IP-ADDRESS,245,222)
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> RETR path-to-file
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> 150 Opening data channel for file download from server of "/path-to-file"
23-2-2018 11:40:40 - my-user-account (194.123.75.2)> TLS connection for data connection established
23-2-2018 12:10:41 - my-user-account (194.123.75.2)> disconnected.
Notice that there is 30 minutes between the disconnect (last line) and the line before it. If it had completed the transfer with success there would also be a line that read 226 Successfully transferred "/path-to-file" before the disconnected line
Progress update:
0.9.43 beta (release date 2014-01-02 according to the change log). Although I did not find anything in the change log that suggests this behavior was ever a fixed bug I am going to upgrade to the latest version 0.9.60.2 (release date 2017-02-08) and execute the test again. Will report back within 24 hours.1800 seconds. A rule has been added to override this between these 2 end points.It turned out that the company firewall (network where the client executes) and not the hosting firewall (network where FileZilla is hosted) was the culprit. It was configured to drop idle connections after 1800 seconds. A rule has been added to override this between these 2 end points.
648
In the Connections pane, click the server name, and then click the Sites node. In the Sites pane, click Set FTP Site Defaults... in the Actions pane. In the Advanced Settings dialog box, expand Connections, specify your time-outs in the Control Channel Timeout and Data Channel Timeout fields, and then click OK.
“FTP connection timed out”This happens when your Internet Service Provider is blocking the FTP port – port 21. More information on how to check if a port is blocked can be found here.
FTP server subcommand The number of seconds the server waits before ending the connection with the client. Inactivity timeout values can range from 1-9,999,999 seconds. The default inactivity timeout value is 300 seconds. The file transfer timeout in seconds.
Probably you should try another implementation of FTP protocol client that isn't built on top of the FtpWebRequest.
Related issues exist for a long time, they don't have a clear solution or answer. So I would try something like FluentFTP, it uses Winsock API directly. XML Documentation Comment states that DownloadFile() should handle large file downloads well:
/// <summary>
/// Downloads the specified file onto the local file system.
/// High-level API that takes care of various edge cases internally.
/// Supports very large files since it downloads data in chunks.
/// </summary>
For more information check:
97
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
