I am using Python to develop an application that does the following:
The main issue I have developing this application is knowing when the file has finished transferring. From what I know the file will be transferred via SFTP to a particular directory. How will Python know when the file has finished transferring? I know that I can use the st_size
attribute from the object returned by os.stat(fileName)
method. Are there more tools that I need to use to accomplish these goals?
There is no way to be absolutely certain a file being written by the FTP daemon is complete. It's even possible that the file transfer failed and then gets restarted and completed. You must poll the file's size and set a time limit, say 5 minutes.
FTP(File Transfer Protocol) To transfer a file, 2 TCP connections are used by FTP in parallel: control connection and data connection. For uploading and downloading the file, we will use ftplib Module in Python. It is an in-built module in Python.
Here, we first move to the targeted directory using the conn.cd() method. Then, we just pass the file name to the conn. get() method. This leads to the downloading of that file into the local directory of our client-side local machine.
I ended up using a combination of watchdog and waited until I can open the file for writing
#If there is no error when trying to read the file, then it has completely loaded
try:
with io.FileIO(fileName, "r+") as fileObj:
'''
Deal with case where FTP client uses extensions such as ".part" and '.filepart" for part of the incomplete downloaded file.
To do this, make sure file exists before adding it to list of completedFiles.
'''
if(os.path.isfile(fileName)):
completedFiles.append(fileName)
print "File=" + fileName + " has completely loaded."
except IOError as ioe:
print str(ioe)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With