Python socket send EOF

Question

I have a simple file transfer socket program where one socket sends file data and another socket receives the data and writes to a file

I need to send an acknowledgment once transfer is finished from the destination to the source

Code for destination

s.accept()
f = s.makefile()
f.read(1024)

Code for source

s.connect(('localhost',6090))
f = s.makefile()
f.write('abcd') 
f.flush()

Here comes the problem, since "abcd" is not 1024 bytes, the destination will block till 1024 bytes are received
The solution would be to close() the socket, but since I need a confirmation from destination, I cannot close the socket.
How do I tell the destination to stop blocking? I was thinking of writing an EOF character I read online that "\x04" is EOF, but it doesn't work.
Also since the data could be binary I don't want to use the readline() method.

Answer 1

Design a protocol (an agreement between client and server) on how to send messages. One simple way is "the first byte is the length of the message, followed by the message". Rough example:

Client

Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from socket import *
>>> s=socket()
>>> s.connect(('localhost',5000))
>>> f=s.makefile()
>>> f.write('\x04abcd')
>>> f.flush()

Server

Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from socket import *
>>> s=socket()
>>> s.bind(('localhost',5000))
>>> s.listen(1)
>>> c,a=s.accept()
>>> f=c.makefile()
>>> length=ord(f.read(1))
>>> f.read(length)
'abcd'

Answer 2

Writing and reading from a socket are separate. You can try to close a socket for writing and leave it open for reading.

See http://docs.python.org/library/socket.html#socket.socket.shutdown

Also, what FTP does is use two sockets: one for data, and one for this "confirmation".

You'd be happier using a second socket for this additional metadata.

Answer 3

You don't actually want to use an 'EOF' marker if you intend to send any binary data through the pipe. What happens if your binary data happens to include that byte sequence? You could 'escape' it to prevent that from happening, but that requires you to check on both ends, and then make sure you escape your escape sequence as well.

If you are doing this for some kind of production system, look around to see if there's a networking API that you can use (something like FTP, HTTP, SFTP, etc).

If this is a school or small-scale project, what you need to do is write a communications protocol. The simplest might be for the sender to broadcast a single binary integer X (pick a data size, and stick with it), and then send X bytes of binary data. The receiver first snags the size, then only expects to receive that amount of data. Remember that if you need to send more bytes than the data size of X can send, you'll need to 'chunk' your data.

Also consider what happens when communications are lost mid-transfer. You can set timeouts on read(), but then you need to recover gracefully if communications resumes immediately after your timeout (although junking all of the data might be considered graceful, depending on the system).