Python socket Recv not working properly could someone explain

Question

i am trying to create a gui client for my command line server. However, i am running into some annoying problems i cant seem to fix.

I'm not 100 % sure of what the actual problem is as sometimes the code will work, other times it wont. I think the main problem is that originally i tried the

while 1:
    self.data = s.recv(1024)
    if not self.data():
          break
    else:
          print self.data()

Then i was sending to it with this

  for f in files:
      s.send(f)

Each f was a string of a filename. I expected it to come out on the recv side as one file name recieved for each recv call but instead on one recv call i got a big chunk of filenames i assume 1024 chars worth

Which made it impossible to check for the end of the data and thus the loop never exited.

This is the code i have now

def get_data(self,size = 1024):

    self.alldata = ""
    while 1:

        while gtk.events_pending():
             gtk.main_iteration()

        self.recvdata = self.s.recv(size)
        self.alldata += self.recvdata
        if self.alldata.find("\r\n\r\nEOF"):
           print "recieved end message"
           self.rdata = self.alldata[:self.alldata.find("\r\n\r\nEOF")]
           break



    print "All data Recieved: " + str(len(self.rdata)) + "Bytes"
    print "All data :\n" + self.rdata + "\n-------------------------------------------------"  

    self.infiles = self.rdata.split("-EOS-")
    for nf in self.infiles:
        if len(nf) > 2:
            self.add_message(self.incomingIcon,nf)

At the minute im trying to get the client to read correctly from the server. What i want to happen is when the command list is typed in and sent to the client the server sends back the data and each file is appended to the list store

some times this works ok, other times only one of 1200 files gets returned, if it executes ok, if i try to type another command and send it , the whole gtk window geys out and the program becomes unresponsive.

Sorry i cant explain this question better, ive tried alot of different solutions all of which give different errors.

if someone could explain the recv command and why it may be giving the errors this is how im sending data to the client

if(commands[0] == 'list'):
    whatpacketshouldlooklike=""
    print "[Request] List files ", address
    fil = list_files(path)
    for f in fil:
         sdata =  f  
         whatpacketshouldlooklike += sdata + "-EOS-"
         newSock.send(sdata +"-EOS-")
         #print "sent: " + sdata
         newSock.send("\r\n\r\nEOF")
    whatpacketshouldlooklike += "\r\n\r\nEOF"
    print "---------------------------------"
    print whatpacketshouldlooklike
    print "---------------------------------"

Answer 1

The problem you had in the first part is that sockets are stream based not message based. You need to come up with a message abstraction to layer on top of the stream. This way the other end of the pipe knows what is going on(how much data to expect as a part of one command) and isn't guessing at what is supposed to happen.

Answer 2

Use an abstraction layer (Pyro, XML-RPC, zeromq) or define your own protocol to distinguish messages.

For example as own protocol you can send the length of a message as a "header" before each string. In this case you should use the struct module to parse the length into a binary format. Ask again, if you want to go this way, but I strongly recommend choosing one of the mentioned abstraction layers.