Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

python 3.6 socket pickle data was truncated

I can not send my numpy array in socket. I use pickle but my client pickle crashes with this error: pickle data was truncated

My server : I create a numpy array and I want to send in my client with pickle (it's work)

import socket, pickle
import numpy as np
from PIL import ImageGrab
import cv2


while(True):
    HOST = 'localhost'
    PORT = 50007
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 4096)
    s.bind((HOST, PORT))
    s.listen(1)
    conn, addr = s.accept()
    print ('Connected by', addr)

    arr = np.array([[0, 1], [2, 3]])
    printscreen_pil=ImageGrab.grab(bbox=(10,10,500,500))
    img = np.array(printscreen_pil) ## Transform to Array
    
    data_string = pickle.dumps(img)
    conn.send(data_string)

    msg_recu = conn.recv(4096)
    print(msg_recu.decode())

    conn.close()

My client He has my numpy array, but I can not load with pickle. I have this error.

import socket, pickle
import numpy as np

HOST = 'localhost'
PORT = 50007
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))

msg_a_envoyer = "hello".encode()
s.send(msg_a_envoyer)


while 1:
    data = s.recv(4096)
    if not data: break
    data_arr = pickle.loads(data)
    print (data_arr)
s.close()
like image 227
sazearte Avatar asked Jun 19 '17 18:06

sazearte


2 Answers

the problem is that if the size of the pickled data is > 4096 you only get the first part of the pickled data (hence the pickle data was truncated message you're getting)

You have to append the data and pickle it only when the reception is complete, for example like this:

data = b""
while True:
    packet = s.recv(4096)
    if not packet: break
    data += packet

data_arr = pickle.loads(data)
print (data_arr)
s.close()

increasing a bytes object is not very performant, would be better to store the parts in a list of objects, then join, though. Faster variant:

data = []
while True:
    packet = s.recv(4096)
    if not packet: break
    data.append(packet)
data_arr = pickle.loads(b"".join(data))
print (data_arr)
s.close()
like image 195
Jean-François Fabre Avatar answered Nov 02 '22 11:11

Jean-François Fabre


In the most Simple words. the file that you are trying to load is not complete. Either you have not downlaoded it correctly or it's just that your pickle file is corrupt. You can create a new pickle to solve this issue

like image 28
CreatorGhost Avatar answered Nov 02 '22 10:11

CreatorGhost