Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can I pickle Python objects in memory instead of a physical file? [duplicate]

I have seen and written code to pickle objects in Python. But they all create physical file to contain data. I want that I write the data in memory and read it and pickle it and transfer it.

Is it even possible ?

from PIL import ImageGrab
import io
import codecs
import pickle


# Model class to send Process Data to the server
class ProcessData:
 process_id = 0
 project_id = 0
 task_id = 0
 start_time = 0
 end_time = 0
 user_id = 0
 weekend_id = 0

# Model class to send image data to the server
class ProcessScreen:
 process_id = 0
 image_data = bytearray()


image_name = "Dharmindar_screen.jpg"
ImageGrab.grab().save(image_name,"JPEG")

image_data = None
with codecs.open(image_name,'rb') as image_file:
 image_data = image_file.read()



serialized_process_data = io.BytesIO()

process_data = ProcessData()
process_data.process_id = 1
process_data.project_id = 2
process_data.task_id = 3
process_data.user_id = 4
process_data.weekend_id = 5
process_data.start_time = 676876
process_data.end_time = 787987

process_screen = ProcessScreen()
process_screen.process_id = process_data.process_id
process_screen.image_data = image_data


prepared_process_data = (process_data, process_screen)

process_data_serializer = pickle.Pickler()
process_data_serializer(serialized_process_data).dump(prepared_process_data)

print('Data serialized.')


if process_data_serializer is not None:
    d = process_data_serializer.getvalue()
    deserialized_data = None
    with open(d, 'rb') as serialized_data_file:
        process_deserializer = pickle.Unpickler(serialized_data_file)
        deserialized_data = process_deserializer.load()
else:
    print('Empty')

The above code throws the TypeError: Required argument 'file' (pos 1) not found

like image 743
Dharmindar Avatar asked Nov 15 '17 12:11

Dharmindar


People also ask

Can all Python objects be pickled?

Python pickle module is used for serializing and de-serializing a Python object structure. Any object in Python can be pickled so that it can be saved on disk.

Can you pickle a Python object?

Python comes with a built-in package, known as pickle , that can be used to perform pickling and unpickling operations. Pickling and unpickling in Python is the process that is used to describe the conversion of objects into byte streams and vice versa - serialization and deserialization, using Python's pickle module.

What objects Cannot be pickled in Python?

With pickle protocol v1, you cannot pickle open file objects, network connections, or database connections.

Can pickle save objects?

Pickling is a method to convert an object (list, dict, etc) to a file and vice versa. The idea is to save one or more objects in one script and load them in another. You can also use it to save program or game states.


1 Answers

The File object you pass to pickle.dump just needs a write method, see https://docs.python.org/2/library/pickle.html#pickle.dump

file must have a write() method that accepts a single string argument. It can thus be a file object opened for writing, a StringIO object, or any other custom object that meets this interface.

Instead of using a StringIO object, you could even create your own class to store the pickled data, for example

class MyFile(object):
    def __init__(self):
        self.data = []
    def write(self, stuff):
        self.data.append(stuff)

and then just pickle to an instance of this class:

class ExampleClass(object):
    def __init__(self, x):
        self.data = x  
a = ExampleClass(123)
f = MyFile()
pickle.dump(a, f)

Another option is, as proposed by @rawing, is to use pickle.dumps which will directly return a string that you can use, compare here.

like image 100
ezdazuzena Avatar answered Sep 27 '22 23:09

ezdazuzena