What is the best way to Serialize/Deserialize Class Objects in Python?

Question

I am trying to log some data from units that I am working with for long term and short term data collection and storage.

Should I use the json library or the pickle library and then how do I load the units' data from a file, be able to manipulate it, add units, etc., and then save the updated data back to the file?

Here is a detailed explanation of my problem and where I am stuck:

For each unit I have some data in a class object:

unit_1.name = 'unit_1'
unit_1.folder = 'C:\\Users\\Thomas\\Documents\\1'
unit_1.serial_number = 1
unit_1.offsets = [0.00, 0.00, 0.00, 0.00 ...]
unit_1.status = 'Open'
etc

I would like to store these objects for long term storage (so in a file), but I also need to be able to access them from the command line as a class object by doing something like this:

>python
Python 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

>from file import unit_log
>unit_log.load()
>unit_1.serial_number
1

and I would like to create new units like this:

>unit_2 = unit_log(2)

which would create the other fields in the init module of the class. After which I could save the units that I created or modified by doing something like:

>unit_log.save()

and it would save all the units to the file. Here is the code for the class:

file.py

class unit_log:
    def __init__ (self, serial_number):
        self.serial_number = serial_number;
        self.unit_name = 'unit_'+str(serial_number)
        self.status = 'Open';
        self.offsets = [];
        self.folder = 'C:\\Users\\Thomas\\Documents\\' + str(serial_number)
    @classmethod
    def load (self):
        #Something that loads the data
    @classmethod
    def save (self):
        #Something that saves the data

My current understanding is that what I am trying to do is called serialization/deserialization and that there are a couple different ways to do this (the big ones being the pickle library and the json library).

My question is: How do I deserialize a json file (or a pickle file) into a bunch of class objects? I am assuming the json file would look something like this:

unit_logs.json
{
    unit_1:{
    {
    "unit_name": "unit_1",
    "serial_number": 1,
    "status": "Open",
    "offsets": [],
    "folder": "C:\\Users\\Thomas\\Documents\\1"
    }
    unit_2...
    unit_3...
    ...
}

and I would load in the file using log = json.load(unit_logs.json). I would like to then have a

for unit in log:
    unit["unit_name"] = unit_log(unit)

but that is not how declaring a variable works.

I believe that the pickle library works similarly but I am less familiar with that one.

Should I use the json library or the pickle library and then how do I load the units' data from a file, be able to manipulate it, add units, etc., and then save the updated data back to the file?

Answer 1

pickle library is easy to use

Dump an object obj to file:

import pickle

with open(f_path, "wb") as f:
    pickle.dump(obj, f)

Load a pickle file:

with open(f_path, "rb") as f:
    obj = pickle.load(f)

Then you can change obj and write it again to file

In case you want to load the same pickle file several times and have different objects you can just:

N=<number of copies>

with open(f_path, "rb") as f:
    obj_list = [pickle.load(f) for _ in range(N)]

Now you can edit each object in obj_list separately, and if you want to save its state to a pickle file, you use pickle.dump as described above.

So you can have a "template" pickle file to load from, and then edit the objects as you want.