Save dict to pickle or write to file?

Question

I have a Python script that returns a dict which I want to store somewhere to be used in a bigger project (the script is slow to run so I don't want to just import the script each time I want the dictionary).

The dict is small so I see two options. I can:

1. Save the dict to a pickle file

2. Write the dict as a literal to a new .py file, like so:

   my_dict = slow_func()
   with open('stored_dict.py', 'w') as py_file:
       file_contents = 'stored_dict = ' + str(my_dict)
       py_file.write(my_dict)
   

   Then I can access the dict literal using from stored_dict import stored_dict

Should I prefer one of these options?

Answer 1

Python dict is implemented like json. You can use the json module to dump a dict into file, and load it back easily:

import json

d = {1: 'a', 2: 'b', 3: 'c'}

with open('C:\temp.txt', 'w') as file:
    json.dump(d, file)

with open('C:\temp.txt', 'r') as file:
    new_d = json.load(file)

>>> new_d
{u'1': u'a', u'3': u'c', u'2': u'b'}

Answer 2

In my personal experience, I suggest using JSON if:

• You want a human-readable file,
• A file type that is compatible across multiple technologies without very much implementation overhead.
• An easily editable file (as it can simply be a text file).

I would suggest using Pickle if:

• The majority of the project will be written in python.
• The dictionary is likely to become very large/complex or rolled into a larger class.
• A non-human readable file is required.
• A file that is difficult to edit without prior knowledge of its construction.

Based on the situation you touch upon in your question, JSON would be the more beneficial choice.