How to save data with Python?

Question

I am working on a program in Python and want users to be able to save data they are working on. I have looked into cPickle; it seems like it would be a fast and easy way to save data, it seems insecure. Since entire functions, classes, etc can be pickled, I am worried that a rogue save file could inject harmful code into the program. Is there a way I can prevent that, or should I look into other methods of saving data, such as directly converting to a string (which also seems insecure,) or creating an XML hierarchy, and putting data in that.

I am new to python, so please bear with me.

Thanks in advance!

EDIT: As for the type of data I am storing, it is mainly dictionaries and lists. Information such as names, speeds, etc. It is fairly simple right now, but may get more complex in the future.

Answer 1

From your description JSON encoding is the secure and fast solution. There is a json module in python2.6, you can use it like this:

import json
obj = {'key1': 'value1', 'key2': [1, 2, 3, 4], 'key3': 1322}
encoded = json.dumps(obj)
obj = json.loads(encoded)

JSON format is human readable and is very similar to the dictionary string representation in python. And doesn't have any security issues like pickle. If you don't have python2.6 you can install cjson or simplejson

You can't use JSON to save python objects like Pickle. But you can use it to save: strings, dictionaries, lists, ... It can be enough for most cases.

To explain why pickle is insecure. From python docs:

Most of the security issues surrounding the pickle and cPickle module involve unpickling. There are no known security vulnerabilities related to pickling because you (the programmer) control the objects that pickle will interact with, and all it produces is a string.

However, for unpickling, it is never a good idea to unpickle an untrusted string whose origins are dubious, for example, strings read from a socket. This is because unpickling can create unexpected objects and even potentially run methods of those objects, such as their class constructor or destructor ... The moral of the story is that you should be really careful about the source of the strings your application unpickles.

There are some ways to defend yourself but it is much easier to use JSON in your case.

Answer 2

You could do something like:

to write

• Pickle
• Sign pickled file
• Done

to read

• Check pickled file's signature
• Unpickle
• Use

I wonder though what makes you think that the data files are going to be tampered but your application is not going to be?