Is `pickle.dump(d, f)` equivalent to `f.write(pickle.dumps(d))`?

Question

Given an arbitrary picklable Python data structure data, is

with open('a', 'bw') as f:
    f.write(pickle.dumps(data))

equivalent to

with open('a', 'bw') as f:
    pickle.dump(data, f)

i.e. can I assume this equivalence when writing code? Could you describe how you reached to this conclusion?

The use-case is to separate serialization from writing, since I may want to write pickle to a non-disk location.

Answer 1

Yes, they are equivalent. Looking at the source code of pickle.py:

def _dump(obj, file, protocol=None, *, fix_imports=True):
    _Pickler(file, protocol, fix_imports=fix_imports).dump(obj)

def _dumps(obj, protocol=None, *, fix_imports=True):
    f = io.BytesIO()
    _Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
    res = f.getvalue()
    assert isinstance(res, bytes_types)
    return res

dumps does the exact same thing as dump, just with an io.BytesIO object instead of a file object. It calls the internal _Pickler().dump() in the same way and simply returns the contents of the io.BytesIO object. Therefore, all f.write(pickle.dumps(data)) does is first forward the result to an io.BytesIO object then to the actual file instead of writing to the file directly.

Answer 2

They are not equivalent if an exception occurs during pickling. f.write(pickle.dumps(data)) will not write anything to the file. Whereas pickle.dump(data, f) will end up with a snapped or partial pickle.