python: pickle.load() raising EOFError

Question

I have a pickle file using .txt format. I want to load this pickle file with python 2.7. The size is 438.5 MB. This is how I load data :

def readpickle(path="C:/Python27/Lib/site-packages/xy/"):
with open(path+"filenamereal2.txt","rb") as f:
    model = pickle.load(f)

return model

And I get this error

    itemmodelreal=readpickle(path="C:/Users/Lab Komputasi/Documents/estu/") 
Traceback (most recent call last):
File "<ipython-input-33-265e46f74915>", line 1, in <module>
    itemmodelreal=readpickle(path="C:/Users/Lab Komputasi/Documents/estu/")

  File "<ipython-input-31-fbd3e8b9e043>", line 3, in readpickle
    model = pickle.load(f)

  File "C:\Users\Lab Komputasi\Anaconda2\lib\pickle.py", line 1384, in load
    return Unpickler(file).load()

  File "C:\Users\Lab Komputasi\Anaconda2\lib\pickle.py", line 864, in load
    dispatch[key](self)

  File "C:\Users\Lab Komputasi\Anaconda2\lib\pickle.py", line 886, in load_eof
    raise EOFError

EOFError

this is the code that i use to write pickle :

 with open("filenamereal3.txt", "wb") as f:
    pickle.dump(result, f)
f.close()

I have used read binary ('rb') to load and write binary ('wb') to write, but it's still have that error. Do you have any idea why it's still error? how can i solve this error?

Thank you for your help....

Answer 1

I encountered the same error while loading a big file dumped in highest protocol.

This seems to be a bug of the pickle library. I solved it using cPickle instead.

import cPickle as pickle

Answer 2

To load data, wouldn't you want to be reading data ("rb") instead of writing data ("wb")?

Loading data should look like this:

 with open("C:/Users/Lab Komputasi/Documents/estu/filenamereal1.txt", "rb") as f:
     data = pickle.load(f)

Also, using f.close() is unnecessary because you are using a with/as clause.