"UnicodeDecodeError: 'charmap' codec can't decode" error in pickle load

Question

I am trying to save an array of Tweet objects in file. Tweet class instances contain utf8 encoding characters. you can see the code below:

class Tweet:
    author='';
    text='';
    time='';
    date='';
    timestamp='';

with open('tweets.dat','wb') as f:
     pickle.dump(all_tweets,f)

with open('tweets.dat') as f:
   all_tweets = pickle.load(f)

When I run the code, it returns an exception on pickle.load(f) line stating that :

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 25: character maps to <undefined>

My machine specification:

Python 3.5.2 |Anaconda 4.2.0 (64-bit)| (default, Jul 5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)] on win32

Answer 1

In Python 3, the pickle module expects the underlying file objects to accept or return bytes. You correctly open the file in binary mode for writing, but failed to do the same for reading. The read part should be:

with open('tweets.dat', 'rb') as f:
   all_tweets = pickle.load(f)

Ref: extract from the documentation of pickle.load(fd):

...Thus file can be an on-disk file opened for binary reading, an io.BytesIO object, or any other custom object that meets this interface.