I am trying to write some result on to pickle file as below:
raw_X = (self.token_ques(text) for text in training_data)
with open('/root/Desktop/classifier_result.pkl', 'wb') as handle:
pickle.dump(raw_X, handle)
Error:
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle generator objects
Any help would be much appreciable.
Don't use a generator expression when you want to pickle data. Use a list comprehension instead, or call list()
on the generator to capture all generated elements for pickling.
For example, the following works just fine:
raw_X = [self.token_ques(text) for text in training_data]
with open('/root/Desktop/classifier_result.pkl', 'wb') as handle:
pickle.dump(raw_X, handle)
as does:
raw_X = (self.token_ques(text) for text in training_data)
with open('/root/Desktop/classifier_result.pkl', 'wb') as handle:
pickle.dump(list(raw_X), handle)
raw_X = (self.token_ques(text) for text in training_data)
This is a generator. As the error says, we cannot pickle generators. Use this instead.
raw_X=[]
for text in data:
raw_X.append(self.token_ques(text))
raw_X=tuple(raw_X)
And pickle raw_X
then.
Edit
This works for me
import pickle
raw_X=[]
data=[1,2,3,4,5,6,2,0]
for text in data:
raw_X.append(str(text))
print pickle.dumps(raw_X)
I'm using str()
instead of your function and dumps()
instead of dump()
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With