FileNotFoundError while writing to a Pickle file

Question

I'm taking all the filenames in a certain directory in a list, and would like to write this list in a pickle file. Here's the code I am using:

import _pickle as pickle
with open(filepath+'filenames.pkl', 'wb') as f:
    pickle.dump(filenames, f)

This gives me the following error:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-32-c59e6889d2fe> in <module>()
      1 import _pickle as pickle
----> 2 with open(dpath+'filenames.pkl', 'wb') as f:
      3     pickle.dump(filenames, f)

FileNotFoundError: [Errno 2] No such file or directory: '/data/train/filenames.pkl'

I am supposed to create this file. Why is this file expected already?

(I'm using Python 3.6.)

Answer 1

In your case most probably /data/train/ directory does not exist

I tried this code and got same error:

import pickle as pickle
filenames='sadasdas'
with open('/tmp/not_exist/filenames.pkl', 'wb') as f:
    pickle.dump(filenames, f)

output:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-7-6a22c2148796> in <module>()
      1 import pickle as pickle
      2 filenames='sadasdas'
----> 3 with open('/tmp/not_exist/filenames.pkl', 'wb') as f:
      4     pickle.dump(filenames, f)

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/not_exist/filenames.pkl'

You can handle this problem by making sure the directory exists before writing to file: Programatic approach to do the same is this

import os
filename = "/tmp/not_exist/filenames.pkl"
os.makedirs(os.path.dirname(filename), exist_ok=True)
data = 'sadasdas'
with open('/tmp/not_exist/filenames.pkl', 'wb') as f:
    pickle.dump(data, f)

ref: https://stackoverflow.com/a/12517490/3027854