"IsADirectoryError: [Errno 21] Is a directory: " It is a file

Question

I already split the data into test and training set into the different folder. Now I need to load the patient data. Each patient has 8 images.

def load_dataset(root_dir, split):     """     load the data set numpy arrays saved by the preprocessing script     :param root_dir: path to input data     :param split: defines whether to load the training or test set     :return: data: dictionary containing one dictionary ({'data', 'seg', 'pid'}) per patient     """     in_dir = os.path.join(root_dir, split)     data_paths = [os.path.join(in_dir, f) for f in os.listdir(in_dir)]     data_and_seg_arr = [np.load(ii, mmap_mode='r') for ii in data_paths]     pids = [ii.split('/')[-1].split('.')[0] for ii in data_paths]     data = OrderedDict()     for ix, pid in enumerate(pids):         data[pid] = {'data': data_and_seg_arr[ix][..., 0], 'seg': data_and_seg_arr[ix][..., 1], 'pid': pid}     return data 

But, the error said:

File "/home/zhe/Research/Seg/heart_seg/data_loader.py", line 61, in load_dataset data_and_seg_arr = [np.load(ii, mmap_mode='r') for ii in data_paths] File "/home/zhe/Research/Seg/heart_seg/data_loader.py", line 61, in <listcomp> data_and_seg_arr = [np.load(ii, mmap_mode='r') for ii in data_paths] File "/home/zhe/anaconda3/envs/tf_env/lib/python3.6/site-packages/numpy/lib/npyio.py", line 372, in load fid = open(file, "rb") IsADirectoryError: [Errno 21] Is a directory: './data/preprocessed_data/train/Patient009969' 

It is already a file name, not a directory. Thanks!

Answer 1

It seems that ./data/preprocessed_data/train/Patient009969 is a directory, not a file.

os.listdir() returns both files and directories.

Maybe try using os.walk() instead. It treats files and directories separately, and can recurse inside the subdirectories to find more files in a iterative way:

data_paths = [os.path.join(pth, f)  [for pth, dirs, files in os.walk(in_dir) for f in files]