How to load multiple .npy files into a single numpy array

Question

I want to load multiple numpy file and putting them in array like this ["file1.npy","file2.npy","file3.npy",......] to apply pca dimensionality reduction on this array.

Any help would be appreciated

code

k=1
for indexPatient in range(0, len(patients)): 
    interictalData_withoutpca=np.concatenate((interictalData, tmpData[0:22,start*256:end]), axis=1)
    x=np.array(interictalData_withoutpca)
    y=np.save('interictalData_matrix'+str(k)+'_'+patients[indexPatient]+'_'+str(l),x)
    k+=1

Answer 1

The easiest way is:

filenames = ["file1.npy", "file2.npy", "file3.npy"]
combined_data = np.array([np.load(fname) for fname in filenames])

This requires that the arrays stored in each of the files have the same shape; otherwise you get an object array rather than a multidimensional array.

If it's a large amount of data and you know the shape of the data, say (n1, n2), it's more efficient (in speed and memory) to preallocate the data:

combined_data = np.zeros((len(filenames), n1, n2))
for i, fn in enumerate(filenames):
   combined_data[i, :, :] = np.load(fn)

Also, note that your code to generate the data can be made more Pythonic:

for k, patient in enumerate(patients, start=1):
    idata = np.concatenate((interictalData, tmpData[0:22, start*256:end]), axis=1)
    np.save(f'interictalData_matrix{k}_{patient}_{l}.npy', idata)