fastest way to load images in python for processing

Question

I want to load more than 10000 images in my 8gb ram in the form of numpy arrays.So far I have tried cv2.imread,keras.preprocessing.image.load_image,pil,imageio,scipy.I want to do it the fastest way possible but I can't figure out which on is it.

Answer 1

One of the fastest ways is to get your multiprocessors to do your job in Parallel. It brings multiple processors to work on your tasks at the same time when concurrent running isn't an issue. Now the example below is just a simple sketch of how it might look, you can practice with small functions and then integrate them with your own code :

from multiprocessing import Process
    #this is the function to be parallelized
    def image_load_here(image_path):
        pass 

if __name__ == '__main__':
    #Start the multiprocesses and provide your dataset.
    p = Process(target=image_load_here,['img1', 'img2', 'img3', 'img4'])
    p.start()
    p.join()

Feel free to write, ill try to help.