Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Performance implications of storing 600,000+ images in the same folder (NTFS)

I need to store about 600,000 images on a web server that uses NTFS. Am I better off storing images in 20,000-image chunks in subfolders? (Windows Server 2008)

I'm concerned about incurring operating system overhead during image retrieval

like image 601
Brian Webster Avatar asked Feb 04 '23 05:02

Brian Webster


1 Answers

Go for it. As long has you have an external index and have a direct file path to each file with out listing the contents of the directory then you are ok.

I have a folder with that is over 500 GB in size with over 4 million folders (which have more folders and files). I have somewhere in the order of 10 million files in total.

If I accidentally open this folder in windows explorer it gets stuck at 100% cpu usage (for one core) until I kill the process. But as long as you directly refer to the file/folder performance is great (meaning I can access any of those 10 million files with no overhead)

like image 132
Pyrolistical Avatar answered Apr 28 '23 11:04

Pyrolistical