Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

200,000 images in single folder in linux, performance issue or not?

I have a php/mysql website with over 200,000 images in single folder (linux server). I don't think, that I will never need to see them in file explorer, instead they will be viewed on website on their individual pages. They are just displayed in product page on website. File system is ext3. so is it wise to save them in single folder? can it slow down the site's performance?

like image 967
Gajendra Bang Avatar asked Oct 23 '10 12:10

Gajendra Bang


People also ask

How many files is too many for a folder?

You can put 4,294,967,295 files into a single folder if drive is formatted with NTFS (would be unusual if it were not) as long as you do not exceed 256 terabytes (single file size and space) or all of disk space that was available whichever is less.

How many files can be stored in a folder in Linux?

It depends on the file system. ext3 suppport ~32000 subdirectories (not files!) in a given directory, with ext4 it's 64000 by default. xfs has no limit to my knowledge. Ext4 has no limit if the dir_index and dir_nlink features are set.

What is a sensible number of files to aim for per folder?

However, the rule of thumb I go by is about 500 per directory. I have image folders with several thousand images in a single directory, and audio folders with even more. 500 is a very very low limit for a windows folder.

How many folders can you have in Linux?

There's no limit on directories in particular beyond this; keep in mind that every file or directory requires at least one filesystem block (typically 4KB), though, even if it's a directory with only a single item in it. As you can see, though, 80,000 inodes is unlikely to be a problem.


2 Answers

Ext3 uses tree to hold directory contents, so its capability to handle a large number of files in a single directory is better than that of those file systems with linear directory listings. Here you can read the description of the tree used to keep directory contents.

However, 200K files is still a huge number. It's reasonable to move them into subdirectories based on first n characters of file names. This approach lets you keep only file names and not directory names, and when you need to access the file, you know where (in which subdirectory) to look for it.

like image 118
Eugene Mayevski 'Callback Avatar answered Oct 24 '22 05:10

Eugene Mayevski 'Callback


This seems to have been answered at the below link.

https://serverfault.com/questions/43133/filesystem-large-number-of-files-in-a-single-directory

like image 30
patlc Avatar answered Oct 24 '22 04:10

patlc