Why are FAT32 disks limited to 4GB files?

Question

I'm not looking for a workaround; I'd like an explanation. Most of the links I found through Google just tell me that the file limit is 4GB, but not why.

I am aware of the explanation by Wikipedia: http://en.wikipedia.org/wiki/File_Allocation_Table#FAT32

But this still does not go into detail about why. (What does SCANDISK have to do with it?)

Maybe then I will understand whether it's possible to overcome the 4GB file size limit.

Answer 1

Because FAT32 stores 32-bit file sizes and the maximum you can store in 32 bits is 2^32-1 ~= 4.29e9. 2^32-1 bytes = 4GB - 1 byte.

(This is, in fact, explained in the Wikipedia article.)

Answer 2

To extend that answer, FAT32 is derived from FAT16 and FAT12. When FAT12 was introduced, PCs ran a 16-bit OS, and there were no drives larger than ten megabytes. I don't think anyone was expecting that anyone on a PC would ever need a file that was four hundred times the size of the largest available drive. Further, even if Microsoft had the foresight to allocate an extra byte for each directory entry to hold bits 32-39 of the file size (allowing files up to one terabyte each) it's doubtful any application programmers would have used it. No languages provided convenient support for integer math bigger than 32 bits; even 32-bit maths were considered pretty fancy.

A bigger question in my mind is why I'm unaware of any efforts to move toward a standard non-FAT32 storage interface that isn't based on sequentially-numbered sectors. Both flash drives and hard drives could benefit from knowing the logical significance of various sector writes (in the case of flash, because it would remove the need to copy deleted sectors when performing wear management; in the case of hard drives, because it would allow writes to be resequenced in ways that would not violate data consistency requirements).