Java load and process big data

Question

I have several big data(1G each) files containing person's information(just name and phone numbers). The format is clear and flexible. The problem is load and process them. Process one of them maybe still feasible but if I want to process them all under certain directory, things get tricky. When I use

 File file = chooser.getSelectFile();

and get a directory, I think the next step is put the file in the file array:

 File[] files = file.ListFile();  

But will that cause a problem? Since each file is 1G, the VM's memory wont ablt to hold all these files together. In order to search them later, I think I may want to sort them first. How can I sort these individual files? Since the total size is so big, the idea: put them into files like A.txt, B.txt which start with letter A and B is not sufficient.

Answer 1

A File just represents the file name, not the contents. Unless you have many thousands of files per directory, you haven't done anything to use much memory yet.

Don't try to process these files by loading each one entirely into memory though.