Reading file containing random numbers, sorting it and than writing to other file

Question

In an interview I was asked the following question,

There is a file named sourceFile.txt containing random numbers aligned one below other like below,

608492
213420
23305
255572
64167
144737
81122
374768
535077
866831
496153
497059
931322

same number can occur more than once. The size of sourceFile.txt is around 65GB.

I need to read that file and write the numbers into new file lets say destinationFile.txt in sorted order.

I wrote the following code for this,

/*
    Copy the numbers present in the file, store in 
    list, sort it and than write into another file.
*/
public static void readFileThanWrite(String sourceFileName,String destinationFileName) throws Exception{
    String line = null;
    BufferedReader reader = new BufferedReader(new FileReader(sourceFileName));
    List<Integer> list = new ArrayList<Integer>();
    do{         
        if(line != null){               
            list.add(Integer.parseInt(line));
        }

        line = reader.readLine();
    }while(line != null);

    Collections.sort(list);

    File file = new File(destinationFileName);
    FileWriter fileWriter = new FileWriter(file,true); // 'True' means write content to end of file
    BufferedWriter buff = new BufferedWriter(fileWriter);
    PrintWriter out = new PrintWriter(buff);

    for(Iterator<Integer> itr = list.iterator();itr.hasNext();){
        out.println(itr.next());
    }

    out.close();
    buff.close();
    fileWriter.close();
}

But the interviewer said the above program will fail to load and sort numbers as the file is large.

What should be the better solution ?

Answer 1

If you know that all the numbers are relatively small, keeping an array of occurences would do just fine. If you don't have any information about the input, you're looking for external sorting. Here's a Java project that could help you, and here is the corresponding class.