Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

reading large files using apache common io

Hi I am using following code to get all the lines in a list of strings using the FileUtils.readLines functionality from Apache commons IO library. here is my code,

List<String> lines=FileUtils.readLines(new File(fileName));

But whenever i send a file say 45MB with 1 million lines it gives me an out of memory error. What should be the solution. I need to process each individual line.

like image 386
P basak Avatar asked Sep 22 '26 09:09

P basak


1 Answers

Read one line after the other

LineIterator it = FileUtils.lineIterator(file, "UTF-8");
 try {
   while (it.hasNext()) {
    String line = it.nextLine();
     // do something with line
   }
 } finally {
  it.close();
 } 
like image 70
René Link Avatar answered Sep 24 '26 22:09

René Link