Is there an elegant way to parse a text file *backwards*? [duplicate]

Question

Possible Duplicate:
How to read a file from bottom to top in Ruby?

In the course of working on my Ruby program, I had the Eureka Moment that it would be much simpler to write if I were able to parse the text files backwards, rather than forward.

It seems like it would be simple to simply read the text file, line by line, into an array, then write the lines backwards into a text file, parse this temp file forwards (which would now effectively be going backwards) make any necessary changes, re-catalog the resulting lines into an array, and write them backwards a second time, restoring the original direction, before saving the modifications as a new file.

While feasible in theory, I see several problems with it in practice, the biggest of which is that if the size of the text file is very large, a single array will not be able to hold the entirety of the document at once.

Is there a more elegant way to accomplish reading a text file backwards?

Answer 1

If you are not using lots of UTF-8 characters you can use Elif library which work just like File.open. just load Elif and replace File.open with Elif.open

Elif.open('read.txt', "r").each_line{ |s|
    puts s
}

This is a great library, but the only problem I am experiencing right now is that it have several problems with line ending in UTF-8. I now have to re-think a way to iterate my files

---

Additional Details

As I google a way to answer this problem for UTF-8 reverse file reading. I found a way that already implemented by File library:

To read a file backward you can try the ff code:

File.readlines('manga_search.test.txt').reverse_each{ |s|
   puts s
}

This can do a good job as well

Answer 2

There's no software limit to Ruby array. There are some memory limitations though: Array size too big - ruby

Your approach would work much faster if you can read everything into memory, operate there and write it back to disk. Assuming the file fits in memory of course.

Answer 3

Let's say your lines are 80 chars wide on average, and you want to read 100 lines. If you want it efficient (as opposed to implemented with the least amount of code), then go back 80*100 bytes from the end (using seek with the "relative to end" option), then read ONE line (this is likely a partial one, so throw it away). Remember your current position via tell, then read everything up until the end.

You now have either more or less than a 100 lines in memory. If less, go back (100+1.5*no_of_missing_lines)*80, and repeat the above steps, but only reading lines until you reach your remembered position from before. Rinse and repeat.