Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the most efficient way to read a big text file backwards?

What is the most efficient way to read a big text file backwards, line by line, using Windows API functions? For example, if a file is:

line 1
...
line 108777
line 108778

the output should be:

line 108778
line 108777
...
line 1

I want to write a C program for this. You don't need to write a code (but if you want, that's great), I am just interested in how to do this having in mind that files are big and that I want program to run as fast as it can.

Also, I am interested in which Windows API functions to use.

like image 249
Matthew Murdock Avatar asked Dec 23 '22 01:12

Matthew Murdock


2 Answers

A more clever solution is to open the file, set the file-offset to the (end of the file - buffersize) and read (buffersize) bytes, u can parse the data in the buffer from back to front to find newlines and do whatever you want, and so on.

like image 98
Quonux Avatar answered Dec 26 '22 12:12

Quonux


If performance is more important than memory utilization, I'd just do a buffered read of the entire text file into memory and then parse it in whatever order you like.

Take a look at memory mapped files, some advantages of which are discussed here.

like image 32
Jim Lamb Avatar answered Dec 26 '22 10:12

Jim Lamb