I need to get the value of the previous line in a file and compare it with the current line as I'm iterating through the file. The file is HUGE so I can't read it whole or randomly accessing a line number with linecache
because the library function still reads the whole file into memory anyway.
EDIT I'm so sorry I forgot the mention that I have to read the file backwardly.
EDIT2
I have tried the following:
f = open("filename", "r")
for line in reversed(f.readlines()): # this doesn't work because there are too many lines to read into memory
line = linecache.getline("filename", num_line) # this also doesn't work due to the same problem above.
f = open("filename", "r") for line in reversed(f.
Method 1: fileobject.readlines() A file object can be created in Python and then readlines() method can be invoked on this object to read lines into a stream.
As we know, Python provides multiple in-built features and modules for handling files. Let's discuss different ways to read last N lines of a file using Python. In this approach, the idea is to use a negative iterator with the readlines() function to read all the lines requested by the user from the end of file.
Just save the previous when you iterate to the next
prevLine = ""
for line in file:
# do some work here
prevLine = line
This will store the previous line in prevLine
while you are looping
edit apparently OP needs to read this file backwards:
aaand after like an hour of research I failed multiple times to do it within memory constraints
Here you go Lim, that guy knows what he's doing, here is his best Idea:
General approach #2: Read the entire file, store position of lines
With this approach, you also read through the entire file once, but instead of storing the entire file (all the text) in memory, you only store the binary positions inside the file where each line started. You can store these positions in a similar data structure as the one storing the lines in the first approach.
Whever you want to read line X, you have to re-read the line from the file, starting at the position you stored for the start of that line.
Pros: Almost as easy to implement as the first approach Cons: can take a while to read large files
@Lim, here's how I would write it (reply to the comments)
def do_stuff_with_two_lines(previous_line, current_line):
print "--------------"
print previous_line
print current_line
my_file = open('my_file.txt', 'r')
if my_file:
current_line = my_file.readline()
for line in my_file:
previous_line = current_line
current_line = line
do_stuff_with_two_lines(previous_line, current_line)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With