Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Read previous line in a file python

Tags:

python

file

loops

I need to get the value of the previous line in a file and compare it with the current line as I'm iterating through the file. The file is HUGE so I can't read it whole or randomly accessing a line number with linecache because the library function still reads the whole file into memory anyway.

EDIT I'm so sorry I forgot the mention that I have to read the file backwardly.

EDIT2

I have tried the following:

 f = open("filename", "r")
 for line in reversed(f.readlines()): # this doesn't work because there are too many lines to read into memory

 line = linecache.getline("filename", num_line) # this also doesn't work due to the same problem above. 
like image 924
Lim H. Avatar asked Jun 28 '13 20:06

Lim H.


People also ask

How do I go to a previous line in Python?

f = open("filename", "r") for line in reversed(f.

How do you read a specific line in a file in Python?

Method 1: fileobject.readlines() A file object can be created in Python and then readlines() method can be invoked on this object to read lines into a stream.

How do I read the last 10 lines of a file in Python?

As we know, Python provides multiple in-built features and modules for handling files. Let's discuss different ways to read last N lines of a file using Python. In this approach, the idea is to use a negative iterator with the readlines() function to read all the lines requested by the user from the end of file.


2 Answers

Just save the previous when you iterate to the next

prevLine = ""
for line in file:
    # do some work here
    prevLine = line

This will store the previous line in prevLine while you are looping

edit apparently OP needs to read this file backwards:

aaand after like an hour of research I failed multiple times to do it within memory constraints

Here you go Lim, that guy knows what he's doing, here is his best Idea:

General approach #2: Read the entire file, store position of lines

With this approach, you also read through the entire file once, but instead of storing the entire file (all the text) in memory, you only store the binary positions inside the file where each line started. You can store these positions in a similar data structure as the one storing the lines in the first approach.

Whever you want to read line X, you have to re-read the line from the file, starting at the position you stored for the start of that line.

Pros: Almost as easy to implement as the first approach Cons: can take a while to read large files

like image 186
Stephan Avatar answered Sep 17 '22 14:09

Stephan


@Lim, here's how I would write it (reply to the comments)

def do_stuff_with_two_lines(previous_line, current_line):
    print "--------------"
    print previous_line
    print current_line

my_file = open('my_file.txt', 'r')

if my_file:
    current_line = my_file.readline()

for line in my_file:

    previous_line = current_line
    current_line = line

    do_stuff_with_two_lines(previous_line, current_line)
like image 34
Diana Avatar answered Sep 19 '22 14:09

Diana