python open(file) only read about 2507000 lines?

Question

I tried to read a big file use python but it seems that python only process about 2507000 lines and then stop. Could you suggest why?

I am using Python 2.7 32bit on windows. I also post the code I am using. Thanks.

counter = 0
with open(input) as file:
    for line in file:
        counter += 1
        if counter % 1000 == 0:
            sys.stderr.write(str(counter) + "lines processed.\n")

Answer 1

Different programs may count lines differently, depending on how they expect lines to be delimited. The DOS/Windows convention is \r\n and the Unix convention is \n as the delimiter.

If you open the file in universal newlines mode using open(filename, "U"), your program will recognize all the different delimiters.