Seek from end of file in python 3

Question

One of the changes in python 3 has been to remove the ability to seek from the end of the file in normal text mode. What is the generally accepted alternative to this?

For example in python 2.7 I would enter file.seek(-3,2)

I've read a bit about why they did this so please don't just link to a PEP. I know that using 'rb' would allow me to seek, but this makes my text file read in the wrong format.

Answer 1

In Python 2, the file data wasn't being decoded while reading. Seeking backwards and multi-byte encodings don't mix well (you can't know where would the next character start), which is why it is disabled for Python 3.

You can still seek on the underlying buffer object, via the TextIOBase.buffer attribute, but then you'll have to reattach a new TextIOBase wrapper, as the current wrapper will no longer know where it is at:

import io

file.buffer.seek(-3, 2)
file = io.TextIOWrapper(
    file.buffer, encoding=file.encoding, errors=file.errors,
    newline=file.newlines)

I've copied across any encoding and line handling information to the io.TextIOWrapper() object.

Take into account that this can decoding could break for UTF-16, UTF-32, UTF-8 and other multi-byte codecs.

Demo:

>>> import io
>>> with open('demo.txt', 'w') as out:
...     out.write('Demonstration\nfor seeking from the end')
... 
38
>>> with open('demo.txt') as inf:
...     print(inf.readline())
...     inf.buffer.seek(-3, 2)
...     inf = io.TextIOWrapper(inf.buffer)
...     print(inf.readline())
... 
Demonstration

35
end

You could wrap this up in a utility function:

import io

def textio_seek(fobj, amount, whence=0):
    fobj.buffer.seek(amount, whence)
    return io.TextIOWrapper(
        fobj.buffer, encoding=fobj.encoding, errors=fobj.errors,
        newline=fobj.newlines)

and use this as:

with open(somefile) as file:
    # ...

    file = textio_seek(file, -2, 3)

    # ...

Using the file object as a context manager just still works, as the original file object reference is still attached to the original file buffer object and thus can still be used to close the file.