Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using io.BufferedReader on a stream obtained with open()?

I want to use a buffered stream because I want to use a peek() method to peek ahead but use my stream with another method that expects a file-like object. (I'd use seek() but may have to handle piped-in I/O that doesn't support random access.)

But this test case fails:

AttributeError: 'file' object has no attribute '_checkReadable'

import sys
import io

srcfile = sys.argv[1]
with open(srcfile, 'rb') as f:
    fbuf = io.BufferedReader(f)
    print fbuf.read(20)

What's going on and how do I fix it? I thought BufferedReader was intended to buffer a stream. If so, why does the open() function not return something that's compatible with it?

like image 335
Jason S Avatar asked Apr 17 '12 21:04

Jason S


People also ask

What is '_ Io BufferedReader?

Introduction. The Java.io.BufferedReader class reads text from a character-input stream, buffering characters so as to provide for the efficient reading of characters, arrays, and lines.Following are the important points about BufferedReader − The buffer size may be specified, or the default size may be used.

What happens if BufferedReader is not closed?

With the BufferdRreader nothing but if you loose the reference to stream and cannot close it and this leads to a memory leak.


3 Answers

By the looks of your print statement, you're using Python 2. On that version, a file is not a valid argument to the BufferedReader constructor:

Under Python 2.x, this is proposed as an alternative to the built-in file object, but in Python 3.x it is the default interface to access files and streams. (1)

You should use io.open instead:

>>> f = io.open(".bashrc", "rb")

If you do this, there's no need to explicitly wrap it in a BufferedReader since that's exactly what io.open returns by default:

>>> type(f)
<type '_io.BufferedReader'>

See its docs for details; there's a buffering argument that controls the buffering.

In Python 3, open is io.open so the two I/O libraries have been merged back into one. It seems that io was added to Python 2.6 mostly for forward compatibility.

like image 152
Fred Foo Avatar answered Sep 24 '22 21:09

Fred Foo


You can set the amount of buffering in bytes by passing the buffering argument to open:

import sys

srcfile = sys.argv[1]
with open(srcfile, 'rb', buffering=30) as f:
    print(f.peek(30))
    print(f.read(20))

This is a BufferedReader:

>>> with open("test.txt", 'rb', buffering=30) as f:
...     type(f)
<class '_io.BufferedReader'>

Note that, by default, it's buffered to 1 - line buffered.

like image 39
Gareth Latty Avatar answered Sep 21 '22 21:09

Gareth Latty


In Python2, if you have to use file object as returned by open (or e.g. provided by some module routines which you cannot modify), you can use file descriptor obtained by fileno() for io.FileIO constructor, then pass io.FileIO object to io.BufferedReader constructor.

So, you sample code can be rewritten as follows:

import sys
import io

srcfile = sys.argv[1]
with open(srcfile, 'rb') as f:
    fio  = io.FileIO(f.fileno())
    fbuf = io.BufferedReader(fio)
    print fbuf.read(20)
like image 41
majkelx Avatar answered Sep 25 '22 21:09

majkelx