How to differentiate a file like object from a file path like object

Question

Summary:

There is a variety of function for which it would be very useful to be able to pass in two kinds of objects: an object that represents a path (usually a string), and an object that represents a stream of some sort (often something derived from IOBase, but not always). How can this variety of function differentiate between these two kinds of objects so they can be handled appropriately?

---

Say I have a function intended to write a file from some kind of object file generator method:

spiff = MySpiffy()

def spiffy_file_makerA(spiffy_obj, file):
    file_str = '\n'.join(spiffy_obj.gen_file()) 
    file.write(file_str)

with open('spiff.out', 'x') as f:
    spiffy_file_makerA(spiff, f)
    ...do other stuff with f...

This works. Yay. But I'd prefer to not have to worry about opening the file first or passing streams around, at least sometimes... so I refactor with the ability to take a file path like object instead of a file like object, and a return statement:

def spiffy_file_makerB(spiffy_obj, file, mode):
    file_str = '\n'.join(spiffy_obj.gen_file()) 
    file = open(file, mode)
    file.write(file_str)
    return file

with spiffy_file_makerB(spiff, 'file.out', 'x') as f:
    ...do other stuff with f...

But now I get the idea that it would be useful to have a third function that combines the other two versions depending on whether file is file like, or file path like, but returns the f destination file like object to a context manager. So that I can write code like this:

with  spiffy_file_makerAB(spiffy_obj, file_path_like, mode = 'x') as f:
    ...do other stuff with f...

...but also like this:

file_like_obj = get_some_socket_or_stream()

with spiffy_file_makerAB(spiffy_obj, file_like_obj, mode = 'x'):
    ...do other stuff with file_like_obj...
    # file_like_obj stream closes when context manager exits 
    # unless `closefd=False` 

Note that this will require something a bit different than the simplified versions provided above.

Try as a I might, I haven't been able to find an obvious way to do this, and the ways I have found seem pretty contrived and just a potential for problems later. For example:

def spiffy_file_makerAB(spiffy_obj, file, mode, *, closefd=True):
    try: 
        # file-like (use the file descriptor to open)
        result_f = open(file.fileno(), mode, closefd=closefd)
    except TypeError: 
        # file-path-like
        result_f = open(file, mode)
    finally: 
        file_str = '\n'.join(spiffy_obj.gen_file()) 
        result_f.write(file_str)
        return result_f

Are there any suggestions for a better way? Am I way off base and need to be handling this completely differently?

Answer 1

For my money, and this is an opinionated answer, checking for the attributes of the file-like object for the operations you will need is a pythonic way to determine an object’s type because that is the nature of pythonic duck tests/duck-typing:

Duck typing is heavily used in Python, with the canonical example being file-like classes (for example, cStringIO allows a Python string to be treated as a file).

Or from the python docs’ definition of duck-typing

A programming style which does not look at an object’s type to determine if it has the right interface; instead, the method or attribute is simply called or used (“If it looks like a duck and quacks like a duck, it must be a duck.”) By emphasizing interfaces rather than specific types, well-designed code improves its flexibility by allowing polymorphic substitution. Duck-typing avoids tests using type() or isinstance(). (Note, however, that duck-typing can be complemented with abstract base classes.) Instead, it typically employs hasattr() tests or EAFP programming.

If you feel very strongly that there is some very good reason that just checking the interface for suitability isn't enough, you can just reverse the test and test for basestring or str to test whether the provided object is path-like. The test will be different depending on your version of python.

is_file_like = not isinstance(fp, basestring) # python 2
is_file_like = not isinstance(fp, str) # python 3

In any case, for your context manager, I would go ahead and make a full-blown object like the below in order to wrap the functionality that you were looking for.

class SpiffyContextGuard(object):
    def __init__(self, spiffy_obj, file, mode, closefd=True):
        self.spiffy_obj = spiffy_obj
        is_file_like = all(hasattr(attr) for attr in ('seek', 'close', 'read', 'write'))
        self.fp = file if is_file_like else open(file, mode)
        self.closefd = closefd

    def __enter__(self):
        return self.fp

    def __exit__(self, type_, value, traceback):
        generated = '\n'.join(self.spiffy_obj.gen_file())
        self.fp.write(generated)
        if self.closefd:
            self.fp.__exit__()

And then use it like this:

with SpiffyContextGuard(obj, 'hamlet.txt', 'w', True) as f:
    f.write('Oh that this too too sullied flesh\n')

fp = open('hamlet.txt', 'a')
with SpiffyContextGuard(obj, fp, 'a', False) as f:
    f.write('Would melt, thaw, resolve itself into a dew\n')

with SpiffyContextGuard(obj, fp, 'a', True) as f:
    f.write('Or that the everlasting had not fixed his canon\n')

If you wanted to use try/catch semantics to check for type suitability, you could also wrap the file operations you wanted to expose on your context guard:

class SpiffyContextGuard(object):
    def __init__(self, spiffy_obj, file, mode, closefd=True):
        self.spiffy_obj = spiffy_obj
        self.fp = self.file_or_path = file 
        self.mode = mode
        self.closefd = closefd

    def seek(self, offset, *args):
        try:
            self.fp.seek(offset, *args)
        except AttributeError:
            self.fp = open(self.file_or_path, mode)
            self.fp.seek(offset, *args)

    # define wrappers for write, read, etc., as well

    def __enter__(self):
        return self

    def __exit__(self, type_, value, traceback):
        generated = '\n'.join(self.spiffy_obj.gen_file())
        self.write(generated)
        if self.closefd:
            self.fp.__exit__()

Answer 2

my suggestion is to pass pathlib.Path objects around. you can simply .write_bytes(...) or .write_text(...) to these objects.

other that that you'd have to check the type of your file variable (this is how polymorphism can be done in python):

from io import IOBase

def some_function(file)
    if isinstance(file, IOBase):
        file.write(...)
    else:
        with open(file, 'w') as file_handler:
            file_handler.write(...)

(i hope io.IOBase is the most basic class to check against...). and you would have to catch possible exceptions around all that.

Answer 3

Probably not the answer you're looking for, but from a taste point of view I think it's better to have functions that only do one thing. Reasoning about them is easier this way.

I'd just have two functions: spiffy_file_makerA(spiffy_obj, file), which handles your first case, and a convenience function that wraps spiffy_file_makerA and creates a file for you.