Difference between generators and functions returning generators

Question

I was debugging some code with generators and came to this question. Assume I have a generator function

def f(x):
    yield x

and a function returning a generator:

def g(x):
    return f(x)

They surely return the same thing. Can there be any differences when using them interchangeably in Python code? Is there any way to distinguish the two (without inspect)?

Answer 1

I like turkus answer, however examples shown are mostly theoretical and aren't common in day to day coding.

The main practical difference between generator function (with yield) and function which returns generator is that the generator function is lazily evaluated.

Consider this session:

$ python
Python 3.6.0 
[GCC 6.3.1 20170109] on linux
>>> def a():
...     print('in a')
...     yield 0
... 
>>> def b():
...     print('in b')
...     return iter(range(1))
... 
>>> aa = a()  # Lazy evaluation - nothing printed after this line.
>>> next(aa)
in a
0
>>> next(aa)
Traceback ...
StopIteration
>>> bb = b()  # Eager evaluation - whole function is executed after this.
in b
>>> next(bb)
0
>>> next(bb)
Traceback ...
StopIteration

None of them is a golden bullet.

To give you a real example of where this lazy evaluation makes a huge difference in your code check this example.

def get_numbers(x: int):
    if x < 0:
        raise ValueError("Value cannot be negative")
    for i in range(x):
        yield i

try:
   numbers = get_numbers(-5)
except ValueError:
   pass  # log or something
else:
   print(list(numbers))  # <== ValueError is thrown here!

Here is where lazy evaluation is actually bad for your function. It will throw exception in arguably wrong place because the intention is to make it fail just at the start, not during iteration. With this implementation you're passing responsibility of triggering the generator function and managing exception to its user which is tedious and somewhat ugly:

import itertools

try:
    numbers = get_numbers(-5)
    first = next(numbers)
    numbers = itertools.chain([first], numbers)
except ValueError:
    ...

The best way to solve this is to make a function that returns a generator instead a generator function:

def get_numbers(x: int):
    if x < 0:
        raise ValueError("Value cannot be negative")
    return (i for i in range(x))  # I know it can be just `return range(x)`, but I keep it that way to make a point.

As you can see there is no "best way" to do it, both options are viable. It all depends on how you want things to work.

Answer 2

The best way to check it out is using inspect.isgeneratorfunction, which is quite simple function:

def ismethod(object):
    """Return true if the object is an instance method.

    Instance method objects provide these attributes:
        __doc__         documentation string
        __name__        name with which this method was defined
        im_class        class object in which this method belongs
        im_func         function object containing implementation of method
        im_self         instance to which this method is bound, or None"""
    return isinstance(object, types.MethodType)

def isfunction(object):
    """Return true if the object is a user-defined function.

    Function objects provide these attributes:
        __doc__         documentation string
        __name__        name with which this function was defined
        func_code       code object containing compiled function bytecode
        func_defaults   tuple of any default values for arguments
        func_doc        (same as __doc__)
        func_globals    global namespace in which this function was defined
        func_name       (same as __name__)"""
    return isinstance(object, types.FunctionType)

def isgeneratorfunction(object):
    """Return true if the object is a user-defined generator function.

    Generator function objects provides same attributes as functions.

    See help(isfunction) for attributes listing."""
    return bool((isfunction(object) or ismethod(object)) and
                object.func_code.co_flags & CO_GENERATOR)

Now, if you declared your generator using a syntax like this:

my_generator = (i*i for i in range(1000000))

In that case, you could check its type quite easily, for instance, __class__ will return <type 'generator'>.