Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python: generator expression vs. yield

In Python, is there any difference between creating a generator object through a generator expression versus using the yield statement?

Using yield:

def Generator(x, y):     for i in xrange(x):         for j in xrange(y):             yield(i, j) 

Using generator expression:

def Generator(x, y):     return ((i, j) for i in xrange(x) for j in xrange(y)) 

Both functions return generator objects, which produce tuples, e.g. (0,0), (0,1) etc.

Any advantages of one or the other? Thoughts?

like image 949
cschol Avatar asked Jan 03 '10 16:01

cschol


People also ask

What is yield in generator function Python?

What Is Yield In Python? The Yield keyword in Python is similar to a return statement used for returning values or objects in Python. However, there is a slight difference. The yield statement returns a generator object to the one who calls the function which contains yield, instead of simply returning a value.

What is a Python generator expression?

A generator expression is an expression that returns a generator object. Basically, a generator function is a function that contains a yield statement and returns a generator object.

What is the difference between yield and return Python?

The yield statement hauls the function and returns back the value to the function caller and restart from where it is left off. The yield statement can be called multiple times. While the return statement ends the execution of the function and returns the value back to the caller.

Are generator expressions faster?

Thus, generator expressions are faster than list comprehension and hence time efficient.


1 Answers

There are only slight differences in the two. You can use the dis module to examine this sort of thing for yourself.

Edit: My first version decompiled the generator expression created at module-scope in the interactive prompt. That's slightly different from the OP's version with it used inside a function. I've modified this to match the actual case in the question.

As you can see below, the "yield" generator (first case) has three extra instructions in the setup, but from the first FOR_ITER they differ in only one respect: the "yield" approach uses a LOAD_FAST in place of a LOAD_DEREF inside the loop. The LOAD_DEREF is "rather slower" than LOAD_FAST, so it makes the "yield" version slightly faster than the generator expression for large enough values of x (the outer loop) because the value of y is loaded slightly faster on each pass. For smaller values of x it would be slightly slower because of the extra overhead of the setup code.

It might also be worth pointing out that the generator expression would usually be used inline in the code, rather than wrapping it with the function like that. That would remove a bit of the setup overhead and keep the generator expression slightly faster for smaller loop values even if LOAD_FAST gave the "yield" version an advantage otherwise.

In neither case would the performance difference be enough to justify deciding between one or the other. Readability counts far more, so use whichever feels most readable for the situation at hand.

>>> def Generator(x, y): ...     for i in xrange(x): ...         for j in xrange(y): ...             yield(i, j) ... >>> dis.dis(Generator)   2           0 SETUP_LOOP              54 (to 57)               3 LOAD_GLOBAL              0 (xrange)               6 LOAD_FAST                0 (x)               9 CALL_FUNCTION            1              12 GET_ITER         >>   13 FOR_ITER                40 (to 56)              16 STORE_FAST               2 (i)    3          19 SETUP_LOOP              31 (to 53)              22 LOAD_GLOBAL              0 (xrange)              25 LOAD_FAST                1 (y)              28 CALL_FUNCTION            1              31 GET_ITER         >>   32 FOR_ITER                17 (to 52)              35 STORE_FAST               3 (j)    4          38 LOAD_FAST                2 (i)              41 LOAD_FAST                3 (j)              44 BUILD_TUPLE              2              47 YIELD_VALUE              48 POP_TOP              49 JUMP_ABSOLUTE           32         >>   52 POP_BLOCK         >>   53 JUMP_ABSOLUTE           13         >>   56 POP_BLOCK         >>   57 LOAD_CONST               0 (None)              60 RETURN_VALUE >>> def Generator_expr(x, y): ...    return ((i, j) for i in xrange(x) for j in xrange(y)) ... >>> dis.dis(Generator_expr.func_code.co_consts[1])   2           0 SETUP_LOOP              47 (to 50)               3 LOAD_FAST                0 (.0)         >>    6 FOR_ITER                40 (to 49)               9 STORE_FAST               1 (i)              12 SETUP_LOOP              31 (to 46)              15 LOAD_GLOBAL              0 (xrange)              18 LOAD_DEREF               0 (y)              21 CALL_FUNCTION            1              24 GET_ITER         >>   25 FOR_ITER                17 (to 45)              28 STORE_FAST               2 (j)              31 LOAD_FAST                1 (i)              34 LOAD_FAST                2 (j)              37 BUILD_TUPLE              2              40 YIELD_VALUE              41 POP_TOP              42 JUMP_ABSOLUTE           25         >>   45 POP_BLOCK         >>   46 JUMP_ABSOLUTE            6         >>   49 POP_BLOCK         >>   50 LOAD_CONST               0 (None)              53 RETURN_VALUE 
like image 132
Peter Hansen Avatar answered Sep 19 '22 14:09

Peter Hansen