No 'print' output when using yield?

Question

I am implementing a web scraper using the scrapy framework. In order to implement pipeline, yield is necessary when parsing the response from the scraper. It seems to me that when using yield at the end of a function, all output from print statements is suppressed and replaced by the generator object.

def testFunc(arg1):
    print arg1
    yield arg1

testFunc('This does not print.')

Results in:

In [7]: testFunc('Will this print?')
Out[7]: <generator object testFunc at 0x10d636d20>

Simply commenting the yield restores the print call:

def testFunc(arg1):
    print arg1

Results in:

In [10]: testFunc('Will this print?')
Will this print?

How do I maintain print output when using yield?

Answer 1

Calling a generator function as in testFunc(1) simply creates a generator instance; it does not run the body of the code. Generators are iterators, so you can pass them to next(). In the case of Generators, the action of its __next__() is essentially to run up to the next yield statement and return the yielded value. So you can do things like:

>>> gen = testFunc(1)
>>> next(gen)
Will this print?
1

or, as others have noted, you can loop over it (though this is not necessary if you want to just yield one value).

When you define a generator, you can almost think of calling a generator as creating an instance of some class that implements a very specific state machine that works as an iterator. To be clear, that's not actually how it works, but it can be written equivalently that way. Generators are a much more elegant way to do this in most cases, however.

Answer 2

The print statements will be executed whenever the generator is iterated over. If you just want to see what the iterator prints, you can call it in a list comprehension without saving the result.

def testFunc(arg1):
    print arg1
    yield arg1

>>> [_ for _ in testFunc(1)]
1
[1]

Answer 3

I don't think anyone gave the answer to "How do I maintain print output when using yield?" from the perspective of the programmer writing the function.

Here it is:

def testFunc(arg1):
    print(arg1)
    def foo():
         yield arg1
    return foo()