Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Generators to iterate over a dictionary uniformly in both Python 2 and 3

Is there a way to efficiently iterate over the values/items in a dictionary that works in both Python 2 and Python 3?

In Python 2, I can write

for x in mydict:
for x in mydict.iterkeys():
for x in mydict.viewkeys():
for x in mydict.itervalues():
for x in mydict.viewvalues():
for x in mydict.iteritems():
for x in mydict.viewitems():

and in Python 3, I have the following possibilities:

for x in mydict:
for x in mydict.keys():
for x in mydict.values():
for x in mydict.items():

So, I can iterate over the keys with for x in mydict, and that works in both Python 2 and 3. But is there a way to iterate over values and key-value pairs (‘items’) that works universally? I need it for writing Python code that can be run in both Python versions.

(On a side note, other obstacles with iterators can be bypassed easily; take for example:

if sys.version_info.major<3:
    from itertools import izip as zip, imap as map

However, the dictionary methods cannot be redefined easily like map and zip.)

Any ideas?

like image 625
Daniel Avatar asked May 02 '12 05:05

Daniel


People also ask

Is range a generator in python3?

range is a class of immutable iterable objects. Their iteration behavior can be compared to list s: you can't call next directly on them; you have to get an iterator by using iter . So no, range is not a generator. They are immutable, so they can be used as dictionary keys.

How many times can you iterate through a generator?

This is because generators, like all iterators, can be exhausted. Unless your generator is infinite, you can iterate through it one time only. Once all values have been evaluated, iteration will stop and the for loop will exit. If you used next() , then instead you'll get an explicit StopIteration exception.

Which is faster iterator or generator in Python?

The yield functions (known as generators) are for speed and generally they can be written without bothering about internal state. So it's less effort to write them and they are fast because Python just manages all the "state". So it's almost 3 times faster just because generators directly populate the __next__ -slot.


3 Answers

values() version of Just another dunce's answer

for value in (mydict[key] for key in mydict):

or

def dict_values(d):
   return (mydict[key] for key in mydict)

def dict_items(d):
   return ((key, mydict[key]) for key in mydict)

for value in dict_values(mydict):
    ...

for value in dict_items(mydict):
    ...

This is pretty crappy though. You could check the python version and instead of returning the generator, return d.items() or d.iteritems() as appropriate

I think a better question may be how to write Python2 code that works well with 2to3 and generates good Python3 code

like image 78
John La Rooy Avatar answered Oct 18 '22 18:10

John La Rooy


The following code works in both Py2 and Py3:

def iteritems(d):
    'Factor-out Py2-to-3 differences in dictionary item iterator methods'
    try:
         return d.iteritems()
    except AttributeError:
         return d.items()

d = dict(red=1, blue=2, green=3)
for k, v in iteritems(d):
    print(k)
    print(v)
    print('')

Note, this solution has the advantage of using the native iterators rather than rolling your own generators and doing your own lookups. The native iterators are faster than the other solutions presented.

like image 29
Raymond Hettinger Avatar answered Oct 18 '22 16:10

Raymond Hettinger


This will give you a generator:

for key,value in ((key,mydict[key] for key in mydict)

If you want a list, use [] around the expression instead of the () on the outside

like image 39
Snakes and Coffee Avatar answered Oct 18 '22 17:10

Snakes and Coffee