How to sort generator type in python

Question

My code has below lines

get_alarm_list = conn.query_alarms(query.filter_expr,
                                   query.orderb) 
print "type is:", type(get_alarm_list)
for alarm in get_alarm_list:
    if alarm.severity == 'critical':
        alarm.severity = 2
    elif alarm.severity == 'moderate':
        alarm.severity = 1
    else:
        alarm.severity = 0

alarm_list = sorted(get_alarm_list),
                    key=lambda a: a.severity,
                    reverse=True)
return [alarms.Alarm.from_db_model(alarm)
        for alarm in alarm_list]

Output:

type is <type 'generator'>

The objects in the list are:

for alarm in get_alarm_list:
     print alarm

Output:
<aodh.storage.models.Alarm object at 0x7fa4c0cb1c50>
<aodh.storage.models.Alarm object at 0x7fa4c0cb17d0>
<aodh.storage.models.Alarm object at 0x7fa4c0d86f10>
<aodh.storage.models.Alarm object at 0x7fa4ca372110>
<aodh.storage.models.Alarm object at 0x7fa4ca372190>
<aodh.storage.models.Alarm object at 0x7fa4c0c55d90>

And each alarm consists of the below data

 {'alarm_actions': [u'log://'], 'ok_actions': [], 'description': u'instance running hot', 'state': u'insufficient data', 'fields': ['alarm_actions', 'ok_actions', 'severity', 'timestamp', 'description', 'time_constraints', 'enabled', 'state_timestamp', 'rule', 'alarm_id', 'state', 'insufficient_data_actions', 'repeat_actions', 'user_id', 'project_id', 'type', 'name'], 'repeat_actions': False, 'enabled': True, 'state_timestamp': datetime.datetime(2016, 5, 27, 6, 41, 5, 987428), 'rule': {u'meter_name': u'cpu_util', u'evaluation_periods': 3, u'period': 600, u'statistic': u'avg', u'threshold': 70.0, u'query': [], u'comparison_operator': u'gt', u'exclude_outliers': False}, 'name': u'ddd', 'alarm_id': u'f5045ed5-5c53-4a6e-be53-23d3368f40c6', 'time_constraints': [], 'insufficient_data_actions': [], 'timestamp': datetime.datetime(2016, 5, 27, 6, 41, 5, 987428), 'user_id': u'9a65b258b5a24e74ac5feae2f6c54229', 'project_id': u'28d1c27e782c4448bf53da00f49d3e1b', 'type': u'threshold', 'severity': 2}

How can i iterate over the generator?

alarm_list = sorted(get_alarm_list,
                    key=lambda a: a.severity,
                    reverse=True)

But here alarm_list is empty. How can i use sort function on generator get_alarm_list

Answer 1

The problem in your code is that you are trying to sort exhausted generator ( you sorting after for loop thought generator). You can sort result produced by generator object, so the option for you is to exhaust generator into variable in other words to create new list based on get_alarm_list as list(generator) and then iterate and sort it with simple sorted function or list.sort method:

get_alarm_list = conn.query_alarms(query.filter_expr, query.orderb) 
sorted_alarm_list = sorted(list(get_alarm_list),
                               key=lambda a: a.severity,
                               reverse=True)
for alarm in sorted_alarm_list:
    print alarm

Note 1: After execution of list(get_alarm_list) - get_alarm_list generator became empty. And the only item which store result of conn.query_alarms is the sorted_get_alarm_list. You can read more about generators on Generators Python Wiki and Understanding Generators in Python

Note 2: Actually you can pass generator object to sorted and you will get same list as passing list(generator), however sorted works faster if you pass a list to it ( see more on SO answer sorted() using Generator Expressions Rather Than Lists).

Answer 2

The problem you're running into is that you're consuming the whole generator in your for loop when you modify the severity attributes of your objects. That means there's nothing left to iterate on when you call sorted, since generators are good for one use only.

You can fix this by getting rid of the first loop, and putting the severity-transforming logic into the key lambda function:

alarms_gen = conn.query_alarms(query.filter_expr, query.orderb) 
alarms_list = sorted(alarms_gen,
                     key=lambda x: {'critical': -2, 'moderate': -1}.get(x.severity, 0))

Note that I've renamed your get_alarms_list variable to be less misleading (it's not a list). I've also made it so that reverse=True is not needed in the sorted call by mapping the higher priorities to negative key values.