Python - Sorting elements in a list of lists

Question

Apologies if this has been answered elsewhere; I've tried searching, but haven't found anything that answers my question (or perhaps I have, but didn't understand it)...

I'm fairly new to Python (v2.6.2) and have a list of lists containing floating point values which looks something like the following (except the full thing has 2+ million entries for each list):

cat = [[152.123, 150.456, 151.789, ...], [4.123, 3.456, 1.789, ...], [20.123, 22.456, 21.789, ...]]

Now what I would like to do is sort all 3 of the lists by ascending order of the elements of the 3rd list, such that I get:

cat_sorted = [[152.123, 151.789, 150.456, ...], [4.123, 1.789, 3.456, ...], [20.123, 21.789, 22.456, ...]]

I've tried a few things, but they don't give me what I'm looking for (or perhaps I'm using them incorrectly). Is there a way to do what I am looking for and if so, what's the easiest & quickest (considering I have 3 x 2million entries)? Is there a way of sorting one list using another?

Answer 1

This is going to be painful, but using default python you have 2 options:

• decorate the 1st and 2nd lists with enumerate(), then sort these using the index to refer to values from the 3rd list:

  cat_sorted = [
      [e for i, e in sorted(enumerate(cat[0]), key=lambda p: cat[2][p[0]])],
      [e for i, e in sorted(enumerate(cat[1]), key=lambda p: cat[2][p[0]])],
      sorted(cat[2])
  ]
  

  although it may help to sort cat[2] in-place instead of using sorted(); you cannot get around using sorted() for the other two.

• zip() the three lists together, then sort on the third element of this new list of lists, then zip() again to get back to the original structure:

  from operator import itemgetter
  cat_sorted = zip(*sorted(zip(*cat), key=itemgetter(2)))
  

Neither will be a performance buster, not with plain python lists of millions of numbers.

Answer 2

If you're willing to use an additional library, I suggest Python Pandas. It has a DataFrame object similar to R's data.frame and accepts a list of lists in the constructor, which will create a 3-column data array. Then you can easily use the built-in pandas.DataFrame.sort function to sort by the third column (ascending or descending).

There are many plain Python ways to do this, but given the size of your problem, using the optimized functions in Pandas is a better approach. And if you need any kind of aggregated statistics from your sorted data, then Pandas is a no-brainer for this.