When to use a custom index rather than ordinary columns in Pandas

Question

In pandas you can replace the default integer-based index with an index made up of any number of columns using set_index().

What confuses me, though, is when you would want to do this. Regardless of whether the series is a column or part of the index, you can filter values in the series using boolean indexing for columns, or xs() for rows. You can sort on the columns or index using either sort_values() or sort_index().

The only real difference I've encountered is that indexes have issues when there are duplicate values, so it seems that using an index is more restrictive, if anything.

Why then, would I want to convert my columns into an index in Pandas?

Answer 1

In my opinion custom indexes are good for quickly selecting data.

They're also useful for aligning data for mapping, for aritmetic operations where the index is used for data alignment, for joining data, and for getting minimal or maximal rows per group.

DatetimeIndex is nice for partial string indexing, for resampling.

But you are right, a duplicate index is problematic, especially for reindexing.

Docs:

• Identifies data (i.e. provides metadata) using known indicators, important for analysis, visualization, and interactive console display
• Enables automatic and explicit data alignment
• Allows intuitive getting and setting of subsets of the data set

Also you can check Modern pandas - Indexes, direct link.