Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does .loc have inclusive behavior for slices?

For some reason, the following 2 calls to iloc / loc produce different behavior:

>>> import pandas as pd >>> df = pd.DataFrame(dict(A=range(3), B=range(3))) >>> df.iloc[:1]    A  B 0  0  0 >>> df.loc[:1]    A  B 0  0  0 1  1  1 

I understand that loc considers the row labels, while iloc considers the integer-based indices of the rows. But why is the upper bound for the loc call considered inclusive, while the iloc bound is considered exclusive?

like image 782
James Ko Avatar asked Apr 22 '18 04:04

James Ko


People also ask

Is ILOC slice inclusive?

iloc . So [1:3] will only return values at index 1 and 2 (typical python behavior). However, when using . loc , the slicing is inclusive at both ends.

Is .LOC inclusive in python?

loc is end-inclusive, we can just say .

Is Panda indexing inclusive?

Pandas provides a suite of methods in order to have purely label based indexing. This is a strict inclusion based protocol.

What is the difference between .LOC & ILOC?

The main difference between pandas loc[] vs iloc[] is loc gets DataFrame rows & columns by labels/names and iloc[] gets by integer Index/position. For loc[], if the label is not present it gives a key error. For iloc[], if the position is not present it gives an index error.


1 Answers

Quick answer:

It often makes more sense to do end-inclusive slicing when using labels, because it requires less knowledge about other rows in the DataFrame.

Whenever you care about labels instead of positions, end-exclusive label slicing introduces position-dependence in a way that can be inconvenient.


Longer answer:

Any function's behavior is a trade-off: you favor some use cases over others. Ultimately the operation of .iloc is a subjective design decision by the Pandas developers (as the comment by @ALlollz indicates, this behavior is intentional). But to understand why they might have designed it that way, think about what makes label slicing different from positional slicing.

Imagine we have two DataFrames df1 and df2:

df1 = pd.DataFrame(dict(X=range(4)), index=['a','b','c','d']) df2 = pd.DataFrame(dict(X=range(4)), index=['b','c','z']) 

df1 contains:

   X Y a  0 b  1 c  2 d  3 

df2 contains:

   X Y b  0 c  1 z  2 

Let's say we have a label-based task to perform: we want to get rows between b and c from both df1 and df2, and we want to do it using the same code for both DataFrames. Because b and c don't have the same positions in both DataFrames, simple positional slicing won't do the trick. So we turn to label-based slicing.

If .loc were end-exclusive, to get rows between b and c we would need to know not only the label of our desired end row, but also the label of the next row after that. As constructed, this next label would be different in each DataFrame.

In this case, we would have two options:

  • Use separate code for each DataFrame: df1.loc['b':'d'] and df2.loc['b':'z']. This is inconvenient because it means we need to know extra information beyond just the rows that we want.
  • Get the positional index first, add 1, and then use positional slicing: df.loc[df.index.get_loc('b'):df.index.get_loc('c')+1]. This is just wordy.

But since .loc is end-inclusive, we can just say .loc['b':'c']. Much simpler!

Whenever you care about labels instead of positions, and you're trying to write position-independent code, end-inclusive label slicing re-introduces position-dependence in a way that can be inconvenient.

That said, maybe there are use cases where you really do want end-exclusive label-based slicing. If so, you can use @Willz's answer in this question:

df.loc[start:end].iloc[:-1] 
like image 105
ASGM Avatar answered Oct 15 '22 00:10

ASGM