I've recently noticed that a function where I iterate over a DataFrame
rows using .iloc
is very slow. I found out that there's a faster method called .iat
, that's said to be equivalent to .iloc
. I tried it and it cut the run time down by about 75%.
But I'm a little hesitant: why is there an "equivalent" method that's faster? There must be some difference between the inner workings of these two and a reason why they both exist and not just the faster one. I've tried looking everywhere but even the pandas documentation just states that
DataFrame.iat
Fast integer location scalar accessor.Similarly to iloc, iat provides integer based lookups. You can also set using these indexers.
And that doesn't help.
Are there limits to using .iat
? Why is faster; is it sloppier? Or do I just switch to using .iat
and happily forget .iloc
ever existed?
at selects a single scalar value in the DataFrame by label only. . iat selects a single scalar value in the DataFrame by integer location only.
The iat method in pandas is used to get a single value from a dataframe based on the row and column index. The iat function needs two parameters: the row and column index. It then returns the value of the specified index.
loc can take multiple rows and columns as input arguments. . at can only take one row and one column as input arguments.
The main difference between pandas loc[] vs iloc[] is loc gets DataFrame rows & columns by labels/names and iloc[] gets by integer Index/position. For loc[], if the label is not present it gives a key error. For iloc[], if the position is not present it gives an index error.
iat
and at
gives only a single value output, while iloc
and loc
can give multiple row output.
Example: iloc[1:2,5:8]
is valid but iat[1:2,5:8]
will throw error
iat
and at
working with scalar only, so very fast. Slower, more general functions are iloc
and loc
.
You can check docs:
Since indexing with [] must handle a lot of cases (single-label access, slicing, boolean indexing, etc.), it has a bit of overhead in order to figure out what you’re asking for. If you only want to access a scalar value, the fastest way is to use the at and iat methods, which are implemented on all of the data structures.
Similarly to loc, at provides label based scalar lookups, while, iat provides integer based lookups analogously to iloc.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With