Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas count number of elements in each column less than x

I have a DataFrame which looks like below. I am trying to count the number of elements less than 2.0 in each column, then I will visualize the result in a bar plot. I did it using lists and loops, but I wonder if there is a "Pandas way" to do this quickly. Thanks!

x = [] for i in range(6):     x.append(df[df.ix[:,i]<2.0].count()[i]) 

then I can get a bar plot using list x.

          A          B          C          D          E          F  0       2.142      1.929      1.674      1.547      3.395      2.382   1       2.077      1.871      1.614      1.491      3.110      2.288   2       2.098      1.889      1.610      1.487      3.020      2.262     3       1.990      1.760      1.479      1.366      2.496      2.128     4       1.935      1.765      1.656      1.530      2.786      2.433 
like image 364
marillion Avatar asked May 23 '14 16:05

marillion


People also ask

How do I count the number of elements in a column in pandas?

To count the number of occurrences in e.g. a column in a dataframe you can use Pandas value_counts() method. For example, if you type df['condition']. value_counts() you will get the frequency of each unique value in the column “condition”.

How do you count the number of unique values in a column in a DataFrame?

You can use the nunique() function to count the number of unique values in a pandas DataFrame.


1 Answers

In [96]:  df = pd.DataFrame({'a':randn(10), 'b':randn(10), 'c':randn(10)}) df Out[96]:           a         b         c 0 -0.849903  0.944912  1.285790 1 -1.038706  1.445381  0.251002 2  0.683135 -0.539052 -0.622439 3 -1.224699 -0.358541  1.361618 4 -0.087021  0.041524  0.151286 5 -0.114031 -0.201018 -0.030050 6  0.001891  1.601687 -0.040442 7  0.024954 -1.839793  0.917328 8 -1.480281  0.079342 -0.405370 9  0.167295 -1.723555 -0.033937  [10 rows x 3 columns] In [97]:  df[df > 1.0].count()  Out[97]: a    0 b    2 c    2 dtype: int64 

So in your case:

df[df < 2.0 ].count()  

should work

EDIT

some timings

In [3]:  %timeit df[df < 1.0 ].count()  %timeit (df < 1.0).sum() %timeit (df < 1.0).apply(np.count_nonzero) 1000 loops, best of 3: 1.47 ms per loop 1000 loops, best of 3: 560 us per loop 1000 loops, best of 3: 529 us per loop 

So @DSM's suggestions are correct and much faster than my suggestion

like image 64
EdChum Avatar answered Sep 18 '22 17:09

EdChum