Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Efficiently finding shape over multiple rows in pandas dataframe

This is about efficiently finding "2D shapes" in a pandas dataframe. We have a frame generated like this:

l = ['>','<','v','^']

def gendf(r,c): 
    return pd.DataFrame(np.random.choice(l,size=(r,c)),columns=[i for i in range(c)])

I'm looking for the following shape:

    ^
<   ?   >
    v

With "?" being any character. I have a very ineffective nested loop which does the job, slowly:

def co(df):
    c=[]
    for i in list(df.index)[:-2]:
        for j in list(df.columns)[1:-1]:
            if df.loc[i,j]=='^':
                if df.loc[i+2,j]=='v' and df.loc[i+1,j-1]=='<' and df.loc[i+1,j+1]=='>':
                    c.append((i,j))
    return c

It just looks for the "top" and checks for the rest when it finds it. As a nested loop, it's very slow - any functions allow me to do this more efficiently?

EDIT: timers for the methods given

%timeit co(df)
8.08 s ± 260 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit x1(df) #Chris A's method
1.89 s ± 102 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit x2(df) #sampers' method
6.22 s ± 289 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Let me know if there's an even better way - this already helps a lot!

like image 955
Jim Eisenberg Avatar asked Aug 09 '26 05:08

Jim Eisenberg


1 Answers

IIUC, you could try numpy.argwhere with multiple boolean conditions using DataFrame.shift and DataFrame.eq:

# setup
np.random.seed(0)
df = gendf(100, 100)

cond1 = df.eq('<').shift(axis=1)
cond2 = df.eq('>').shift(-1, axis=1)
cond3 = df.eq('^').shift()
cond4 = df.eq('v').shift(-1)

coords = np.argwhere((cond1 & cond2 & cond3 & cond4).to_numpy())

[out]

[[ 1  1]
 [ 3 85]
 [11 16]
 [14 87]
 [22 92]
 [24 25]
 [25 14]
 [26 38]
 [27 69]
 [27 85]
 [32 54]
 [36 61]
 [39 29]
 ...
 [69 82]
 [70 76]
 [76 41]
 [79 81]
 [81 60]
 [82 13]
 [83 83]
 [87 12]
 [89 16]
 [94  5]
 [94 69]
 [98 16]]
like image 152
Chris Adams Avatar answered Aug 11 '26 06:08

Chris Adams



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!