find the nearest previous negative value in a column

Question

I have a data frame df:

library(tidyverse)
t <- c(103,104,108,120,127,129,140,142,150,151,160,177,178,183,186,187,191,194,198,199)
w <- c(1,1,1,-1,-1,-1,-1,-1,1,1,-1,-1,1,1,1,-1,1,1,-1,-1)

df <- data_frame(t, w)

> dput(df)
structure(list(t = c(103, 104, 108, 120, 127, 129, 140, 142, 
150, 151, 160, 177, 178, 183, 186, 187, 191, 194, 198, 199), 
w = c(1, 1, 1, -1, -1, -1, -1, -1, 1, 1, -1, -1, 1, 1, 1, 
-1, 1, 1, -1, -1)), .Names = c("t", "w"), row.names = c(NA, 
-20L), class = c("tbl_df", "tbl", "data.frame"))

> df
# A tibble: 20 x 2
       t     w
   <dbl> <dbl>
 1   103  1.00
 2   104  1.00
 3   108  1.00
 4   120 -1.00
 5   127 -1.00
 6   129 -1.00
 7   140 -1.00
 8   142 -1.00
 9   150  1.00
10   151  1.00
11   160 -1.00
12   177 -1.00
13   178  1.00
14   183  1.00
15   186  1.00
16   187 -1.00
17   191  1.00
18   194  1.00
19   198 -1.00
20   199 -1.00

Now, if the value in w is larger than zero, find the nearest previous negative w, and assign the difference between the corresponding t values to a new column d. Otherwise, d is equal to zero. I.e. the desired output should look like this:

       t     w   d
     103  1.00  NA   (there is no previous w < 0)
     104  1.00  NA   (there is no previous w < 0)
     108  1.00  NA   (there is no previous w < 0)
     120 -1.00   0
     127 -1.00   0
     129 -1.00   0
     140 -1.00   0
     142 -1.00   0
     150  1.00   8   = 150 - 142
     151  1.00   9   = 151 - 142
     160 -1.00   0
     177 -1.00   0
     178  1.00   1   = 178 - 177
     183  1.00   6   = 183 - 177
     186  1.00   9   = 186 - 177
     187 -1.00   0
     191  1.00   4   = 191 - 187
     194  1.00   7   = 194 - 187
     198 -1.00   0
     199 -1.00   0

(The NAs above might be zero as well.)

Since yesterday I'm trying to attack this problem using findInterval(), which(), etc. but without success. Another way I was thinking about is to introduce somehow a variable shift in lag() function...

Ideally, I would like to have a tidyverse-like solution.

Any help would be very much appreciated. Thank you in advance!

Answer 1

Using data.table (since tidyverse currently has no non-equi joins):

library(data.table)
DT = data.table(df)

DT[, v := 0]
DT[w > 0, v := 
  DT[w < 0][.SD, on=.(t < t), mult="last", i.t - x.t]
]

      t  w  v
 1: 103  1 NA
 2: 104  1 NA
 3: 108  1 NA
 4: 120 -1  0
 5: 127 -1  0
 6: 129 -1  0
 7: 140 -1  0
 8: 142 -1  0
 9: 150  1  8
10: 151  1  9
11: 160 -1  0
12: 177 -1  0
13: 178  1  1
14: 183  1  6
15: 186  1  9
16: 187 -1  0
17: 191  1  4
18: 194  1  7
19: 198 -1  0
20: 199 -1  0

It initializes the new column to 0, then replaces it on the subset of rows where w > 0. The replacement uses a join of the subset of data, .SD, where w > 0 to the part of the table where w < 0, DT[w < 0]. The join syntax is x[i, on=, j] where in this case...

• x = DT[w < 0]
• i = .SD = DT[w > 0]

The join uses each row of i to look up rows in x based on the rules in on=. When multiple matches are found, we take only the last (mult = "last").

j is what we use the join to do, here calculate the difference between two columns. To disambiguate columns from each table, we use prefixes x.* and i.*.

---

Using cummax. I'm not sure if this generalizes, but it works for the example:

DT[, v := t - cummax(t*(w < 0))]
DT[cumsum(w < 0) == 0, v := NA]

I guess this requires that the t column is sorted in increasing order.