For instance, if I have the below table, then I want to remove the 3rd row:
Stock Price
-------------------
GOOG 101
GOOG 102
GOOG 102 <- want to remove this
GOOG 101
Note: even though row 4 is a duplicate of row 1, I don't want to remove it as it is not a consecutive duplicate. That is, it is not a duplicate of the row immediately above.
I would also want to do check for duplicates across multiple fields, not just Price
.
d:([]Stock:4#`GOOG;Price:101 102 102 101)
q)d
Stock Price
-----------
GOOG 101
GOOG 102
GOOG 102
GOOG 101
q)d where not d~'prev d
Stock Price
-----------
GOOG 101
GOOG 102
GOOG 101
You can also use differ
q)t:([]stock:4#`GOOG; price:101 102 102 101)
q)differ t
1101b
q)t where differ t
stock price
-----------
GOOG 101
GOOG 102
GOOG 101
now let's suppose there is a time column, as you indicate in your comment above
q)t:update time:til count i from t
q)t
stock price time
----------------
GOOG 101 0
GOOG 102 1
GOOG 102 2
GOOG 101 3
q)t where differ `stock`price#t
stock price time
----------------
GOOG 101 0
GOOG 102 1
GOOG 101 3
Now going back to the t without a time column, for simplicity. This gives you speed up over the alternative method proposed by @jgleeson (which I think is great, but a speed up is always welcomed so thought I'd share this regardless)
q)\ts do[10000;r:t where differ t]
31 1184j
q)\ts do[10000;r2:t where not t~'prev t]
62 1488j
q)r~r2
1b
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With