Sumologic: How to get average time difference between two messages

Question

Having a set of logs like:

Log10:[requestId=2][taskId=C][message='End']
Log9: [requestId=2][taskId=C][message='Start']
Log8: [requestId=2][taskId=B][message='End']
Log7: [requestId=1][taskId=B][message='End']
Log6: [requestId=1][taskId=B][message='Start']
Log5: [requestId=1][taskId=A][message='End']
Log4: [requestId=2][taskId=B][message='Start']
Log3: [requestId=2][taskId=A][message='End']
Log2: [requestId=2][taskId=A][message='Start']
Log1: [requestId=1][taskId=A][message='Start']

First, I wanted to calculate the avg time each task takes to complete. I was able to that with transactionize:

* | concat(requestId,":",taskId) as transactionKey | transactionize transactionKey avg(_group_duration) group by taskId

Now, I'm willing to know how much time (avg) is happening between one task finishes and the next one is starting.

In this concrete example, my desired output would be:

((Log9 - Log8) + (Log4 - Log3) + (Log6 - Log5)) / 3

Any clue is appreciated.

Answer 1

Thanks to @chadoliver, he pointed me to the diff operator.

* | keyvalue auto | diff _messagetime by requestId | where message = "End" | avg(_diff) | ceil(_avg)