Comparison of sublist and take operator(#) in q kdb

Question

While fetching a subset of data from a table when I use sublist or take operator(#), most of the times take operator is slow compared to sublist function.

5 observations posted in comment after querying table t in hdb which consists of 231131 rows and 71 cols.

\t 10000 10 sublist select from t where date=.z.d-5 /Time taken - 62j 92j 68j  63j 65j 
\t 10000 10#select from t where date=.z.d-5 / Time taken - 544j 546j 567j 569j 585j

With this small sample, it seems like sublist is faster as compared to take operator.
But when I see the code of sublist, it internally uses take operator, wondering how does sublist manage to be more efficient?

Answer 1

The syntax you have used for timing the operations means that you are timing two different operations.

\t 10000 10 sublist select from t where date=.z.d-5 /Time taken - 62j 92j 68j  63j 65j

This sublist operation shows a slice of length 10 starting at the 10000th entry.

\t 10000 10#select from t where date=.z.d-5 / Time taken - 544j 546j 567j 569j 585j

This # operation would return 10000 rows, each consisting of 10 rows from the result of the select function.

What you probably meant to do is:

\t:10000 10 sublist select from t where date=.z.d-5 /Time taken - 62j 92j 68j  63j 65j 
\t:10000 10#select from t where date=.z.d-5 / Time taken - 544j 546j 567j 569j 585j

Which would time each of the functions 10000 times. From my own testing, sublist and # give similar timings.