How can I measure the performance of my q code?

The simplest way to time something in q is to use the \t command:

q)f: {[] do[100000; 2 + 2]}
q)\t f[]

\t executes its argument and returns the number of milliseconds that elapsed – i.e., wall clock time (except when you are using \t in its other sense, i.e., setting or getting the time between timer events).

When using kdb as a database server, you will want to use one or more of Simon’s logging scripts as a starting point. These tremendously useful scripts enable you to record every incoming query as well as measure the time taken by every incoming query. That way you can calculate summary statistics on actual database performance in order to find and fix intermittent problems.

I know k sits under q. By porting q code to k, will it run faster?

No. The interpreter boils it down to the same bytecode.

Consider Problem 1 from Project Euler: sum the multiples of 3 and 5 which are less than 1000. You may want to try this before reading further. The problems from Project Euler are great exercises for learning q!

The code below shows one solution written in both languages; f is written in q, while g is written in k.

q)f: {(+/) distinct where (0 = x mod 3) | 0 = x mod 5}
g: {+/?&(0={x-y*x div y}[x;3])|0={x-y*x div y}[x;5]}

You can see the bytecode for a function (as well as some generally more useful information, like the function’s arguments) by calling value on it:

q)value f
k){x-y*x div y}
“{(+/) distinct where (0 = x mod 3) | 0 = x mod 5}”

As you can see, when passed a function, value returns a list. The first element of that list is the function’s bytecode. Now it is easy to show that f and g have the same bytecode:

q)first value f
q)first value g
q)all (first value f) = first value g

In q, however, you probably would have written the solution a tiny bit differently:

q)f: {sum distinct where (0 = x mod 3) | 0 = x mod 5}

This results in a q version that is actually two bytes smaller and (maybe) a hair faster:

q)first value f
q)first value g
q)\t do[100000; f til 1000] 3240
q)\t do[100000; g til 1000] 3253

Thanks to Charlie Skelton from kx for pointing out errors in the original version of this article.

I noticed a list with a `s# prefix attached. What does this mean?

`s is one of q’s data attributes, and indicates that the data in the list is sorted. When q knows that a list is sorted, it can perform some operations much faster. In contrast to the other attribute types, the sorted attribute has zero memory overhead (no data structure needs to be created to support it):

q)d: (-10000 ? `4) ¡ til 10000
q)key d
q)value d
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ..
q)\t do[100000; d key[d] 5000] 443
q)d: (asc key d) ! value[d] iasc key d
q)key d
q)\t do[100000; d key[d] 5000] 89

Notice that value d above did not have the `s attribute; that is, the til function does not return a list with the `s attribute applied. In general, for a list to have the `s attribute, you must either sort the list (with asc as above) or let q know explicitly that the list is sorted by applying the `s attribute to a list that you know is already sorted:

q)x: `s # til 10
`s#0 1 2 3 4 5 6 7 8 9

If you try to apply `s to a list that isn’t actually sorted, you’ll get an ‘s-fail error:

q)`s # 3 2 1

You can add elements to the end of a sorted list without losing the sorted attribute if the new elements do not violate the sorted-ness of the result:

q)x: `s # 1 2 3
`s#1 2 3
q)x ,: 4 5 6
`s#1 2 3 4 5 6
q)x ,: 7 7 7
`s#1 2 3 4 5 6 7 7 7

However, other modifications to a sorted list will cause the list to lose its attribute:

q)x: `s # 1 2 3
`s#1 2 3
q)x, 0
1 2 3 0
q)reverse x / reverse function removes it
3 2 1

This is true even if the modifications result in a sorted list:

q) x: `s # 1 2 3
q)1 _ x
2 3
q)2 # x
1 2

You can remove the sorted attribute explicitly by applying a null attribute to the list:

q)`#1 2 3 / application of a null attribute
1 2 3

See also the following function definitions: attr and #.

For even more detail, see Section 42 of the Abridged Q Language Manual.

This work is licensed under a Creative Commons License.
The views and opinions expressed herein are those of the authors and do not necessarily reflect those of any other person or legal entity.
Kdb+ is the registered trademark of Kx Systems, Inc.