How do I execute a q function call with parameters over IPC?

Short answer:

q)phandle: hopen `:serverhost:5012
q)phandle (function; arg1; arg2 ..)
q)hclose phandle

To invoke a function in another q process (i.e., a function that already exists in that q process), you need the following:

  1. a process handle to the other q process,
  2. the name of the pre-existing function in the other process, and
  3. the parameters you want to pass.

Then all you need to do is to apply the process handle to a list whose first element is the function name (as a symbol) and whose remaining elements are the arguments.

For example, let’s start one q process, which will be our server, listening on TCP port 5012. In our server, we’ll define a function square (we’ll make the background color different for the server to make it easier to distinguish from the client):

$ q -p 5012             // server
q)\p                    // display the listening port
q)square: {x * x}
q)square 3

Next, we’ll start up another q process (the client), create a process handle connected to the first q process (the server), and request the first process to compute square 5:

$ q                     // client
q)phandle: hopen `:localhost:5012
q)phandle (`square; 5)  // remote execution

To call a function with more parameters, simply add them to the end of the list. We’ll demonstrate by defining a 2-argument function on the server that calculates the length of a right triangle’s hypotenuse:

q)hypotenuse: {sqrt sum square x,y}
q)phandle (`hypotenuse; 5; 12)

What if the function you’re calling doesn’t take any parameters? For example, we’ll define a function in the server called serverTime that returns the local time according to the server:

q)serverTime: {[] .z.T}

You can’t pass zero parameters over IPC:

q)phandle enlist `serverTime  // a list with just
'length                       // the function name
q)                            // is not allowed

However, you can pass whatever you like, and it won’t matter:

q)phandle `serverTime`ignored
q)phandle (`serverTime; ())    // () is often used
q)phandle (`serverTime; ::)    // so is ::

See :: (generic null).

So far, all of our examples involved a client invoking a predefined function on the server. You can also pass a function defined on the client to be executed on the server. To see this, let’s define a global variable on the server:


Now, on the client, we’ll define a function called getSERVERGLOBAL to retrieve the value of SERVERGLOBAL on the server. Instead of passing the name of the function (i.e., `getSERVERGLOBAL), we pass the function’s value:

'SERVERGLOBAL                   // not defined here
q)phandle (getSERVERGLOBAL; ::) // note the missing `

Notice that this operation does not cause getSERVERGLOBAL to become defined on the server:


This technique can be used to run built-in functions and anonymous functions (aka lambdas) on the server as well:

q)phandle (+; 2; 4)
q)phandle ({til SERVERGLOBAL - x}; 42)
0 1 2 3 4

There is one more way to convey code to the server to run: you can pass the code in a string.

q)phandle "SERVERGLOBAL + 4"

We prefer passing a function over a string, because – especially as the expression to be passed gets more complex – it’s easier to read.

How do I get the length of a list?

Short answer:

count list
count each column_name
to get the length of a list-valued field in a query.

The length of any list can be found using the count function; the only complication occurs with certain applications of count in queries. We’ll illustrate with a table of simulated trade data:

time         sym price size
09:30:17.623 glo 26.79 3200
09:30:26.602 ojb 28.17 1600
09:30:33.657 fjc 91.31 5400
09:30:40.884 knb 59.7  2900
09:31:19.256 apl 1.26  7900

The following query uses count to return the number of rows that satisfy the where clause:

q)exec count i from trade where sym = `aif

This is usually what you want.

However, suppose we created another table (perhaps to improve the performance of certain queries) that grouped prices for each symbol:

q)grouped: select price by sym from trade
sym| price                                                   ..
---| --------------------------------------------------------..
acl| 18.1 85.5 86.31 45.65 10.91 31.25 5.49 51.66 1.02 31.82 ..
aif| 44.02 47.74 11.83 14.28 85.11 99.09                     ..
alk| 51.12 16.7 78.71 96.56 26.55 32.09 33.66 30.01 83.24 30...
amg| 59.25 56.18 92.63 46.57 57.25 14.58 69.21 88.25 94.1 28...
aog| 92.59 51.52 96.95 83.2 6.21 59.77 44.19 94.19 3.13 41.94..

In table grouped, the price column is of type list-of-float – note the uppercase type letter F:

q)meta grouped
c    | t f a
-----| -----
sym  | s
price| F

It appears that we can retrieve the prices for a given symbol easily enough:

q)exec price from grouped where sym = `aif
44.02 47.74 11.83 14.28 85.11 99.09

Suppose, however, that we want to know how many trades occurred for a particular symbol:

q)exec count price from grouped where sym = `aif

What went wrong? A where function yields a list of row indexes that meet the constraints, and then each projection (i.e., the column names between exec and from — in this case, price) yields a list of corresponding field values. Since the number of rows that met our sole constraint is 1, the result of the projection is an untyped list with a single element:

q)type exec price from grouped where sym = `aif
q)count exec price from grouped where sym = `aif

Projection results are the arguments to aggregation functions in queries. In other words, the untyped list in our example is the same as the one passed to count. Since the projection’s single element contains the list of prices we want to count, the way out is to combine count with first:

q)exec count first price from grouped where sum = `aif

Applying the same logic when counting the trades for every symbol, we need to use count each:

q)select count price by sym from grouped
sym| price
---| -----
acl| 1
aif| 1
alk| 1
amg| 1
aog| 1
q)select count each price by sym from grouped
sym| price
---| -----
acl| 14
aif| 6
alk| 10
amg| 12
aog| 13

Constraints in where clauses behave the same as well. Consider the following select statement:

x y z
a 1 "xyzzy"
b 2 "grue"
c 3 "frobozz"
q)select from t where 5 < count z
x y z

Newcomers to q often expect the above query to return the rows from t whose z column has more than 5 characters (i.e., c 3 frobozz). Rather than counting the contents of each z field, however, count is actually counting the list t `z:

q)count t `z
q)t `z

This insight suggests the solution:

q)count each t `z
5 6 7
q)select from t where 5 < count each z
x y z
c 3 "frobozz"

This work is licensed under a Creative Commons License.
The views and opinions expressed herein are those of the authors and do not necessarily reflect those of any other person or legal entity.
Kdb+ is the registered trademark of Kx Systems, Inc.