Entries in apply (3)

How do I use the functional forms of apply and amend?

Typical q code operates on all of the elements of a container at once:

q)container: 1 2 3
q)100 * container
100 200 300
q)

Sometimes we are interested in only a subset of elements from a container:

q)container: til 10
q)container
0 1 2 3 4 5 6 7 8 9
q)container[where 0 = container mod 2]
0 2 4 6 8
q)                                                                                                         

Sometimes, however, you need to update particular elements of a structure while leaving the remaining elements unchanged. That's what functional apply and amend are for; they transform specific elements of a container without touching the others. The variations are distinguished by 3 choices:

1. @ or .

Which operator is used, @ or ., determines the interpretation of the indices used to select the elements to transform.

2. container or name

The first argument is either the value of a container or the name of a global variable referring to a container. In the former case, a new object is returned; in the latter, the global variable is modified and its name is returned.

3. monadic or dyadic function

If the transformation requires additional information beyond that contained in each element itself, that is accomplished by using a dyadic function and supplying the additional information in a fourth argument to the operator.

For a detailed discussion, please see the following faqs:

Functional @ (apply)
Functional . (apply)

Lastly, note that there is another pair of overloads for @ and . - each with three arguments - called protected execution, which are invoked when the first argument to @ or . is a function or projection; protected execution is discussed in another faq.

How do I use the functional form of @ (at) apply?

Short answer:

1. @[container; indices; function]
2. @[container; indices; function; second_args]

(Note that there is another overload for @ with three arguments, called protected execution, which is invoked when the first argument to @ is a function or projection; protected execution is discussed in another faq.)


In the 3-argument case, q applies function to those elements of container specified by indices, leaving the rest of container alone. In other words, the behavior of 3-argument @ (apply) resembles that of the following function:

Although this model breaks down when the first argument is a global variable name, it is very helpful in understanding what is going on even in that case. Let's look at a couple of simple examples to clarify this:

q)list: 1 + til 8
q)list
1 2 3 4 5 6 7 8
q)@[list; 0; neg]         / negate element 0
-1 2 3 4 5 6 7 8
q)list                    / list remains unmodified
1 2 3 4 5 6 7 8
q)@[list; 1 3 4; {x * x}] / square elements 1, 3, and 4
1 4 3 16 25 6 7 8
q)

The diagram below illustrates the last example above:


Because the new container has the same type and shape as the original, function must return the same type as its argument, unless the container is a mixed list (i.e., type 0):

q)@[list; 4 5 6 7; {x div 2}] / list is an int list
1 2 3 4 2 3 3 4
q)@[list; 4 5 6 7; {x % 2}]   / but % returns float
'type
q)mixed_list: 1 2, `3`4
q)type mixed_list
0h
q)@[mixed_list; 0 3; string] / anything goes with type 0
,"1"
2
`3
,"4"
q)

Next, we'll consider the second case listed at the top of this faq. When function is dyadic and a fourth argument is supplied, @ (apply) behaves like the following function:

The basic idea is still the same, i.e., to transform selected elements while leaving the rest of the container intact. The difference is that, instead of function being modified by each, it is modified by '(each-both) (also see this faq on each-both and multivalent each), so that the selected elements of container are paired up with the corresponding elements from second_args:

q)@[list; 1 3 5 7; +; 10 20 30 40]
1 12 3 24 5 36 7 48
q)@[list; (1 3 5 7; 0 2 4 6); *; 2 -2]
-2 4 -6 8 -10 12 -14 16
q)

One function often used with 4-argument @ (apply), is : (amend). We can replace selected elements with a certain value (or values) as follows:

q)list: 1 + til 8
q)@[list; 1 3 5 7; :; 47]
1 47 3 47 5 47 7 47
q)@[list; 1 3 5 7; :; 10 20 30 40]
1 10 3 20 5 30 7 40
q)

It might appear that the above example exposes a flaw in our model, apply_dyadic, of @ (apply). However, recall that, when modified by an adverb, : (amend) does not modify its first argument:

q):'[list 1 2 3; 20 30 40]
20 30 40                     / :' returns its 2nd argument
q)list                       / without modifying its 1st
1 2 3 4 5 6 7 8
q)

Where our models do break down is when, as alluded to earlier, the first argument is the name of global variable referring to a container, rather than the value of a container. In that case, the mechanics of the operation are the same, but original container is modified, and the return value is the name:

q)list: 1 + til 8
q)@[`list; 0; neg]
`list
q)list
-1 2 3 4 5 6 7 8
q)

This behavior is handy for writing functions of your own that work in both scenarios - either creating a new value or modifying one in place - like the following:


We can also use @ to apply a function to selected entries in a dictionary:

q)dictionary: `a`b`c`d ! `1`2, 3 4
q)dictionary
a| `1
b| `2
c| 3
d| 4
q)@[dictionary; `a`b; {"I"$ string x}]
a| 1
b| 2
c| 3
d| 4
q)

Tables can be transformed as well. Consider the following table:

q)t: ([] col1: `foo`bar`baz; col2: 5 10 15f)
q)t
col1 col2
---------
foo  5   
bar  10  
baz  15  
q)

Recall that each row of a table is a dictionary:

q)t 0
col1| `foo
col2| 5f
q)

Thus, we can use @ (apply) to accomplish an update in the following manner:

q)update col2 - 4 from t where i > 0
col1 col2
---------
foo  5   
bar  6  
baz  11  
q)@[t; 1 2; {[row] row[`col2] -: 4; row}]
col1 col2
---------
foo  5   
bar  6  
baz  11  
q)

We could also obtain the previous result using nested calls to @ (apply):

q)@[t; 1 2; @[; `col2; -[; 4]]]
col1 col2 
---------
foo  5 
bar  6
baz  11
q)

Since we can index tables using column names, we can double all of the entries in col2 as follows:

q)@[t; `col2; 2*]
col1 col2
---------
foo  10  
bar  20  
baz  30  
q)

How do I use the functional form of . (dot) apply?

Short answer:

1. .[container; indices; function]
2. .[container; indices; function; second_args]

(Note that there is another overload for . with three arguments, called protected execution, which is invoked when the first argument to . is a function or projection; protected execution is discussed in another faq.)


In the 3-argument case, q applies function to those elements of container specified by indices, leaving the rest of container alone.

The behavior of . (apply) is very similar to that of @ (apply) (which is described in another faq). The only difference between @ (apply) and . (apply) is that the indices, in the case of . (apply), are applied at depth along the dimensions of the container. The following code behaves like . (apply) for a two-dimensional container:

To explore the different treatment of the indices argument between @ (apply) and . (apply), we'll consider the simplest, multi-dimensional container: a two-dimensional list, aka a matrix.

q)matrix: 0N 4 # til 16
q)matrix
0  1  2  3 
4  5  6  7 
8  9  10 11
12 13 14 15
q)

We can use simple indexing of matrix with both @ (index) and . (index) to illustrate the difference:

q)@[matrix; 0]           // fetch row 0
0 1 2 3
q)@[matrix; 0 0]         // fetch matrix[0;0] fails.  why?
0 1 2 3                  // @ cannot index more
0 1 2 3                  // than one dimension
q)
q).[matrix; 0 0]         // dot indexes "at depth"
0

Now let's apply a function to selected elements from matrix using . (apply):

q).[matrix; (0; ::); 100+]
100 101 102 103
4   5   6   7  
8   9   10  11 
12  13  14  15 
q).[matrix; 0 0; 100+]
100 1  2  3 
4   5  6  7 
8   9  10 11
12  13 14 15
q).[matrix; (::; 0); 100+]
100 1  2  3 
104 5  6  7 
108 9  10 11
112 13 14 15
q)

Notice that we used :: in order to elide a dimension from the indices.

We can also use . (apply) with dictionaries of lists:

q)dict: `a`b ! (1 2 3; 4 5 6)
q).[dict; (`a; 0); 50*]
a| 50 2 3
b| 4  5 6
q).[dict; (::; 1); 50*]
a| 1 100 3
b| 4 250 6
q)

The general case of applying a function at depth via functional . (apply) is not implemented for tables:

q)t: ([] a: `foo`bar; b: 5 10f)
q).[t; (0; `b); %[; 5]]
'nyi
q).[t; (`b; 0); %[; 5]]
'nyi
q)

The . form of apply has another trick up its sleeve: the empty list index. When the second parameter to . (apply) is (), the entire container is passed to function in a single invocation, as the following example demonstrates:

q).[matrix; (); 0N!]
(0 1 2 3;4 25 36 7;8 81 100 11;12 13 14 15)
0  1  2   3 
4  25 36  7 
8  81 100 11
12 13 14  15
q)

When using . (apply) in this manner, we can return anything from function; the type and shape do not matter:

q).[matrix; (); {"hello"}]
"hello"
q)


Returning to the second case, when function is dyadic, . (apply) takes a fourth argument (named second_args in our example), and each indexed element of container is paired with the corresponding element of second_args. This means that second_args must conform to indices (or be an atom). The following function expresses this behavior:

(Also see this faq on each-both and multivalent each.)

We can demonstrate how this works by altering the center square of matrix:

q).[matrix; (1 2; 1 2); *; (10 20; 100 200)]
0  1   2    3 
4  50  120  7 
8  900 2000 11
12 13  14   15
q)