Short answer: Check that the keys of your dictionary are uniformly atoms or lists, not both.


To experiment, let’s construct two dictionaries, one with atomic keys and one with list keys.

q)dict1: `keya`keyb`keyc ! 10 20 30
q)dict1                // dictionary with atomic keys
keya| 10
keyb| 20
keyc| 30
q)dict2: (`keya1`keya2; `keyb1`keyb2; `keyc1`keyc2) ! 1 2 3
q)dict2                // dictionary with list keys
keya1 keya2| 1
keyb1 keyb2| 2
keyc1 keyc2| 3
q)dict1[`keya]         // index by atom key
10
q)dict2[`keya1`keya2]  // index by list key
1
q)

Note how dict1 contains a keys made of exclusively of atoms, and dict2 contains keys that are lists.

Now let’s consider a dictionary whose keys are both atoms and lists:

q)dict3: (`keya1; `keyb1`keyb2; `keyc1`keyc2) ! 1 2 3
q)key dict3
`keya1                // The first key is an atomic symbol,
`keyb1`keyb2          // while the rest of the keys are
`keyc1`keyc2          // lists of symbols.
q)dict3[`keya1]       // The first key indexes fine,
1
q)dict3[`keyb1`keyb2] // but the remaining keys are broken;
0N 0N
q)dict3[`keyc1`keyc2] // they are treated as multiple keys.
0N 0N
q)

Even though the keys are present they cannot be found by a dictionary lookup. Reverse lookup does work:

q)dict3
`keya1      | 1    // ok
`keyb1`keyb2| 2    // broken
`keyc1`keyc2| 3    // broken
q)dict3 ? 2
`keyb1`keyb2
q)dict3 ? 3
`keyc1`keyc2
q)

Analogous behavior can be observed when searching a list that contains both atoms and lists:

q)list: key dict3
q)list ? `keya1
0q)list ? `keyb1`keyb2
3 3
q)

Always remember that a dictionary lookup is essentially the following:

q)lookup: {[dict; thekey] value[dict] @ key[dict] ? thekey}
q)

Sometimes types in q behave differently depending on the data contained within. Take a look at this related faq to read more.

Short answer: A failed dictionary search returns the null corresponding to the first dictionary entry’s value.


Let’s test with the following two dictionaries, sfdict and fsdict:

q)sfdict: `symbol`float ! (`abc; 123.4)  // type sym is first
q)fsdict: `float`symbol ! (123.4; `abc)  // type float is first
q)sfdict
symbol| `abc
float | 123.4
q)fsdict
float | 123.4
symbol| `abc
q)

Note that while both dictionaries contain the same key-value pairs, because dictionaries are an ordered list of key value pairs, the above two dictionaries do not match.

q)sfdict = fsdict
symbol| 1
float | 1
q)sfdict ~ fsdict
0b
q)

A dictionary search returns a null value when the key sought is not found:

q)null sfdict[`badkey]
1b
q)null fsdict[`badkey]
1b
q)

Getting a null back from a failed lookup is expected. On the other hand, many are surprised to find that the two nulls returned for the above two lookups differ:

q)sfdict[`badkey] ~ fsdict[`badkey]
0b
q)sfdict[`badkey]
`                    // null symbol
q)fsdict[`badkey]
0n                   // null float
q)

The types of the returned nulls are different according to the types of the first values in sfdict and fsdict. This is the same behavior observed when indexing a mixed list with an out-of-range index:

q)(`a; `b; 1f; 2f)[5]
`
q)(1f; 2f; `a; `b)[5]
0n
q)

There’s a similar “first element wins” behavior with the atomicity of dictionary keys. see this faq on dictionary indexing to learn more.

Short answer: cast the column to long before applying sum:

select sum `long $ size from large_table

In many programming languages, including q, anytime you add integers (unless you somehow know for sure that the sum will fit in – roughly – 31 bits) you risk integer overflow. If you’re lucky, the error will be obvious (e.g., you’ll get a negative number when you expected a positive one). Casting the arguments to sum to long will give you (almost) 63 bits of breathing room.

If your sum won’t fit in 63 bits, you’ll need to explore other options:

  • Switch to floats. Although overflow is still possible, it’s rare. However, you lose some precision.
  • Use a bignum library such as gmp.
  • Use a language (e.g., haskell or clojure) that has built-in support for arbitrary-precision arithmetic.

Lastly, keep in mind that literal values that cannot fit into an int must be suffixed with j – even if the context suggests that a long is expected:

q)select from meta large_table where c = `id
c | t f a
--| -----
id| j
q)count select from large_table where id = 84066472837652480
'84066472837652480
q)count select from large_table where id = 84066472837652480j
1
q)