I wish to keep sym file small. How can I find out if I will be introducing a new symbol before splaying the data out to disk?

You can detect if the use of a symbol will increased the total number of interned symbols in the pool using .Q.w:

$ rlwrap q
KDB+ 2.7 2011.02.16 Copyright (C) 1993-2011 Kx Systems
q).Q.w[]
used| 108432
heap| 67108864
peak| 67108864
wmax| 0
mmap| 0
syms| 537
symw| 15616
q)`1              // force kdb to internalize a new symbol
`1
q).Q.w[]
used| 108432
heap| 67108864
peak| 67108864
wmax| 0
mmap| 0
syms| 538         // increased by 1
symw| 15634       // increase of +18 bytes
q)`12             // force kdb to internalize another
`12
q).Q.w[]
used| 108432
heap| 67108864
peak| 67108864
wmax| 0
mmap| 0
syms| 539         // increase by 1
symw| 15653       // increase of 19 bytes
q)`123
`123
q).Q.w[]
used| 108432
heap| 67108864
peak| 67108864
wmax| 0
mmap| 0
syms| 540         // increase by 1
symw| 15673       // increase of 20 bytes
q)

See this solution to compacting a bloated sym file

Why doesn’t my query ‘select from reallybigtable’ ever return?

It’s a valid query, but q will run out of memory before returning. You’ll have to add some constraints to the query to reduce the size of the result set. Something like

select from reallybigtable where date = YYYY.MM.DD

would be a good start.

How do i interpret the output of function .Q.w[]?

An example explains it best:

$ rlwrap q
KDB+ 2.7 2011.02.16 Copyright © 1993-2011 Kx Systems
q).Q.w[] used| 108432   / bytes malloced
heap| 67108864 / heap bytes available
peak| 67108864 / heap high-watermark in bytes
wmax| 0        / workspace limit from -w param
mmap| 0        / amount of memory mapped
syms| 537      / number of symbols interned
symw| 15616    / bytes used by 537 symbols
q)

Note how .Q.w is simply a pretty print output of \w and the workspace memory footprint from \w 0

q).Q.w
k){`used`heap`peak`wmax`mmap`syms`symw!(.”\\w”),.”\\w 0″}
q)\w
108304 67108864 67108864 0 0j
q)


This work is licensed under a Creative Commons License.
The views and opinions expressed herein are those of the authors and do not necessarily reflect those of any other person or legal entity.
Kdb+ is the registered trademark of Kx Systems, Inc.