How can I capture STDOUT, STDERR and the exit status of a system command invoked from q?

Short answer: 2>&1 (i.e., redirect STDERR to STDOUT), echo $?, and add parentheses.

Note: This applies to bash, and has not been tested in csh, ksh, etc.

We can invoke arbitrary external programs from q using the system command. For example:

q)system "echo Hello, world!"
"Hello, world!"

External programs often signal errors by returning a non-zero exit code, which is stored in the shell variable ?:

$ ls nonexistent_file
ls: nonexistent_file: No such file or directory
$ echo $?
$ foo
-bash: foo: command not found
$ echo $?

When an external program called from system in q runs into a situation like this, q detects the non-zero exit code from the child process and raises a signal, ‘os:

q)system "ls nonexistent_file"
ls: nonexistent_file: No such file or directory

When trying out ideas at the console as in the above example, the full details of the error are available on the screen. Getting those error details programmatically is a bit trickier, however. Consider the following function, in which we use protected execution to invoke an error handler when the ‘os signal is raised:

use_protected_execution: {[]
: @[system;
"ls nonexistent_file";
{[error_info] -1 "error is ", error_info;}];

The error handler in use_protected_execution knows only that an operating system error occurred. None of the error details are available within the code:

ls: nonexistent_file: No such file or directory  // invisible
error is os

We can improve the situation somewhat by appending “; echo $?” to the command:

q)result: system "ls nonexistent_file; echo $?"
,"1"    // actually a list of strings

This enables us to distinguish error codes as follows:

distinguish_error_codes: {[]
result: system "ls nonexistent_file; echo $?";
exit_code: "I" $ last result;
$[0   = exit_code;
; // everything's alright
1   = exit_code;
; // minor problem
2   = exit_code;
; // major problem
127 = exit_code;
; // command not found
/ else
// unhandled exit code

What we really want, though, is to capture the output sent by the failed command to STDERR. You might expect the usual shell redirection syntax to do the job:

q)result: system "ls nonexistent_file 2>&1; echo $?"
ls: nonexistent_file: No such file or directory

No such luck. The trick is to put parentheses around the whole thing:

q)result: system "(ls nonexistent_file 2>&1; echo $?)"
"ls: nonexistent_file: No such file or directory"

We can wrap all this up in a handy function that, instead of raising os, raises the descriptive message output to STDERR by the child process:

call_external: {[command]
result: system "(", command, " 2>&1; echo $?)";
exit_code: "I" $ last result;
if [0 < exit_code;
' raze -1 _ result];
-1 _ result}

How does ‘protected execution’ (or exception handling) work in q?

Protected execution is q’s model of exception handling. If you are familiar with exception models (such as try, catch, and throw) from another language, you will notice a resemblance.

In any exception handling model, there is a block of code to attempt to execute and one or more blocks of code to invoke should the attempted block fail. Moreover, the error handling blocks have a proscribed form. Lastly, there is a mechanism to raise an exception.

Let’s look at a simple example in q and break it down:

LaidBackLoad: {[file]
@[{system "l ", x; 1b}; / code to attempt
file;                 / argument
{[err] 0N! err; 0b}]} / error handler

In q, there is only one error handling block, and both the try block and the error handling block must be functions. In addition, the error handler takes a single parameter, which – if the error handler is called – will be a string.

In our example, the function to try takes only a single parameter, and that’s why the protected execution expression starts with an @. If your try function takes more than one parameter, use . instead:

LaidBackEqual: {
.[{x = y};              / code to attempt
(x; y);               / arguments
{[err] 0N! err; 0b}]} / error handler

You signal an error (which may be a symbol or a string) using the ‘ (signal) operator:

CantBeNegative: {if [x < 0; ‘ “Must be >= 0”]; x}

This is exactly the same mechanism that q uses to let us know when we have done something wrong:

q)(1 2 3) = 1 2

There is one last detail to note when reviewing the above examples. Protected execution expressions are expressions, and they have a value. If the try function does not signal, then the expression’s value is the try function’s return value. Otherwise, the protected execution expression’s value is the value returned by the error handler.

This work is licensed under a Creative Commons License.
The views and opinions expressed herein are those of the authors and do not necessarily reflect those of any other person or legal entity.
Kdb+ is the registered trademark of Kx Systems, Inc.