« What does error output 'nyi mean? | Main | Can q script be compiled down to native code? »

I know kdb is a in-memory database. my dataset is over two terabytes in size. However my RAM is limited to 32GB. How will this work?

Like any computer process, the maximum amount of data loaded into memory is limited by the real physical memory addressable by your process. On a 32-bit server, this can be little as 2GB. On a 64-bit server, 64TB is possible as of this writing, and that limit will continue to increase in the future.

However, with memory mapped tables, the size of your table can theoretically be as large as your disk (assuming it does not exceed the kdb limitations). As long as the operations you perform on your database don't require more than your RAM can handle at once, you'll be fine. That's how kdb has been handling multi-terabyte databases for years.

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (1)

Can you give an example how to map (on linux) the disc as a table KDB+?

regards

July 30, 2011 | Unregistered Commenterdanice

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
All HTML will be escaped. Textile formatting is allowed.