Speed Tables

Karl Lehenbauer

Speed tables provides an interface for defining tables containing zero or more rows, with each row containing one or more fields. The speed table compiler reads the table definition and generates C code to create and manage corresponding structures, currently producing a set of C access routines and a C language extension for Tcl to create, access and manipulate those tables. It then compiles the extension, links it as a shared library, and makes it loadable on demand via Tcl's "package require" mechanism.

Speed tables are well-suited for applications for which this table/row/field abstraction is useful, with row counts from the dozens to the tens of millions, for which the performance requirements for access or update frequency exceed those of the available SQL database, a C compiler chain is available, and the application does not require "no transaction loss" behavior in the event of a crash.

In contrast to ad-hoc tables implemented with some combination of arrays, lists, upvar, namespaces, or even using dicts, Speed tables memory footprint is far smaller (1/10th to 1/20th the size is typical) and performance far higher. Speed tables have an Tk-like flavor, including methods supporting getting and setting values, moving array contents into and out of rows, tab-separated reading and writing to files and TCP/IP sockets, as well as a direct C interface to PostgreSQL, importing SQL query results into a speed table as well as copying from a speed table to a database table without any Tcl code running on a per row basis. Speed tables search function provides a number of powerful capabilities including query-speeding indexes, results sorting, setting offsets and limits, specifying match expressions, and counting.

Speed tables are mature, in production with tables containing millions of rows, open source, and include sixty pages of documentation.