Monday, January 9, 2017

Lispy indexing

I would assume that most coders out there know at least a couple of ways to get around the problem of quickly and flexibly indexing data on composite keys. I know I'm responsible for my share of the maps of maps and fixed format string key karma. It's Greenspun's Tenth Rule for indexing; any project that doesn't have access to general purpose indexes will eventually solve the same problems in other, less rational ways.

general purpose indexes


I've come to define a general purpose index as an ordered set of records with optionally unique composite keys. The Common Lisp implementation described here uses functions to specify keys. Since indexes delegate actual indexing to ordered sets, they support the same features out of the box.


transactions


The other reason many projects develop an SQL dependency is transaction support. As data and access patterns become more complex, manual coordination quickly looses it's appeal. Transaction support may sound daunting, but we're not aiming for the clouds. Assuming single threaded use; automatically tracking added and removed items, and performing opposite operations on rollback; is trivial to implement and frees mental resources for more interesting tasks.


A final example before we wrap up, this one shows how the API supports non unique use:


A full implementation of these ideas and more may be found here.

peace, out