Message-ID: <d8ad40b50809220846x7929e3b0y68e99114c14bb20c@mail.gmail.com>
Date: 2008-09-22T15:46:52Z
From: Barry Rowlingson
Subject: Manage huge database
In-Reply-To: <644e1f320809220841j7be64fd6jfc433809e6491772@mail.gmail.com>
2008/9/22 jim holtman <jholtman at gmail.com>:
> Why don't you make one pass through your data and encode you
> characters as integers (it would appear that you only have 16
> combinations). You might also want to consider using the 'raw' object
> since these only take up one byte of storage -- will reduce your
> storage requirements by 4. Then store each row in a 'filehash' object
> so you can quickly retrieve a row at a time and then index directly to
> the byte(s) that have the information that you want.
My original response of specifying a relational database now seems
somewhat comical :)
Barry