Friday, July 1, 2011

python gdbm, like rrd, is 64 bit specific

If all your machines are 64 bit installs, then this issue is not something you will likely come across.

However if you have some 32 bit and some 64 bit and try and take mydb.gdbm across from 32 bit to 64 bit machine, or vice versa, then look out.

gdbm is wordsize specific. That means that your sample 10 row database created on a 32 bit machine, will look different than the same 10 row database created on a 64 bit machine.


gdbm fatal: lseek error

Not the most helpful message, but should you encounter this message then you have probably hit the issue I described above.

Solution: Create your data on the appropriate architecture machine.

Round robin databases (.rrd) have a similar issue and this restriction is long understood.

GDBM files are not portable between different architectures.

If a database being wordsize specific, is a serious limitation for you, then gdbm and rrd are probably not the technology you should invest your time in. Perhaps sqlite or mongodb are alternatives you might consider.


Notes and Further Reading:

gdbm was originally created as part of the GNU project.

The GNU project acted as an incubator and first implementation for this project, which has now gone on to maturity / stability.

1 comment:

Gary said...

Just how 'mature' is this gdbm thing?

I was interested so I had a look, seems version 1.1 was released in 1990

So having the database file format specific to 32 bits, was not really much of a limitation back then.

It would be another ten years or so before folks started pondering 64 bit for desktop computers.

For posterity I reproduce some of the changelog below:


Mon Apr 30 10:18:24 1990 Phil Nelson (phil at unicorn.wwu.edu)

* Version 1.2 released.

Fri Apr 27 09:49:59 1990 Phil Nelson (phil at unicorn.wwu.edu)

* Makefile: Added the dist option. Similar to the
addition from hack@ai.mit.edu, but correct.

* falloc.c (push_avail_block): Change the avail count
to reflect the correct number of entries.

* dbminit.c (dbminit): Changed the stat call to stat the
".dir" file as is done in dbmopen.c (dbm_open).

Tue Apr 10 12:18:40 1990 Phil Nelson (phil at unicorn.wwu.edu)

* Version 1.1. released.

* gdbmopen.c (gdbm_open): Clear out bucket cache.