Tuesday, March 15, 2011

web2py, shelve, debian FHS

Debian has a Filesystem Hierarchy Standard (FHS), which aids System Administration (in my opinion)

You can read more about the standard here [ wiki.debian.org ]

When new packages are added to Debian, one of the things that a packager might do, is to adjust paths (where necessary) to fit with that FHS.

web2py is not a new project, however it is new for Debian packaging.

I am currently experimenting with test .deb files which are being prepared for Debian, so as, to get a better feel for how web2py is likely to eventually install.
( Note: These are unofficial .deb files in their current state, so they should only be installed in a test environment / at users own risk )

Taking into account the FHS it would be really good if the caching (shelve) code here could be configurable.

Having the cache for the 'welcome' application at:


...might be possible with some configuration.

This is just a loose suggestion, as I do not personally know enough about Python shelve, and the dbm setup it uses to actually store a serialization.

If you see an error message like:

ERROR:web2py.cache:corrupted file: ~/web2py/applications/welcome/cache/cache.shelve

then it might just be that there is some setup error, that has made the Python shelve cache not be writeable.
( If you want to look more closely at try catch Python code, then look in web2py.gluon.cache )

Trying the following in a terminal:

touch ~/web2py/applications/welcome/cache/cache.shelve

if you see 'Permission denied', then you might want to look at your config more closely.

Does web2py require Java?

The answer is NO, but what might be puzzling is why you might ask that question.

Here the startup script starts the web2py local server, web2py cron, and then hands off to the system configured browser.

In my case x-www-browser is set to be Epiphany, and those OpenJDK Java messages, are Epiphany being chatty about loading it's Java web plugin.

Notes and Further Reading:

As I write this I am aware that version 1.93 of web2py is now available, and I will tomorrow purge and reinstall to bring my installation up to date with the latest test .deb

Although web2py does not require Java, there are some pages where you might see them mentioned together. Specifically web2py is capable of being made to work indirectly via Jython. However there are reasons why you might want to read up, before experimenting with such a path.


Gary said...

I was interested in what web2py on my 64 bit debian machine is using to 'shelve', and here is some info:

cache/cache.shelve: Berkeley DB (Hash, version 9, native byte-order)

>>> import whichdb
>>> whichdb.whichdb('/home/gnubyexample/web2py/applications/test1/cache/cache.shelve')

From the Python shelve documentation:
"The choice of which database package will be used (such as dbm, gdbm or bsddb) depends on which interface is available."

Note: That cache.shelve shown above is not in /applications/welcome/ but is from a personal test app, which would not have any write permission problems (see main post)

apt-get install python-gdbm

Above will give you an extra option in terms of manually deciding how to shelve.

However depending on how your system is configured, you might find that anydbm calls still create using 'dbhash'

Gary said...

Quote from Martelli "Python in a Nutshell":

"When anydbm.open() creates a new DBM file, open chooses the first available DBM module in order of preference:
dbhash, gdbm, dbm, or dumbdbm."

Apparently dbhash is a little preferable to gdbm, however I know little about the internals myself, to be able to comment further on this.

( Perhaps it is just a preference based on cross platform availability of the DBM modules )

Gary said...

My last comment was relevant to Python 2 (python in current Debian), however it is all change for Python3

"The dbhash module has been deprecated for removal in Python 3.0."


Seems that in future, Python will use the generic module dbm in place of anydbm, and the choices of underlying database will be gdbm, ndbm, or a naive alternative.

Gary said...

gdbm is a GNU project and implements a C library which other applications can utilize.

The python documentation for gdbm is useful, but probably needs a few more examples.

Here is a Batman, Robin example of using gdbm from Python including how to delete an unwanted key.

Extract is from Programming Python By Mark Lutz