Modules that block Python’s help()

Python has somewhat insecure way of implementing builtin help in it’s interpreter. One feature that can be especially irritating is it’s module enumeration. To get it one have to enter commands:

python
help()
modules

You can get the same effect by typing

python -c "help('modules')"

in command line. The problem occurs in brain-dead implementations of modules. Some authors think that merely importing a module should start it’s operation. Effect? Modules can run arbitrary applications and arbitrary code (compiz), block help() so that it never finishes (again, compiz) or even force quit on interpreter completely (sk1). What a mess!

While it is true that such effects represent bugs in offending modules, which should be reported and fixed, I’m not willing to wait for that to use help().

There are three solutions:

  1. Find offending modules and remove them – simplest – see below
  2. Find offending modules and hardcode their – more complicated, but preferred in situations where we don’t want to remove modules
  3. Find offending modules and fix them – good idea if we want to send patches to the authors, but can take a LOT of time

Whichever way we choose, we always have to discover which module is actually causing problems. Fortunately the code of pydoc.py (module responsible for help() and friends) is very well documented and written. On Linuxes you can find them in /usr/lib/python/pydoc.py or similar location. Try

locate pydoc.py

if you have locate up and running (which is likely true). Find the definition of function listmodules. It should read like this:

    def listmodules(self, key=''):
        if key:
            self.output.write('''
Here is a list of matching modules.  Enter any module name to get more help.

''')
            apropos(key)
        else:
            self.output.write('''
Please wait a moment while I gather a list of all available modules...

''')
            modules = {}
            def callback(path, modname, desc, modules=modules):
                if modname and modname[-9:] == '.__init__':
                    modname = modname[:-9] + ' (package)'
                if find(modname, '.') < 0:
                    modules&#91;modname&#93; = 1
            def onerror(modname):
                callback(None, modname, None)
            ModuleScanner().run(callback, onerror=onerror)
            self.list(modules.keys())
            self.output.write('''
Enter any module name to get more help.  Or, type "modules spam" to search
for modules whose descriptions contain the word "spam".
''')
&#91;/sourcecode&#93;
You can easily spot the 'callback' function inside. We need to change it's definition so it will show us some debug information:
&#91;sourcecode lang='python'&#93;
            def callback(path, modname, desc, modules=modules):
                print 'callback called:', (path,modname,desc,len(modules))
                # rest of the function follows
&#91;/sourcecode&#93;
Now when we can find out where is our problem:
<pre>
[tener@laptener python2.6]$ python -c "help('modules')"

Please wait a moment while I gather a list of all available modules...

callback called: (None, '__builtin__', '', 0)
callback called: (None, '_ast', '', 1)
(...)
callback called: (None, 'simplejson.tool', '', 416)
callback called: (None, 'sip', '', 416)
callback called: (None, 'sipconfig', '', 417)
callback called: (None, 'sipdistutils', '', 418)
callback called: (None, 'sk1', '', 419)
shared memory images supported
[tener@laptener python2.6]$ # python has quit
</pre>
As you can see, the problematic package is called 'sk1'. We can either remove it from our system completely, or try to write a workaround. Unfortunately I didn't succeed in writing generic workaround in pydoc.py module, though it is likely to be possible. So what is exactly the problem with <strong>sk1</strong>? See it for yourself:

#! /usr/bin/python
# -*- coding: utf-8 -*-

# Copyright (C) 2007 by Igor E. Novikov
#
# This library is covered by GNU Library General Public License.
# For more info see COPYRIGHTS file in root directory.


import sys, os, warnings

warnings.filterwarnings("ignore")

_pkgdir = __path__[0]
sys.path.insert(1, _pkgdir)
_ttkdir = os.path.join(_pkgdir, 'app/UI/lib-ttk')
sys.path.insert(1, _ttkdir)

import app
app.config.sk_command = sys.argv[0]
app.main.main()

It should have checked the ‘__name__’ variable and start the app only if it equals ‘__main__’. You can read it in every Python tutorial for beginners… Brain-dead and nasty indeed. But easy to fix as well. This code fixes the problem:

#! /usr/bin/python
# -*- coding: utf-8 -*-

# Copyright (C) 2007 by Igor E. Novikov
#
# This library is covered by GNU Library General Public License.
# For more info see COPYRIGHTS file in root directory.


import sys, os, warnings

def main():
    warnings.filterwarnings("ignore")

    _pkgdir = __path__[0]
    sys.path.insert(1, _pkgdir)
    _ttkdir = os.path.join(_pkgdir, 'app/UI/lib-ttk')
    sys.path.insert(1, _ttkdir)

    import app
    app.config.sk_command = sys.argv[0]
    app.main.main()

if __name__ == '__main__':
    main()

We also need to fix /usr/bin/sk1:

#!/usr/bin/env python
# lengthy copyright info here 
import sk1
sk1.main()

I’ve actually filled a bug report here. I hope they’ll fix that soon.

I don’t know if in Python 3K help(‘modules’) will be any better. I guess not: from what I’ve seen it hasn’t changed much between 2.6.2 and 3.1 versions.

Update: according their SVN repo this bug is already fixed: someone has found it independently and fixed 47 hours ago. Too bad they didn’t put a ticket for bug, now they have a duplicate bug report (kind of). And I still find my patch better 🙂

Advertisements

~ by Tener on 04/08/2009.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: