Menno's Musings

software | life | whatever

Sat, 30 Jan 2010

At my employer we are in the process of migrating from Python 2.4 to 2.6. When running some existing code under Python 2.6 we started getting DeprecationWarnings about "object.__new__() takes no parameters" and "object.__init__() takes no parameters".

A simple example that triggers the warning:

class MyClass(object):

    def __new__(cls, a, b):
        print 'MyClass.__new__', a, b
        return super(MyClass, cls).__new__(cls, a, b)

    def __init__(self, a, b):
        print 'MyClass.__init__', a, b
        super(MyClass, self).__init__(a, b)

obj = MyClass(6, 7)

This gives:

$ python2.4 simple-warning.py
MyClass.__new__ 6 7
MyClass.__init__ 6 7

$ python2.6 simple-warning.py
MyClass.__new__ 6 7
simple-warning.py:5: DeprecationWarning: object.__new__() takes no parameters
  return super(MyClass, cls).__new__(cls, a, b)
MyClass.__init__ 6 7
simple-warning.py:9: DeprecationWarning: object.__init__() takes no parameters
  super(MyClass, self).__init__(a, b)

It turns out that a change to Python for 2.6 (and 3) means that object.__new__ and object.__init__ no longer take arguments - a TypeError is raised when arguments are passed. To avoid breaking too much pre-existing code, there is a special case that will cause a DeprecationWarning instead of TypeError if both __init__ and __new__ are overridden. This is the case we were running into with our code at work.

The reason for this change seems to make enough sense: object doesn't do anything with arguments to __init__ and __new__ so it shouldn't accept them. Raising an error when arguments are passed highlights code where the code might be doing the wrong thing.

Unfortunately this change also breaks Python's multiple inheritance in a fairly serious way when cooperative super calls are used. Looking at the ticket for this change, this issue was thought about but perhaps the implications were not fully understood. Given that using super with multiple inheritance is common and "correct" practice, it would seem that this change to Python is a step backwards.

I just scratched an itch by writing a small plugin for PyBlosxom that allows the rst (reStructured Text) and readmore plugins to work together [1]. It defines a reST "break" directive which gets transformed into the breakpoint string the readmore plugin looks out for. This allows for "Read more..." breaks to be inserted in for reST based articles.

For further information see the Code page here and at the top of the plugin itself.

Sun, 24 Jan 2010

IMAPClient 0.5.2 has just been released. This release fixes 2 bugs (#28 and #33). Much thanks to Fergal Daly and Mark Eichin for reporting these bugs.

Install from the tarball or zip or upgrade using easy_install or pip.

Tue, 12 Jan 2010

Here's something I just learned the hard way.

If you edit a crontab with "crontab -e", cron won't reload the updated crontab immediately. Changes will be read at 1 second past the next minute boundary. For example, if you change the crontab at 10:54:32, cron will reload it at 10:55:01. This means if you're trying to test how something runs under cron and you're impatient so you set that thing to run at the next minute, you won't see it run!

I spent a good half hour chasing my tail on this one. Set the test entry to run 2 minutes ahead instead.

Sat, 26 Dec 2009

I've had several requests over the last few weeks to open up the IMAPClient Trac instance so that anyone can submit tickets. Initially I changed access levels so that anoymous users could create tickets. This turned out to be fairly inflexible: it doesn't allow people to add attachments or modify tickets later. This approach also resulted in one strange ticket being created where all fields were filled with random characters - a bot looking for buffer overruns?

Since then, I've disabled anonymous ticket creation and have set up the fantastic AccountManagerPlugin which allows people to register accounts for themselves. Once someone has created an account and logged in they can create and modify tickets. I have a feeling I'm going to have to turn on the optional CAPTCHA support, but I'm willing to see how it goes for a while first.

Tue, 15 Dec 2009

I've just made a quick point release of IMAPClient. Mark Hammond is interested in using it for a project he's working on but the licenses (GPL and MPL) were incompatible. I was thinking about relaxing the license of IMAPClient anyway so this presented a good opportunity to make the switch.

Work on the 0.6 release is coming along. This version will fix a number issues with parsing of FETCH responses - the FETCH parsing code is being completely rewritten. This is the first time that IMAPClient will bypass most of imaplib for some functionality. It's looking like that at some point IMAPClient may not use imaplib at all.

IMAPClient can be installed from PyPI using pip install IMAPClient or easy_install IMAPClient. It can also be downloaded from the IMAPClient project page. As always, feedback and patches are most welcome.

Wed, 25 Nov 2009

I've had a few requests for the little hack I created to import comments from PyBlosxom into Disqus. A cleaned up version of disqus-import.py is now on the Code page. There's some docs at the top of the file.

Thu, 19 Nov 2009

I recently sorted out an issue with the IMAPClient Trac instance that's been bugging me for a while.

The problem was that whenever the web server logs were rotated logrotate would restart Lighttpd. The web server restart would in turn restart the Trac (FastCGI) processes. Unfortunately, the Trac processes would fail to start with the following error.

pkg_resources.ExtractionError: Can't extract file(s) to egg cache

The following error occurred while trying to extract file(s) to the Python egg
cache:

  [Errno 13] Permission denied: '/root/.python-eggs'

The Python egg cache directory is currently set to:

  /root/.python-eggs

Bang, no IMAPClient web site (the rest of the site was ok). To band-aid the problem when it happened (and I noticed!) I issue a sudo /etc/init.d/lighttpd restart and everything would be fine again.

After some investigation I found that running /etc/init.d/lighttpd restart as root always triggered the problem where-as restarting using sudo always worked. My guess is that restarting when logged in as root was leaving $HOME at /root even after Lighttpd had dropped to its unprivileged user account. The unprivileged user isn't allowed to write to /root so Trac blows up. setuptools seems to use $HOME instead of looking up the actual home directory of the current user.

The fix for me was to set the PYTHON_EGG_CACHE environment variable for the FastCGI processes to somewhere they are allowed to write to. This is done with the bin-environment option if you're using Lighttpd like me.

I imagine similar problems can happen with any Python app deployed using FastCGI.

Thu, 05 Nov 2009

I recently moved my blog comments away the PyBlosxom comments plugin to a hosted system. The main driver was the ability for people to subscribe to comments for an article using email or RSS. It's a pain for people to have to check back to the site to see if someone has replied to their comments. I was also keen on user-experience-enhancing features such as integration with external systems like OpenID, Twitter and Yahoo.

My criteria were:

• email subscription, RSS a bonus
• support for pre-formatted text sections in comments (essential for code samples)
• an import mechanism for existing comments
• threading of comments to allow commenters to respond to each other sensibly
• clean look with some ability to customise
• support for a variety of authentication/profile systems

There are a number of hosted comment systems out there. The most popular options seem to be Disqus, JS-Kit Echo and IntenseDebate.

IntenseDebate was eliminated first because it doesn't seem to provide an import mechanism for custom websites. Import only seems to be supported for well known blog platforms such as Wordpress. There's no comment API either. The approach seems to be to leave your old comment system in place and just have new comments go into IntenseDebate. Not good enough, I wanted to completely replace the existing comments system.

After some deliberation I decided on JS-Kit Echo for one tiny reason: it supports the <pre> tag. The closest Disqus supported was the <code> tag which doesn't preserve white-space (useless for Python code samples).

So I paid my US$12 (it's the only service that doesn't have a free option) and started looking at how to import my existing comments using their API ... and quickly found that it sucks. Comments can be submitted but you can't specify a timestamp so they are dated with the import date. Far from ideal. Then there's the API for retrieving comments: it returns the data as JavaScript code (no not JSON)! It's pretty clear that the API is what they use with the JavaScript for Echo itself and geared for that use only. They've just thrown it out there and documented it, warts and all.

Back to the drawing board.

The only showstopper for Disqus was the lack of <pre>. Everything else about it was great: it met all my requirements and the API was clean and comprehensive. If only there was a way to have properly formatted source code in the comments.

Light bulb moment: use a CSS hack to make <code> in comments behave like <pre>. The trick is to turn code into a block element and change how white-space is handled. The CSS snippet looks like:

.dsq-comment-message code {
  display:block;
  white-space:pre;
}

Works great.

With the only blocker gone, I wrote a Python script with the help of Ian Lewis' excellent disqus-python-client package to pull in the existing comments from the old system. Within an hour or so it was ready to go.

Hopefully this article saves someone else some time if they decide to use one of these systems. Getting things running chewed up a lot more time then I had expected.

Sun, 18 Oct 2009

I've just (finally) finished setting up a proper website for IMAPClient. The new home for the project is http://imapclient.freshfoo.com/.

It's a Trac instance with Mercurial support that monitors the main trunk repository. All items from the TODO file in the source have been converted to tickets in the bug tracker. I've even created a hokey little logo.

Let me know me know if anything looks broken.

Time to work on some long-standing bugs...

Here:

• Photos
• Articles
• Code - browse | repos
• Presentations
• Feeds
• RSS
• RSS 2.0
• Atom
• Back to top

Tags:

Highlights:

• Linux on the Sony Vaio VGN-BZ11
• Using Optus 3G with Linux
• Catching Unhandled Exceptions in .NET
• PyCon Italia Presentation
• Fedora on an AOpen 1557GLS
• GO-Sim Review

Archives:

• Jan 2010
• Dec 2009
• Nov 2009
• Oct 2009
• Aug 2009
• Jul 2009
• Jun 2009
• Apr 2009
• Mar 2009
• Jan 2009
• Dec 2008
• Oct 2008
• Sep 2008
• Aug 2008
• Jul 2008
• Jun 2008
• May 2008
• Apr 2008
• Jan 2008
• Nov 2007
• Oct 2007
• Sep 2007
• Jul 2007
• May 2007
• Apr 2007
• Mar 2007
• Feb 2007
• Jan 2007
• Nov 2006
• Oct 2006
• Sep 2006
• Aug 2006
• Jul 2006
• Jun 2006
• May 2006
• Apr 2006

Links:

• BATS Trading (my employer)
• Resolver Systems
• London Python

Contact me | Back to top
© 2009 Menno Smits | Based on design by Andreas Viklund