IMAPClient now all at Bitbucket

I've been wanting to do this for a while: IMAPClient is now completely hosted on Bitbucket. The Trac instance is no more and all tickets have been migrated to the Bitbucket issue tracker. now redirects to the Bitbucket site.

The primary motivation for this change is that it makes it possible for anyone to easily interact with IMAPClient's tickets using the Bitbucket account they probably already have. Due to spam issues, I had to turn off public write access to the Trac bug tracker a long time ago meaning I was the only one creating and updating tickets. This change also means less maintenance overhead for me, so more time to spend on IMAPClient.

Please let me know if you see anything related to the migration that doesn't look right.

Go for Pythonistas

I gave a talk last night at the monthly Christchurch Python meetup titled "Go For Pythonistas". It was an introduction to the Go programming language from the perspective of a Python developer. The talk went well, with plenty of questions and comments throughout. Thanks to all who attended for your interest!

I'm not sure how useful they'll be on their own but the slides for the talk are available here.

I mentioned Juju in the talk a few times which piqued a few people's interest so I also ended up doing a short impromptu talk about that. Juju is the amazing cloud orchestration software that I hack on at Canonical these days (written in Go). It's worth a few blog articles in itself.

Inbox is sponsoring IMAPClient development

Now that they have officially launched I can happily announce that the good folks at Inbox are sponsoring the development of certain features and fixes for IMAPClient. Inbox have just released the initial version of their open source email sync engine which provides a clean REST API for dealing with email - hiding all the archaic intricacies of protocols such as IMAP and MIME. IMAPClient is used by the Inbox engine to interact with IMAP servers.

The sponsorship of IMAPClient by Inbox will help to increase the speed of IMAPClient development and all improvements will be open source, feeding directly in to trunk so that all IMAPClient users benefit. Thanks Inbox!

The first request from Inbox is to fix some unicode/bytes handling issues that crept in as part of the addition of Python 3 support. It's a non-trivial amount of work but things are going well. Watch this space...

IMAPClient 0.11

This is a somewhat belated announcement that IMAPClient 0.11 is out (and has been for a little over a week now). Notable changes:

  • Support for raw Gmail search API when connected to Gmail's IMAP
  • ENVELOPE FETCH responses are now returned as Envelope instances. These objects are namedtuples providing convenient attribute and positional based access to envelope fields and the Date field is converted to a datetime instance.
  • BODY and BODYSTRUCTURE responses are now processed recusively so multipart sections within other multipart sections are returned correctly. NOTE: code that expects the old (broken) behaviour will need to be updated.
  • SELECT response bug fix with respect to handling of square brackets in flags

Big thanks go out to John Louis del Rosario, Naveen Nathan, Brandon Rhodes and Benjamin Morrise for their contributions towards this release.

See the NEWS.rst file and manual for more details.

IMAPClient can be installed from PyPI (pip install imapclient) or downloaded from the IMAPClient site.

IMAPClient 0.10

IMAPClient 0.10 has just been released. This is an important release because it's the first to support Python 3!

Here's the highlights:

  • Python 3.2 and 3.3 are now officially supported. This release also means that Python versions older than 2.6 are no longer supported. Be sure to see the NEWS.rst file for more information on this change.
  • The HIGHESTMODSEQ item in SELECT responses is now parsed correctly
  • Fixed daylight saving handling in FixedOffset class
  • Fixed --port command line bug in imapclient.interact when SSL connections are made.
  • Michael Foord's excellent Mock library is now longer included with the IMAPClient package (it is listed as external test dependency)
  • Live tests that aren't UID related are now only run once per run
  • Live tests now perform far less logins to the server under test
  • Unit tests can now be run for all supported Python versions using tox.
  • python test now runs the unit tests
  • Many documentation fixes and improvements.

A massive thank you to Mathieu Agopian for his massive contribution to getting the Python 3 support finished. His changes and ideas feature heavily in this release.

See the NEWS.rst file and manual for more details.

IMAPClient can be installed from PyPI (pip install imapclient) or downloaded from the IMAPClient site.

Raspberry Pi: driving a VU meter using a digital-to-analog converter

As I've mentioned in previous blog articles, my wife and I have been working on driving an analog VU meter based on the sound going out the Raspberry Pi's audio outputs. This now works!

Here's a video demonstrating the result:

The music [1] is playing from a Raspberry Pi, with software running on the Pi digitally sampling the peak output audio level and writing that out to an 8-bit digital-to-analog converter (DAC). The DAC output is then used to drive the analog meter. If you're interesting in knowing how all this hangs together, keep reading.

Read more…

Monitoring Audio Levels with PulseAudio

I'm working on driving an analog VU meter from my Raspberry Pi using whatever audio is going out the Pi's sound outputs. The de facto Linux sound system, PulseAudio, allows any sound output (or "sink" in PulseAudio's nonclementure) to be monitored. In PulseAudio land, each sink has a corresponding "source" called the monitor source which can be read just like any other other PulseAudio input such as a microphone. In fact, to help with volume meter style applications, PulseAudio even allows you to ask for peak level measurements, which means you can sample the monitor sink at a low frequency, with low CPU utilisation, but still produce a useful volume display. When this feature is used, each sample read indicates the peak level since the last sample.

Read more…

IMAPClient 0.9.2

IMAPClient 0.9.2 was released yesterday. In this release:

  • The IMAP THREAD command is now supported. Thanks to Lukasz Mierzwa for the patches.
  • Enhanced CAPABILITY querying
  • Better documentation for contributors (see HACKING file)
  • Copyright date update for 2013.

See the NEWS file and manual for more details.

IMAPClient can be installed from PyPI (pip install imapclient) or downloaded from the IMAPClient site.

Note that the official project source repository is now on Bitbucket. is still the offical home page and is still used for project tracking. It is only the source respository that has moved.

IMAPClient 0.9.1

IMAPClient 0.9.1 is out! In this release:

  • OAUTH2 support: OAUTH v1 is still supported by Gmail but is now deprecated. Thanks to Zac Witte for the patch for version 2 support.
  • Stream support: It is now possible to have IMAPClient run an external command to establish a connection to an IMAP server. This is useful for exotic connection or authentication setups. Thanks to Dave Eckhardt for the original patch.
  • livetest updates to deal with the way Gmail now handles with new messages.

See the NEWS file and manual for more details.

As always, IMAPClient can be installed from PyPI (pip install imapclient) or downloaded from the IMAPClient site.


A relatively unknown part of the Python standard library that I find myself using fairly regularly at work these days is the groupby function in the itertools module. In a nutshell, groupby takes an iterator and breaks it up into sub-iterators based on changes in the "key" of the main iterator. This is of course done without reading the entire source iterator into memory.

The "key" is almost always based on some part of the items returned by the iterator. It is defined by a "key function", much like the sorted builtin function. groupby probably works best when the data is grouped by the key but this isn't strictly necessary. It depends on the use case.

I've successfully used groupby for splitting up the results of large database queries or the contents of large data files. The resulting code ends up being clean and small.

Here's an example:

from itertools import groupby
from operator import itemgetter

things = [('2009-09-02', 11),
          ('2009-09-02', 3),
          ('2009-09-03', 10),
          ('2009-09-03', 4),
          ('2009-09-03', 22),
          ('2009-09-06', 33)]

for key, items in groupby(things, itemgetter(0)):
    print key
    for subitem in items:
        print subitem
    print '-' * 20

Here the dummy data in the "things" list is grouped by the first item of each element (that is, the key is the first element). For each key, the key is printed followed by the items returned by each sub-iterator.

The output looks like:

('2009-09-02', 11)
('2009-09-02', 3)
('2009-09-03', 10)
('2009-09-03', 4)
('2009-09-03', 22)
('2009-09-06', 33)

The "things" list is a contrived example. In a real world situation this could be a database cursor object or a CSV reader object. Any iterable object can be used.

Here's a closer look at what groupby is doing using the Python interactive shell:

>>> iterator = groupby(things, itemgetter(0))
>>> iterator
<itertools.groupby object at 0x95d3acc>
('2009-09-02', <itertools._grouper object at 0x95e0d0c>)
('2009-09-03', <itertools._grouper object at 0x95e0aec>)

You can see how a key and sub-iterator are returned for each pass through the groupby iterator.

groupby is a handy tool to have under your belt. Think of it whenever you need to split up a dataset by some criteria.