Codelog

Oh yeah!. With the power of DBus and libpurple APIs it is possible to give your boss the illusion of managing you. Just run the following script (under WTFPL). Tested with jabber accounts in a live office environment :P.

#!/usr/bin/env python
# By Sudharshan S, released under WTFPL
 
import dbus
import gobject
import time
 
class PointyHairedBoss:
 
    def __init__(self, boss_id, source, frequency=30):
        self.boss_id = boss_id
        self.source = source
        self.frequency = frequency
        bus = dbus.SessionBus()
        _pidgin_proxy = bus.get_object("im.pidgin.purple.PurpleService", \
                                                       "/im/pidgin/purple/PurpleObject")
        self.purple = dbus.Interface (_pidgin_proxy, "im.pidgin.purple.PurpleService")
        # FIXME:
        account_id = self.purple.PurpleAccountsGetAllActive()[0]
        self.boss_conversation = self.purple.PurpleConversationNew(1, account_id, self.boss_id)
        self.boss_im = self.purple.PurpleConvIm(self.boss_conversation)
        print self.boss_im
 
    def start_nonsense(self):
        question_list = open(self.source)
        for q in question_list:
            self.purple.PurpleConvImSend(self.boss_im, q)
            time.sleep(self.frequency)
 
if __name__ == "__main__":
   # Change the jabber id and the path to your questions, with an optional frequency
   o = PointyHairedBoss("foo@gmail.com", "questions")
   o.start_nonsense()

>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

PEP 20

This problem has haunted me many times when I want to work on two frameworks (Turbogears and Pylons) and there are some packages which interfere with each other. Having both installed in /usr/lib/python-2.5/site-packages is a nightmare. You won’t know what causes your nosetests to fail, you will have to comment out the other dirty package and try again. Also some packages may be upgraded which will cause some problem in some place in your application.

All this can be easily solved by using the virtualenv package in python. What it does is - create isolated python working environments. So that packages you install in one environment doesn’t interfere with the other. Also if you can’t install packages in the global site-packages, this will be really helpful.

You just have to easy_install virtualenv and it gets installed as a executable. If you can’t easy_install, there is a single python script (virtualenv.py) which can be used instead.

You can create your sandbox with the command $ virtualenv --no-site-packages env. The –no-site-packages option make sure that the global site-packages doesn’t inherit any packages from the global site-packages.

It will create a directory ‘env’ in your current directory. The bin folder inside this has its own python executable and a seperate site-packages in the lib directory. It also has easy_install installed in the bin.

Now you can install whatever package you want by typing $ env/bin/easy_install packagename

All your commands need to be prefixed with the env/bin/ path else it will use the wrong python version. But you can avoid typing the long commands by using the activate script bundled with the bin folder.

On *nix systems, you have to $ source bin/activate and on a windows machine > \path\to\env\bin\activate.bat. After you are finished with it, you just have to type deactivate and you get back to the old shell.

Most of the times, it would work perfectly, but sometimes the shell will cache the path of the commands. So, it is always advisable to check whether the path is set correctly by typing which python.

Now this solves my problem of mismatched and wrong packages and can start working in Pylons.

Here is a linkdump of resources related to python which I had in my bookmarks. These are some advanced topics which would really help you to better understand the internals and write better code.

• Python’s dot operator and the magic self object - explains about the way object orientedness works in python.
• Want to do operator overloading? Want to know the list of all special __underscore__ methods? It would really help you to have this as a quick reference.
• Metaclasses are classes whose instances are classes. This is a bit confusing to understand and would require you to read through the Part 1, Part 2 and Part 3 of the IBM’s developer works Metaclass programming in python series.

This post is due for a long time since my post on Singular form of a word program and here it is - Doctests in Python. From the python library reference,

The doctest module searches for pieces of text that look like interactive Python sessions, and then executes those sessions to verify that they work exactly as shown.

Doctests are an easy way to test small functions by specifying the output of the function in docstrings. In python you can (should) write docstrings for classes and methods which can be used to generate the documentation for the module. In those docstrings, you just specify calls to the function with the return string specified. The doctest module automatically asserts the output and verifies whether the test passes or fails.

Lets see an example of doctest and how to run them. Lets create a function which adds two numbers.

def add(x, y):
    """Adds two numbers/strings together

    >>> add(1,2)
    3
    >>> add('foo','bar')
    'foobar'
    """
    return x+y

Here we have written the docstrings and have four lines which look like the python shell. There we call the function with various arguments and also specify how the function will return.

So, how do we run the test? Just include these lines in the module and run the file directly. This is the usual way we run doctest (in the __main__ part of the file).

def _test():
    import doctest
    doctest.testmod()

if __name__ == "__main__":
    _test()

You just have to call the doctest.testmod() function and it will search for all such tests in the docstrings and outputs nothing if the tests pass. If you want to get more verbose output, run the file as python filename.py -v

There are many advantages of doctests (wikipedia has a lot more):

• Easy to understand what a function is going to do if called
• Tests and code together - easy to update the test if code changes
• Most of the times I execute my functions on the shell - so I just have to copy-paste the output from the shell to the code

Once you start using doctests, you would find yourself trying to use it in all possible places. For more elaborate testing, you should probably use pyunit.

Is there any other programming language which has tests written directly in the documentations? If you were to design such a testing module for you favourite language, how would you do it (especially if it doesn’t have a interactive shell)? Do explain your views on it below.

In turbogears you have three important config files. One is the dev.cfg file used during development, another is prod.cfg used during deploying your code. The last one is app.cfg file which has parameters which are common to both dev and prod.

When you store the parameters in the app.cfg file, it is good to group related parameters together under a section. A new section can be created by enclosing a word with [ and ], eg: [API]

All this is fine till we come to the point of reading back the values from the config file. We faced this same problem and whenever we tried to read it back, it used to give some error about no such variable present.

After looking through the code of the configparser.py for cherrypy, we saw that there was one more additional parameter called path. The path parameter takes in the section name and the variable inside the section is taken in by the parameter key.

So instead of doing a tg.config.get('url'), you would need to do a tg.config.get('url', path='API').

Though this was slightly different than the usual config parser module, it solved the problem of properly categorizing the config file.

On reading my last post singular form of a word, a friend of mine (Sathya) didn’t understand what I had done in the lambda functions. It was of the form lambda w: w[-3:] == ‘ies’ and w[:-3] + ‘y’. I had forgotten to explain about that in that post.

There is one small restriction in lambdas - you can’t use if-else constructs in them. If you want to use a condition check in your lambda, then it has to be of this form

lambda : <condition> and <if block> or <else block>

This is easy to understand when you compare it with conditional or ternary operators in C (?:) - condition?if block:else block.

Last post I talked about using list comprehensions and promised to post a simple method to find the singular form of a word. When I wanted to write such a function, I initially thought of using a chain of if-elif-else constructs. But I didn’t like it that way. So I used lambda functions and list comprehensions to do the same job.

def singularize(word):
    """Return the singular form of a word
 
    >>> singularize('rabbits')
    'rabbit'
    >>> singularize('potatoes')
    'potato'
    >>> singularize('leaves')
    'leaf'
    >>> singularize('knives')
    'knife'
    >>> singularize('spies')
    'spy'
    """
    sing_rules = [lambda w: w[-3:] == 'ies' and w[:-3] + 'y',
                  lambda w: w[-4:] == 'ives' and w[:-4] + 'ife',
                  lambda w: w[-3:] == 'ves' and w[:-3] + 'f',
                  lambda w: w[-2:] == 'es' and w[:-2],
                  lambda w: w[-1:] == 's' and w[:-1],
                  lambda w: w,
                  ]
    word = word.strip()
    singleword = [f(word) for f in sing_rules if f(word) is not False][0]
    return singleword
 
def _test():
    import doctest
    doctest.testmod()
 
if __name__ == '__main__':
    _test()

This method is simple if you know the rules of plurals in the english language. I have converted each rule into a lambda function which returns the corresponding singular word depending on the word ending. The order of the rules are important, for eg., i should first check for -es before checking for -s. I haven’t taken care of the special plurals like men-man, people-person, children-child.

I then use list comprehensions to apply each function on the input word and pick the 0th element. Now you get the singular for of the word.

If you notice in the function, I have used something called doctests which is a very easy way to test your functions. You can download the source code and run it to run the doctest. Wow, now I got another topic to write about in the next post - doctests in python.

In the last post about Map-Filter-Reduce before finishing the post, I told about slight changes to those functions and how you can do the same job using list comprehensions. I would be talking about it here today.

Three years back Guido van Rossum, the BFDL made a blog post about the fate of reduce() in Python 3000. He stated that map and filter confusing to many people and that list comprehensions were simple to read and understand. Guido wanted reduce() to be totally removed from the language as there weren’t any good uses of it except + and *.

List Comprehensions
So, what is this list comprehensions that Guido was talking about. It is just a concise way to create lists from another list (just like map() and filter()). It is easy on the eyes and clearer to understand.

It consists of an expression followed by a for clause and more for/if clauses. The result of this is a list gotten by evaluating the expression in the context of the for and if clauses.

>>> words = ['FOO', 'BAR', 'BAZ']
>>> [word.lower() for word in words]
['foo', 'bar', 'baz']

>>> nums = [1,2,3,4]
>>> [(x, x*x) for x in nums]
[(1, 1), (2, 4), (3, 9), (4, 16)]

This looks just like the map function. Now just add an if clause after the for, and you get the filter equivalent.

>>> nums = [1,2,3,4,5,6]
>>> [i for i in nums if i%2 == 0]
[2, 4, 6]

List comprehensions are easier to understand by just reading it as words - create a list of is for every i in nums if i%2 is equal to 0. Also list comprehensions are faster than map and filter as it doesn’t have to make function calls.

So, even though Guido wanted all the four functions to be removed from the standard library, he has announced that lambda, map and filter would stay - with the latter two having small changes - returning iterators instead of a list. Unfortunately reduce() is moved over to functools module.

So my next post is going to be a small example of using list comprehensions to find the singular form of a word.