virtualenv, python-rope and pylint

python-ropemac is really useful for developing python in emacs, and pylint is also very handy as a analyzer. However, they both don’t work very well with virtualenv, especially because I always run emacs in server mode, and the server instance is usually not under virtualenv.

Here is how to make things work:

Edit .ropeproject/config.py:

# You can extend python path for looking up modules
    prefs.add('python_path',
              '/your-virtualenv-dir/lib/python2.7/site-packages/')

For pylint, generate (by pylint --generate-rcfile) or copy your default pylintrc to project root dir. Edit it:

# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
init-hook='this_file="/your-virtialenv-dir/bin/activate_this.py";execfile(this_file, dict(__file__=this_file))'

After this pylint will work even in emacs.

Curious case of closure in Go and Python

At http://tour.golang.org/#39, I found the following sample code:

package main

import "fmt"

func adder() func(int) int {
	sum := 0
	return func(x int) int {
		sum += x
		return sum
	}
}

func main() {
	pos, neg := adder(), adder()
	for i := 0; i < 10; i++ {
		fmt.Println(
			pos(i),
			neg(-2*i),
		)
	}
}

Its document reads:

… functions are full closures. The adder function returns a closure. Each closure is bound to its own sum variable.

Take that into consideration, it might be easier to understand the execution result:

0 0
1 -2
3 -6
6 -12
10 -20
15 -30
21 -42
28 -56
36 -72
45 -90

To me, variable sum is similiar to ‘instance variable’, in Object-oriented’s terminology. However, in Python, things can be quite different.

def adder():
    sum = 0

    def f(x):
        sum += x
        return sum
    return f


def main():
    pos, neg = adder(), adder()
    for i in xrange(0, 10):
        print pos(i), neg(-2 * i)

The code looks roughly the same, but it will raise the following exception:

Traceback (most recent call last):
  File "closure.py", line 17, in 
    main()
  File "closure.py", line 13, in main
    print pos(i), neg(-2 * i)
  File "closure.py", line 5, in f
    sum += x
UnboundLocalError: local variable 'sum' referenced before assignment

This is because if sum is to be modified, Python must decide which variable to change. sum += x is the same as sum = sum + x, and sum = suggests it’s a local variable, since all variable is by default local in Python. Given that, expression sum + x can not be evaluated because sum, as a local variable, is still undefined here.

If the sum += x line is removed, and sum + x is returned directly, the result will be:

0 0
1 -2
2 -4
3 -6
4 -8
5 -10
6 -12
7 -14
8 -16
9 -18

It runs okay, but the result is wrong. Where does function f get the value of sum? If Python cannot find a variable in locals(), it will try to find it from the scope above it, i.e. function adder, and sum is indeed defined in it. The real Python equivelent of the Go program above will be:

class adder:
    def __init__(self):
        self.sum = 0
    def __call__(self, x):
        self.sum += x
        return self.sum


def main():
    pos, neg = adder(), adder()
    for i in xrange(0, 10):
        print pos(i), neg(-2 * i)


if __name__ == '__main__':
    main()

Functions are already first class objects in Python. Here we create a class that its instance behaves like a function, so it is a function because of duck typing.

Multi-thread testing in Pyramid

If you want to do multi-thread testing in Pyramid, it probably won’t work the first time because request and registry are thread local, and things like get_renderer will call get_current_registry. When it happens in a thread, it won’t get the same value as it would have in the main thread.

So, here is a hack to address this:

import pyramid.threadlocal
from threading import Thread, Lock

candidates = [
    (self._test1, ()),
    (self._test2, ()),
    (self._test3, ()),
    ]

def random_func(pyramid_thread_locals):
    pyramid.threadlocal.manager.push(pyramid_thread_locals)
    time.sleep(random.random())  # 0 ~ 1 sec
    func, args = random.choice(candidates)
    func(*args)

pyramid_thread_locals = pyramid.threadlocal.manager.get()
threads = [Thread(target=random_func, args=(pyramid_thread_locals, ),)
           for i in range(100)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()

There is no guarantee that pyramid.threadlocal.manager will always be there. Even if it’s there, there’s no guarantee it can be used this way. So, this should only be considered as a temporary workaround.

WordPress to blogger

I wanted to move this blog to blogger for more flexible css and java script manipulation, so I wrote this py-w2b project to do that. So far I’m able to transfer blog posts, but not comments, since every comment posted by an authenticated session will become my own comments instead of the original authors. It’s trivial to use so I’ll skip the introduction here.

bug in python-opengl + mesa

Since like forever, I have never succeed in executing python-opengl demos on debian lenny.  It always segfault during glutInitDisplayMode.  After some debugging, here is why.

One should be able to call glGetError at any time.  If there’s nothing wrong, it should just return something like GLNOERROR.  Any raw (native) function call in python-opengl, no matter it belongs to GL, GLU or GLUT, will always call glGetError right after each call to check for error.  I think this is wrong since functions like glutInitDisplayMode have NOTHING to do with glGetError.  With mesa 7.0.3-5 in my system, the call to glGetError without glInit will cause segfault.  I check the same thing on Ubuntu, which has an older version of mesa and python-opengl, and it does not happen.

python-opengl enabled error checking by default with OpenGL.ERROR_CHECKING, which is set to True in OpenGL/__init__.py.  This code snippet can disable it:

import OpenGL
OpenGL.ERROR_CHECKING = False
# import other stuffs such as OpenGL.GL, etc.

or you can just modify the __init__.py to disable it by default.

Given the current status of python-opengl and low level x protocol support in python, I think the best language to do 3D in FOSS world will still be pure simple C.

generate object methods at runtime

I have been working on a dialer button class since yesterday. It makes sense to use the command design pattern here, and I want to separate the commands and the buttons so I can change the functionality of every button at runtime.

So we are talking about something like this:

class DialerButtons:
    def __init__(self):
        self.command_table = [
            self.numpad_1, self.numpad_2, self.numpad_3,
            self.numpad_4, self.numpad_5, self.numpad_6,
            self.numpad_7, self.numpad_8, self.numpad_9,
            self.cancel, self.numpad_0, self.dial]

    def numpad_0(self):
        self.text.append('0')

    def execute(self, n):
        self.command_table[n]()

and you can use this class like this:

dialer = DialerButtons()
dialer.execute(0)
dialer.execute(8)

Now obviously define numpad_0 to numpad_9 is a boring task. What happens if you need to define numpad 0 to 99? So, I came out with this code piece:

    @classmethod
    def _numpad_commands_factory(cls):
        for n in xrange(0, 10):
            setattr(cls, 'numpad_%d' % n, lambda self: self.text.append(str(n)))

DialerButtons._numpad_commands_factory()

This way you initialize DialerButtons AFTER you start the program and make methods numpad_0 to 9 on the fly. At least that’s what I was trying to do. However, it didn’t come out as I expected. Every numpad method will just add ‘9’ to self.text, instead of the respective ‘0’ to ‘9’. Why?

The reason is that the context of numpad_0, for example, is actually “f(self): self.text.append(str(n)))” instead of “f(self): self.text.append(‘0’)”. so, what it does here is that it refers to the variable n inside _numpad_commands_factory, and the value of n is 9 after you executed it.

The correct code piece is:

    @classmethod
    def _numpad_commands_factory(cls):
        def f(chr):
            return lambda self: self.text.append(chr)
        for n in xrange(0, 10):
            setattr(cls, 'numpad_%d' % n, f(str(n)))

This way we can evaluate the value of str(n) first, then generate the appropriate function and assign it to numpad_n.

python-efl on ubuntu gutsy

hey if you have a extremely slow laptop (Pentium III) like me yet you still want to use python-efl on it, here is how.

first, compiling the whole e17 from scratch is not an option. it will take forever. there’s a ubuntu package repository for e17:

$ cat /etc/apt/sources.list.d/e17.list
deb http://e17.dunnewind.net/ubuntu gutsy e17
deb-src http://e17.dunnewind.net/ubuntu gutsy e17
$ sudo apt-get update
$ sudo apt-get install efl-dev libecore-imf-evas-dev

this can save a lot of time.

the other problem is that the latest cython release (0.9.8) does NOT work with python-efl. you have to install 0.9.6.14 instead. don’t forget to install python-pyrex as well.

now you’re all set, get python-efl.

cvs -d :pserver:anonymous@anoncvs.enlightenment.org:/var/cvs/e login
cvs -z3 -d :pserver:anonymous@anoncvs.enlightenment.org:/var/cvs/e co e17

it’s under e17/proto/python-efl. another funny gotcha is that the binary packages I installed were built on 20080309. so the latest python-efl will not build successfully.

cvs -z3 update -dP -D 20080309

fixs this.

the default build-all.sh under python-efl directory builds evas ecore edje emotion e_dbus epsilon. that’s too many. for me I just need evas ecore edje e_dbus.