As promised last week, I've been working on making Psil work in a Python 3 environment. But first, a small but entirely relevant digression which includes a video of me giving a presentation:

A few weeks ago, I attended Kiwi PyCon here in Christchurch. I had thought about maybe possibly presenting something when I signed up, but I didn't actually prepare anything ahead of time. Immediately after lunch on the first day, there was a "lightning talk" session where people get 5 minutes to present something, no questions from the audience. During lunch, I decided that my opportunity was right then, and that if I didn't try to present something then I'd probably be annoyed at myself for not having tried.

Anyway, I hurriedly prepared a presentation (PDF, 8 slides) in the 30 minutes before the presentation, and wondered whether I would even be able to find 5 minutes worth of stuff to say. It turns out I did, because although it felt like I had zipped through my presentation in like 3 minutes, I actually took slightly over 5 minutes. There is a video recording of my (terrible) presentation that you can watch online (skip forward to about 52:00 using the slider).

Right, now that you're back, I'll continue with what I was going to write about.

The Psil compiler (as opposed to the interpreter) compiles Psil code into Python code, which is then executed directly by the Python runtime system. For example, here is a simple function and its use:

    Psil                            Python
    ----                            ------

    (define (sq x) (* x x))         def sq(x):
                                        return x * x

    (print (sq (+ 2 3)))            print sq(2 + 3)

Previously (before the current Python 3 migration), I did this by generating literal Python source code as shown in the right hand column above. But instead of generating the source code directly, I generated a Python AST (Abstract Syntax Tree) representation, then implemented a "de-parser" that creates equivalent Python source code from an AST. The original idea was to have Python compile and execte the AST directly (rather than going through the source code step), but I think the version of Python 2.x that I was using didn't support passing an AST to the compile() function.

Since Psil now requires Python 3, the compile() function supports direct compilation of an AST that represents Python code. So now Psil can generate the AST, pass it to compile() without going through source code, and execute the resulting code object directly. One of the difficulties with this was that the internal Python AST changed completely from version 2 to 3, so I had to rework most of the compiler code. Fortunately, it turned out to be reasonably straightforward. I also maintained the de-parser because debugging actual source code is a lot easier than debugging with only an AST.

Right now I've got this all working for simple examples. I still need to re-implement the "lambda lift" which promotes a Psil lambda expression (which may contain an arbitrary number of forms) to a Python function (because Python's lambdas can only contain a single expression).

The next interesting thing I can do with an AST is annotate the nodes with source line and column numbers. This means that if a Python exception happens at runtime, the traceback will actually point to the location in the Psil source code that caused it! That's a pretty cool feature and I'm looking forward to playing with that.