Slithering Into Java's Heart

Jython has a pretty good relationship with Java. Python code seamlessly calls any Java code and Python classes can subclass Java classes and implement Java interfaces. However the relationship in the other direction is a little more awkward. To call Python code from Java, one either uses jythonc to compile the Python source to Java source, or exposes Python classes through Java code that exists solely to instantiate it.

Since jythonc is completely gone in Jython 2.5, and the wrapping is painful at best, one of the goals for Jython 2.5.1 is to make the Java to Python relationship as cordial as the Python to Java relationship. I administered the first congeniality injection to Jython's trunk earlier this weekend in the form of the ability to name Jython's proxy classes and to write them out to disk.

To explain why this makes things more pleasant from Java requires a little background on how Jython allows Python classes to subclass Java classes. Every time a Python class subclasses a Java class, Jython dynamically generates a proxy class to that Java class. The proxy class overrides all of the methods of Java classes and implements every method coming from Java interfaces. In its implementations of the Java methods, it dispatches from Java into an instance of the Python class it represents by looking up that method name on the instance and calling whatever it finds with the arguments the method received.

Up until this weekend, this was an entirely dynamic process. The proxy class would be generated at the time the Python class was created, and the Java class name of the proxy was created by a combination of the Python module and class and a counter that was incremented with every created proxy. This made things difficult for Java code in a couple ways. First, since the proxy classes were always generated dynamically, Java source couldn't import or call them directly as there weren't classes around at compile time for Java to work with. Second, since the names in part depended on the order in which the proxy classes were created, even if you were able to get your Java code to wait to resolve the classes till runtime, you couldn't depend on the generated class names for things like Class.forName and other Java reflection facilities.

This weekend's commits addresses both of these issues. The proxy classes can now be written to disk by setting sys.javaproxy_dir from Python or Py.getSystemState().javaproxy_dir from Java. Setting that field causes all proxy classes generated after that point to be written to the given directory when they're created. The non-deterministic proxy class name problem is addressed by setting __javaname__ in a Python class. If that's set, the proxy class is given the name from __javaname__.

This opens up some nice possibilities. If you generate the proxies for your code with a fixed name, you can add them to your Java classpath and import and use them normally. You can even pass the classes to something like Guice to let it inject Python classes into your Java code. clamp uses __javaname__ to automatically set the names of its generated proxies to the fully-qualified name of the Python class, so you don't need to specify those by hand. Combined with its method and constructor generation code, clamp will let you write Python classes that can be used from Java without scaffolding Java interfaces and classes to Python to extend.

There's still a fair amount of work to do before this code makes it into a real release: now that Jython proxies can persist beyond the lifetime of a JVM, they'll need to grow a version number so we can safely change them in later releases; clamp still has several items on its 0.1 TODO list; Jython needs tools to generate static proxies for its standard library and to allow users to generate static proxies for their own libraries, so Jython can be used in environments that don't allow classes to be created dynamically as jythonc once allowed. However all of that work is still much less than what went into 2.5.0, so it shouldn't be too long before 2.5.1 starts to bridge the Java to Python gap.

ibug for the Mobile Safari of Today

Joe Hewitt's bridge to get Firebug like features on the iPhone, ibug, is delightful. Unfortunately, recent additions to Mobile Safari block its use of iframes to transmit data from the phone to the server and vice-versa. I've gotten it working again using XMLHttpRequest from the desktop browser and jsonp-like script tags on the phone.

Using the updated version differs from Joe's original description only in download location:

1. Download ibug
2. Untar it
3. cd into its directory
4. Run python ibug.py launch
5. Add the printed snippet of script to the pages you want to log from

Enjoy classic logging based debugging on your phone-of-the-future today!

gleaner.py - tag splits and docstrings!

I've just updated gleaner.py, my wrangler for Wesabe's XML. The gentlemen hackers at Wesabe were good enough to add tag splits to the wealth of information already contained in their XML export which necessitated an update.

There were only a couple changes needed to take advantage of it. First, the TAG filter grew some logic to reject anything tagged with a split value of 0. Any existing code should take advantage of this automatically. Results will just be a little more accurate now if you've used a one-time tag with a value of 0 to cancel out a recurring tag for a single transaction. Second, Txaction got a "total" method that takes TAG filters. Call it with TAGs for the splits you want to include, and it'll return the sum of the values of each of those tags on the Txaction.

If that's sounding a little hazy, hopefully the second group of changes I made will help with that. gleaner actually has docstrings like a good Python module now! Fire up the Python interpreter and call help to your heart's content.

The Wesabeans also added account balances to the XML last week, so some code to pull that out will probably come next. I can't write my "real-time pile of gold doubloons purchasable by all worldly resources" dashboard widget until I have that information.

Wesabe Export Wrangler

I just uploaded gleaner.py, a Python module to parse Wesabe's XML export and make little text reports based on their content. It really only consists of two things at this point: read_txactions(), a function that reads "accounts.xml" in the current directory and returns a list of Transaction objects, and a set of "filter functions" that operate on the Transaction objects.

The filter functions are the ones that are in all caps. So far I have TAG, MERCHANT, ACCOUNT, AFTER, BEFORE, MONTH, AND, OR and NOT. To use a filter function, you call it with what you want to match and it returns a new function that returns true when passed Transactions that meet that criteria and false for those that don't make the grade. In the example below, TAG('charlie') creates a function that returns true for any transaction that has the 'charlie' tag. MERCHANT and ACCOUNT are fairly similar to TAG. They take a piece of text and return true if a transaction's merchant is the same as that text or its account is the same respectively. AFTER and BEFORE take a day formatted like '2007-05-01' and make a filter that matches transactions after or before that date. MONTH matches any transaction in a given month. It either takes a full month like '2007-05' or just a month number like '5'. If given the second form, it assumes the month is in the current year.

An example report is probably better than actually talking through this:

from gleaner import *

txs = read_txactions()

total = 0
for month in [MONTH(m) for m in '3', '4', '5']:

    spending = sum(t.amount for t in filter(AND(TAG('charlie'), month), txs))

    print '%s: %s' % (month.name, spending)
    total += spending

print 'Total: ', total

I tag everything I spend on myself with 'charlie'. So

AND(TAG('charlie'), month)

just matches what I spent on myself in a given month. When I run it(with fictional data), the output is:

$ python charlie_report.py
during Mar 2007: -457.23
during Apr 2007: -318.45
during May 2007: -412.58
Total:  -1188.26

Of course, that's pretty easily figured out from the things built-in to Wesabe's site. Here's a more complicated report:

from gleaner import *

txs = read_txactions()

def sum_txactions(*filters):
    return sum(t.amount for t in filter(AND(*filters), txs))

def month_summary(name, income, spendable, spent, diff):

    sep = '=' * (len(name) + 2)

    print '''%(name)s
%(sep)s
Income: %(income)s
Spendable: %(spendable)s
Spent: %(spent)s
Diff: %(diff)s
''' % locals()

net = 0

computer_spending = sum_txactions(TAG('computer'), TAG('chelsea'))

for month in [MONTH(m) for m in '3', '4', '5']:

    income = sum_txactions(TAG('birdnerd'), month)
    computer_deduct = 200

    #Don't deduct for March since computer was purchased after that
    if 'Mar' in month.name:
        computer_deduct = 0

    spendable = income/2 - computer_deduct
    spent = sum_txactions(TAG('chelsea'), month, NOT(TAG('computer')))

    diff = spendable + spent
    month_summary(month.name, income, spendable, spent, diff)

    net += diff
    computer_spending += computer_deduct

print 'Remaining Laptop: %s' % computer_spending

print 'Net: %s' % net

My lovely wife sells her art through the net under the moniker birdnerd. She wants to put half of her birdnerd profits back into the family's coffers and then spend the other half on things that strike her fancy. To throw a wrench into the works, she bought a most excellent MacBook a month ago and would like to pay for that $200 a month out of the "strike her fancy" half of the birdnerd profits. This report prints out the birdnerd income month by month, figures out how much of that she can spend on herself minus the monthly laptop payment, and keeps track of how much of the laptop remains to be paid off to print at the end.

Right now this doesn't handle using tag splits to determine amounts since the tag splits aren't actually in the XML export. The Wesabeans saw the use of that when I mentioned it to them, so they're going to add it at some point in the future. Next up for me is to add some chartsengrafs to the output mix.

Jython 2.2b2 released!

In news from the all-consuming destroyer of free time front, I've just released the second beta towards Jython 2.2. Download it and tell us if anything is broken!

The changes aren't actually that dramatic as the first beta was almost feature complete for 2.2 and the beta users didn't uncover many bugs. We did manage to fix 30 bugs, complete the new-style class integration and add a cool new customizable object adaptation system.

There are only a couple small changes to make a release candidate, so it should appear in a couple weeks. As long as all goes well there we can make the release a week or so after that and have the first stable release of Jython in over five years. It's a testament to the power of the idea of Python on the JVM that a project as dormant as Jython is awakening from its slumber.

Wesabe Export Downloader

Wesabe has an excellent interface to sanitize and categorize your financial information, so I've been using it for the past couple months to keep track of the dollars around here. They also allow you to download all of your data in a tidy XML format that contains all of the tagging and naming information you've added through said excellent interface. Since the reports they offer are still pretty basic, I've taken to chopping up the XML a little in Python to produce more complicated views of where the money has gone month by month, category by category.

The tedium of logging into the site, clicking through to the export page and actually downloading the XML every time I wanted to produce an updated report started to wear on me a bit though. grabber.py is a dumb little script that takes your Wesabe username and password as arguments, logs into wesabe, and downloads the XML as accounts.xml. They also have a csv format which grabber.py will grab if you pass it 'csv' as the final argument.

I've got another script that actually does more interesting things like parsing the XML with elementtree and making reports. I'll probably polish it up in the next few days and post it here as well.

Nailgun Your Way to a Quick Jython Startup

Charles Nutter just mentioned Nailgun on jruby-dev as a possible way to speed up jruby startup time. Turns out it works pretty well for jython too.

Regular Jython:

groves@muhngole ~/Desktop/nailgun-0.7.1
$ time java -jar d:/home/groves/dev/jython/trunk/dist/jython.jar -c 'print "hello"'
hello

real    0m2.390s
user    0m0.015s
sys     0m0.015s

Nailgunned Jython:

groves@muhngole ~/Desktop/nailgun-0.7.1
$ time ./ng.exe org.python.util.jython -c 'print "hello"'
hello

real    0m0.062s
user    0m0.015s
sys     0m0.015s

CPython:

groves@muhngole ~/dev/jython/trunk
$ time python -c 'print "hello"'
hello

real    0m0.093s
user    0m0.030s
sys     0m0.093s

So with Nailgun, Jython can print 'hello' more quickly than CPython itself. Unfortunately, much of the time of the regular Java invocation of Jython is spent initializing Jython so the first run of Jython from inside Nailgun is actually a little slower than a regular run. After that, all of the initialization is done inside of Nailgun's VM and things run pretty quickly as shown in the numbers above.

From within Nailgun's directory it's just a three step process to run Jython:

1. java -jar nailgun-0.7.1.jar
2. ./ng ng-cp jython.jar
3. ./ng org.python.util.jython -c 'print "hello"'
   

Since Nailgun is just passing its command line arguments through to Jython, you can just pass the script you want Jython to run to ng instead of the -c option I'm using as an example.