Category Archives: python

A new project in my spare time…helping my nephew create games with #Python and #Pygame

My nephew expressed interest last year in creating games and programs on his computer. I bought him a super cool book, Making Games with Python and Pygame. He’s been busy with other things, but today I got a call from him asking how to set up Python.

I was driving so I couldn’t get very far. But the pure fact that his parents have a Mac made me smile. We could speak on the same wavelength. I promised to send a follow up email with some HOWTO steps.

The first thing I did was reach for Asciidoctor and start writing up a handful of steps to get started. I had to really dial up the thinking, because I wasn’t writing for a pro developer. I was writing for a kid that doesn’t know any of this.

I also decided to host it all at https://github.com/gregturn/so-you-want-to-learn-python as the place to manage everything. After getting that fixed up, I realized I could pipe the generated HTML into a gh-pages branch, and host the polished stuff at So You Want To Learn Python.

Should be enough to get the ball rolling. Waiting eagerly for a response!

Scripting does the trick!

Last night I finally googled around and found http://python-wordpress-xmlrpc.readthedocs.org/en/latest/. This library uses WordPress’s XML-RPC API hook to let you connect, fetch content, and make edits. It was fabulous!

Having migrated from Blogger, I still had a huge number of links that were pointed at Picasaweb, where I have images stored. I had wanted to somehow update all my old blog posts and point them at the images I pulled down from Picasaweb and uploaded here.

This library was perfect. Their intro docs show you the basics of how to get started. It wasn’t long before I figured out how to scan every post, look for particular patterns I wanted to clean up, and then replace them properly. For example, my blog used to be greg-turnquist.blogspot.com. Several of my posts were still linked as such. Blogger kindly would forward uses hitting those links, and I left sufficient settings behind so they would keep getting redirected here. But to finally erase that old domain from all cross posted blog entries was perfect.

The other thing that I need to wrap up is how I still have some blog entries that hyperlink to the old label system of blogger (/search/labels/foobar). In WordPress, I use /category/foobar, so I need to scan for them all and make updates. I also need to replace things like real%20estate with real-estate.

The library even had the ability to pull down information on every image uploaded to this site. I naturally checked the name of each image here and on Blogger, and they all have the same root filename. I just needed to update the link and URI path.

To top things off, as usual, I dug up Python’s formidable re library. I always have to pull up the reference docs. Something about regular expressions always requires me to reboot my brain on using it. I am quite skilled and have been writing regular expressions since I learned PERL probably twenty years ago. But to really sink your teeth into a problem, write the correct pattern, and extract the groups through their API…well, let’s just say it doesn’t stick to me quite like certain other things. No doubt, this site helped me craft the pattern I needed.

After all was said and done, I finally pulled the trigger and issued a handful of edits. In no time, I had updated both this blog and my financial one, ensuring that I was now using local images.

Introducing…toolbox!

No, this isn’t a new open source project. Instead, its my toolbox of useful scripts.

I spent half a day figuring out how to limit network traffic on my system to emulate a customer’s slow network link. After manually typing ipfw commands to turn on bandwidth limits and then turn them off, I decided to capture the sequence in a command script.

Having just updated this blog site to include a link to my github handle, the idea became crystal clear: put the script under git version control and share it with others on github.

The pros:

  • it provides online backup of my toolbox of scripts
  • it is easy to version the updates and distribute them to my other boxes
  • over time, these scripts can probably be enhanced to run on other platforms
  • you can partake of the benefits!
  • hopefully, in the future, I can partake of yours.
The cons: none that I can think of
I hope you find it useful.

See how Spring Python works with Jython

I recently completed a patch that replaces the amara library with python’s default elementTree library. As much as I like amara’s API, its C-extension basis was unacceptable. I just merged the patch into the trunk, and verified it works with Jython 2.5.1.FINAL. When Spring Python’s 1.1.0.M2 release comes out, we will be Jython-compatible.

With this change, there were only a couple of tweaks to the rest of Spring Python.

What does this mean? How about looking at some code to see how we can start with a Java app, and migrate things over to Python. To kick this off, let’s start with an uber-contrived example Java app: a wiki engine. The wiki engine calls into a data access layer in order to look up some statistics on the site.

public interface DataAccess {

public void statistics();

}
public class MySqlDataAccess implements DataAccess {

public void statistics() {
System.out.println("MySQL: You are calling for some statistics.");
}

}
public class WikiService {

private DataAccess dataAccess;

public DataAccess getDataAccess() {
return this.dataAccess;
}

public void setDataAccess(DataAccess dataAccess) {
this.dataAccess = dataAccess;
}

public void calculateWikiStats() {
System.out.println("You are passing through a pure Java WikiService...");
dataAccess.statistics();
}

}

Let’s assume this is a Spring app. That means we need an XML application context. Let’s create one and call it javaBeans.xml.



xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">








We can start up our pure java app along with its context by writing jython script javabeans.py. It uses Spring Python’s SpringJavaConfig parser.

if __name__ == "__main__":
from springpython.context import ApplicationContext
from springpython.config import SpringJavaConfig
ctx = ApplicationContext(SpringJavaConfig("javaBeans.xml"))
service = ctx.get_object("wikiService")
service.calculateWikiStats()

We just need to compile the java code, and then run our script using jython.

gregturn@startrek:~$ javac *.java
gregturn@startrek:~$ jython javabeans.py
You are passing through a pure Java WikiService...
MySQL: You are calling for some statistics.
gregturn@startrek:~$

Okay, we’re off to a good start. We have tapped a running java system. For our next step, let’s write a pure python version of the data access code and put it into Py.py.

import DataAccess

class MySqlDataAccess(DataAccess):
def statistics(self):
print "PyMySQL: You are calling for some statistics."

class StubDataAccess(DataAccess):
def statistics(self):
print "PyStub: You are calling for some statistics."

Now, to really crank things up a bit, let’s migrate from the Spring Java XML configuration over to Spring Python’s pure Python container.

from springpython.config import PythonConfig, Object
from Py import MySqlDataAccess
import WikiService

class WikiProductionAppConfig(PythonConfig):
def __init__(self):
PythonConfig.__init__(self)

@Object
def data_access(self):
return MySqlDataAccess()

@Object
def wiki_service(self):
results = WikiService()
results.dataAccess = self.data_access()
return results

if __name__ == "__main__":
from springpython.context import ApplicationContext
ctx = ApplicationContext(WikiProductionAppConfig())
service = ctx.get_object("wiki_service")
service.calculateWikiStats()

The top half of our mixed.py script contains an IoC container, except there is no XML whatsoever. Instead, its raw python. You can see how we are also using our python version of MySqlDataAccess.

gregturn@startrek:~$ jython mixed.py
You are passing through a pure Java WikiService...
PyMySQL: You are calling for some statistics.
gregturn@startrek:~$

This shows that we are still using the pure java WikiService class, and it calls into our pure python MySqlDataAccess.

Another useful feature that Spring Python provides is easy access to Pyro, the Python Remote Objects library. This library is for making RPC calls from Python-to-Python. Spring Python makes it easy to link clients and services together with Pyro.

First, let’s take our wiki service, and export it with a server script.


from springpython.config import PythonConfig, Object
from springpython.remoting.pyro import *

from Py import MySqlDataAccess
import WikiService

class WikiProductionAppConfig(PythonConfig):
def __init__(self):
PythonConfig.__init__(self)

@Object
def data_access(self):
return MySqlDataAccess()

@Object
def wiki_service(self):
results = WikiService()
results.dataAccess = self.data_access()
return results

@Object
def exported_wiki_service(self):
return PyroServiceExporter(service=self.wiki_service(), service_name="wiki", service_port=9000)

if __name__ == "__main__":
from springpython.context import ApplicationContext
print "Starting up exported WikiService..."
ctx = ApplicationContext(WikiProductionAppConfig())

We made some slight tweaks to the IoC container by adding exported_wiki_service. It targets wiki_service, and links it with a name and port to expose it as a Pyro service. Spring Python by default will instantiate every object, so in this case, it creates the PyroProxyExporter. Because this isn’t the caller, we don’t need to fetch anything from the context.

Next, let’s code a client to call our service.


from springpython.config import PythonConfig, Object
from springpython.remoting.pyro import *

class WikiProductionAppConfig(PythonConfig):
def __init__(self):
PythonConfig.__init__(self)

@Object
def wiki_service(self):
results = PyroProxyFactory()
results.service_url="PYROLOC://localhost:9000/wiki"
return results

if __name__ == "__main__":
from springpython.context import ApplicationContext
ctx = ApplicationContext(WikiProductionAppConfig())
service = ctx.get_object("wiki_service")
service.calculateWikiStats()

Now, let’s start up the server.


gregturn@startrek:~$ jython mixed_with_pyro_server.py &
[1] 4953
Starting up exported WikiService...

The script is backgrounded, because it is running a Pyro daemon advertising our service. Now, let’s launch the client script.


gregturn@startrek:~$ jython mixed_with_pyro_client.py
You are passing through a pure Java WikiService...
PyMySQL: You are calling for some statistics.
gregturn@startrek:~$

As you can see, our client makes remote to the exported WikiService, which happens to be a pure java service. The java service then calls our python version of MySqlDataAccess to do its statistics call. This goes to show how easy Spring Python makes it mix and match Java and Python.

Is Spring Python a framework or a library?

I was tracking some comments, and recently noticed a thread of discussion (in English) on a python mailing list. This is a follow-up to someone else posting about his discovery of Spring Python through the news announcement from SpringSource. (I appreciated that the person also caught on to my article defending the usage of AOP in python.)

They concluded their response by indicating:

Aunque para convencerme de usarlo todavía le falta compararse con otros frameworks de python.

(Run this through Google Translate if you need to.) Apparently, this person, while slightly impressed that I wasn’t coding in horrible java paradigms, didn’t see enough to use this instead of other python frameworks.  I knew that he was comparing it to other python web frameworks.

History Lesson

I started thinking about the basis of all this, and realized a little bit of a history lesson is in order. Anybody who has ever heard of Spring, knows it by its full name, The Spring Framework. This should help illuminate why Spring is referred to as a framework, and why Spring Python is often thought of as a framework itself.

Starting around 1999, the first EJB spec came out. This was the technology everyone in the java industry started using, and people were throwing together any and all java web apps using EJBs. It caused a resurgance in the value of java, and arguably catapulted java into the enterprise space. EJBs promised much. The application servers that would run your EJB-based app had many features, like transactions, security, failover, messaging, and web services to name a few. All you had to do was extend a few interfaces, write a few configuration files, and do a little configuration to the app server.

Well, that was how it was supposed to be, right? The truth was, EJB apps required lots of XML file configuration. Each container had deviations, so your app tended to be container-specific. People usually had too many services turned on, and unit testing was next to impossible. You couldn’t test your app outside the container, so turn around time to see your changes was a nightmare. That is what generated the flurry of frameworks, like Struts, to help make EJB app development simpler. By extending these framework’s classes, you had a little bit less to extend, less to write, and less to configure. But you still had to do it anyway. And that turn around time to test your app? No better. And you had to make your WHOLE app do this. There was no such thing as, “Part of my app is using Struts, another part is using Wicket.”

Then in 2002, Rod Johnson, a java software consultant, wrote a book Expert One-on-One J2EE Development without EJB. When Rod’s book hit the shelves, the message he was sending was completely unorthodox. He showed tactics of code development that quickly made these container-based services obsolete. You didn’t have to extend anyone’s classes. And…you didn’t have to adopt any framework lock-stock-and-barrel. Instead, your app could be 99% based on POJOs. The code he was writing for examples in the book was very pragmatic, and was just like the other frameworks, only it was a framework based on POJOs. Another consultant approached Rod about releasing that software as an open source project, and thus was born The Spring Framework.

This code, while acting similar in output as other frameworks, had a peculiar difference: it did not require actually extending anybody’s classes. In fact, you could write your code 99% framework-free, and Rod & Co. encouraged this.

“So…is Spring a framework or a library?”

I argue: both!

Considering that a library is a set of functions and classes you download and install in your software development toolbox, Spring fits this perfectly. It was inspired by the need to overcome the lethargic development process of the EJB spec. Not only did they improve over EJB’s way of doing things, they also enhanced JDBC, JDNI, JMX, and other things.

  • Ben Alex decided to develop a Security library based on the pragmatic concepts behind Spring, which has since become the most widely adopted security solution for java.
  • Mark Pollack was also watching Spring cut through the java industry and coded a similar solution for C#/.NET and called it Spring .NET.
  • …and the list goes on of other widely adopted libraries/tools developed for the java industry based on Spring’s approach to solving problems

So, yes, Spring is a library. But it is ALSO a framework, in that its original intention was to faciliate building enterprise-grade web applications inside EJB containers. They simply showed that the servlet container was enough. In fact, many shops started fresh adopted the concept of using Apache Tomcat or Jetty as the servlet container, which has been challening the commerical app server market. Combined with the fact that your app could run inside and outside an EJB container, and also be easily JUnit-tested without the slow make-deploy cycle made it a hit.

And…its NOT a framework!

“But, you just SAID Spring is a framework!”

Yeah, I know I did, but many people interpret framework as code meant to be extended. Spring doesn’t require this. It may accomplish the same end result as these other EJB frameworks, but through simpler means. I know this first hand, because I have developed thick Java client apps using Swing. No EJB container to be found! And yet, I have found Spring provided much leverage and functionality. I didn’t have to extend anything. Okay…I had a unique situation involving security where I needed to extend a Spring Security interface, but 99% of my code doesn’t do that. And that’s the point.

“Enough Java already! I thought this was a Python blog!”

Okay, if you’re still reading, it means you are really a cross technology buff. Or maybe you just read blogs instead of sword fighting while waiting while waiting for your code to compile. Otherwise, you freaked that I spent so much time talking about Java, XML, EJBS, and fled the scene. Well, I’m a polyglot programmer. I use all kinds of stuff: C/C++, Java, Python, Tcl/Tk/Expect, PERL, bash/csh, jython, scala, groovy, clojure, anything! Actually, I had never heard the term until I saw Russ’s Skills Matter presentation where he presented himself as an “self professed, unapologetic, polyglot programmer”. But I immediately connected with it.

Anyway, the point I’m trying to make is that the concepts Rod presented are of incredible value. Its not the fact that he was working in Java, but instead the idea that pragmatism is valuable. Instead of always resting comfortably against specs, we need to always be on the lookout for places and paradigms where we are wasting countless cycles in coding and re-coding the same thing. I contend that repetition is one of the top ranking source of bugs, if not the highest.

I believe Spring Python is a library

My first entry into the world of Spring was through their fabulous JdbcTemplate. Writing SQL queries without leaking cursors and connections was of high important to me. When I decided to code  a carputer application, I picked Python because reading Dive Into Python had sold me on the beauty of it. However, I quickly realized I was missing my lean-and-mean JdbcTemplate. I started coding a DatabaseTemplate to help me, and realized I was starting to do something just like Rod. Well, I decided then and there to share my library with anyone who was interested. Hope it helps you like it has helped me.

I think, and it’s not a mockery, that the Java world is rather a framework type of environment which leads usually to have as many frameworks as you have developers for the very reason I described above. Python has also its share of frameworks but the language is flexible enough to encourage libraries. I *strongly* believe that Python Spring[sic] has both functionalities that could be attractive to many developers while having the chance of doing so as a library rather than a framework. –Sylvain while commenting about Spring Python w/CherryPy 3

And that’s why I call it Spring Python. It takes the cool concepts of Spring, and write them in a slick-and-quick, pythonic library. And yes, it is a library.

Why use Spring Python and not just plain python?

I have bumped into articles and forum postings, blog entries, other blog entries, and yet more blog entries. In fact, while writing this blog entry, someone posed the question “what is the aim of Spring Python?” (Click and see my answer.) There are a diverse range of view points, and many in disagreement about the viability of DI in python.

Many of these opinions raise a good question. Why do you need some complicated thing like Spring Python when you could be using pure python to do your job? I was very impressed with Jamis’ article LEGOs, Play-Doh, and Programming. The core question is, why use complicated Java idioms that can be easily and dynamically expressed in pure ruby? I value this article because it encourages me to recheck my assumptions and ideas. It doesn’t matter how good your solution is. If you built it on a platform of invalid, unwise, or ridiculous assumptions, you probably aren’t going anywhere fast.

So, what ARE my assumptions?

  • Being able to apply extra services to my code without touching the code is good.
  • Being able to inject test dummies into my code without editing my code is good.
  • Simplifying and reducing code volume is good.
  • Using fancy tricks that are hard to read the next day is bad.
  • Using Problem A’s solution for Problem B without analysis is dumb.

I want to compare this against what some people may think my assumptions are.

  • XML is the way to configure everything.
  • Java’s solutions are perfect for Python.
  • Dependency Injection should be used for every problem.

Let’s look at this in closer detail. Hopefully, I can show some insight into places where I think Spring Python can help, and other places it would be overkill to use it. Nothing works everywhere, right?

Being able to apply extra services to my code without touching the code is good – True

The first couple of things I can think of are transactions and security. These are functions, that when you implement them by hand, are a pain in the neck! You keep writing the same boilerplate plumbing code wrapping logical blocks, and you have to do it right every time, or you may find yourself in a big hole.

Sure, any part of your code not working correctly is an issue. But this isn’t YOUR code we are talking about. This is simply a layer of protection you are trying to wrap around code to make it safe. There isn’t a whole lot of logic in transactions or security, just simple, copy-and-pasted stuff you need. And we know what risk we are at when copy-and-paste starts happening.

That is the definite code smell that AOP is needed. Spring Python provides an AOP solution. You can write any type of advice you need. But considering how prevalant transactions and security are, we went ahead and coded a flexible, pythonic way to do it.

But don’t forget. The premise is that this is applied without modifying your working code or your calling code. If only we could have something that sat between the business code and the “main” function that calls the code, which managed instantiation of dependencies, and which we could modify with no impact…..you mean, like an IoC container?

I know what you’re thinking. “Oh no! Not that XML stuff! I want to stick to python.” Who said you have to use XML? You can code your transactions simply using python decorators and a pure python container. XML is only provided to help people coming from Java-land migrate to Python. Some people like XML. Some people don’t. Its a choice, not a requirement.

I know there is something else to consider. If I’m using TDD and I am spotting the places where I need to make changes, why can’t I just make the changes right then and there? If there is only one time and one place where you are injecting your AOP advice, you are probably right. It is when you need to apply one piece of advice in multiple places where you start doing the copy-and-paste routine. And when you need to add more advice, and repeat this whole process over again where things get costly. Instead, one refactoring to expose that service as an injection point, and you can easily adjust things in the future.

Being able to inject test dummies into my code without editing my code is good – True

Testability of dynamic code is key. We preach that dynamic typing works, provided you adequately test your code. If you remember anything from your computer engineering courses in college, the more components you have, the greater the number of permutations of state. Being able to isolate functionality is the way to increase confidence in test cases, and that means we need to be able to replace production dependencies with test doubles.

It is easy to get caught up in the “I want to inject everything” hype. Don’t! Instead, only inject key things. TDD will expose what test doubles you want to inject, and thus what parts need to be wired by IoC. If your application is small enough, you may not need any at all. And that’s alright! Perhaps you start off, and all you need to inject is a database test double. Assuming you have defined MyApp and DataAccess, your container definition can be as simple as:

class MyAppContext(PythonConfig):
def __init__(self):
super(MyAppContext, self).__init__()

@Object
def my_app(self):
return MyApp(self.data_access())

@Object
def data_access(self):
return MyDataAccess()

For your test case, you just need an alternative:

class MyTestableAppContext(MyAppContext):
def __init__(self):
super(MyTestableAppContext, self).__init__()

@Object
def data_access(self):
return DataAccessStub()

How hard was that? You just extend your existing container, overriding a key injection point, and poof! You’re there with a nicely isolated, testable version of your code! I don’t know about you, but that is arguably a DSL, written in pure python.

Simplifying and reducing code volume is good – True

I mentioned transactions earlier. Let’s look at an ugly example of some code with transactions:

class Bank:
def __init__(self):
self.factory = factory.MySQLConnectionFactory("springpython", "springpython", "localhost", "springpython")
self.dt = DatabaseTemplate(self.factory)

def balance(self, account_num):
results = self.dt.query_for_list("select BALANCE from ACCOUNT where ACCOUNT_NUM = %s", (account_num,))
if len(results) != 1:
raise InvalidBankAccount("There were %s accounts that matched %s." % (len(results), account_num))
return results[0][0]

def withdraw(self, amount, source_account_num):
if self.balance(source_account_num) < amount:
raise InsufficientFunds("Account %s did not have enough funds to transfer %s" % (source_account_num, amount))
self.dt.execute("update ACCOUNT set BALANCE = BALANCE - %s where ACCOUNT_NUM = %s", (amount, source_account_num))

def deposit(self, amount, target_account_num):
self.balance(target_account_num) # Implicitly testing for valid account number
self.dt.execute("update ACCOUNT set BALANCE = BALANCE + %s where ACCOUNT_NUM = %s", (amount, target_account_num))

def transfer(self, transfer_amount, source_account_num, target_account_num):
try:
cursor = self.factory.getConnection().cursor() # DB-2.0 API spec says that creating a cursor implicitly starts a transaction
self.withdraw(transfer_amount, source_account_num)
self.deposit(transfer_amount, target_account_num)
self.factory.getConnection().commit()
cursor.close() # There wasn't anything in this cursor, but it is good to close an opened cursor
except InvalidBankAccount, InsufficientFunds:
self.factory.getConnection().rollback()

If you can rip out that plumbing code, your code becomes cleaner, easier to read, and easier to maintain. The following shows the code where you don’t have to explicitly manage the transaction. Instead, it is declared using a python decorator.

class Bank:
def __init__(self):
self.factory = factory.MySQLConnectionFactory("springpython", "springpython", "localhost", "springpython")
self.dt = DatabaseTemplate(self.factory)

def balance(self, account_num):
results = self.dt.query_for_list("select BALANCE from ACCOUNT where ACCOUNT_NUM = %s", (account_num,))
if len(results) != 1:
raise InvalidBankAccount("There were %s accounts that matched %s." % (len(results), account_num))
return results[0][0]

def withdraw(self, amount, source_account_num):
if self.balance(source_account_num) < amount:
raise InsufficientFunds("Account %s did not have enough funds to transfer %s" % (source_account_num, amount))
self.dt.execute("update ACCOUNT set BALANCE = BALANCE - %s where ACCOUNT_NUM = %s", (amount, source_account_num))

def deposit(self, amount, target_account_num):
self.balance(target_account_num) # Implicitly testing for valid account number
self.dt.execute("update ACCOUNT set BALANCE = BALANCE + %s where ACCOUNT_NUM = %s", (amount, target_account_num))

@transactional(["PROPAGATION_REQUIRED"])
def transfer(self, transfer_amount, source_account_num, target_account_num):
self.withdraw(transfer_amount, source_account_num)
self.deposit(transfer_amount, target_account_num)

If you focus your attention on the transfer method, you should notice an improvement in readability. At first, maybe it doesn’t like much. But try to visualize cleaning up a whole slew of transactional blocks of code you have to maintain, and the benefits starts to add up.

Also, I don’t know if you noticed, but the withdraw and deposit operations aren’t safe enough to be part of the public API. They need to be transactional as well. So why aren’t they? Well, I wanted this to be an apples-to-apples comparison. You can’t just blindly wrap the first sample of deposit with a transaction, because the transfer function may have already started a transaction. You need to test to see if you are already part of a transactions.

Guess what: @transactional handles this for you. The second example is easy. Just put @transactional in front of withdraw and deposit and you are done. The PROPAGATION_REQUIRED tag indicates a transaction will be started if one doesn’t already exist. Doing this with the first example would have required hand-written code to track whether you were in or out of transactions.

class Bank:
def __init__(self):
self.factory = factory.MySQLConnectionFactory("springpython", "springpython", "localhost", "springpython")
self.dt = DatabaseTemplate(self.factory)

def balance(self, account_num):
results = self.dt.query_for_list("select BALANCE from ACCOUNT where ACCOUNT_NUM = %s", (account_num,))
if len(results) != 1:
raise InvalidBankAccount("There were %s accounts that matched %s." % (len(results), account_num))
return results[0][0]

@transactional(["PROPAGATION_REQUIRED"])
def withdraw(self, amount, source_account_num):
if self.balance(source_account_num) < amount:
raise InsufficientFunds("Account %s did not have enough funds to transfer %s" % (source_account_num, amount))
self.dt.execute("update ACCOUNT set BALANCE = BALANCE - %s where ACCOUNT_NUM = %s", (amount, source_account_num))

@transactional(["PROPAGATION_REQUIRED"])
def deposit(self, amount, target_account_num):
self.balance(target_account_num) # Implicitly testing for valid account number
self.dt.execute("update ACCOUNT set BALANCE = BALANCE + %s where ACCOUNT_NUM = %s", (amount, target_account_num))

@transactional(["PROPAGATION_REQUIRED"])
def transfer(self, transfer_amount, source_account_num, target_account_num):
self.withdraw(transfer_amount, source_account_num)
self.deposit(transfer_amount, target_account_num)

To tell you the truth, writing functional code I can maintain is my #1 goal in software development. I have inherited many projects that were impossible to read. They worked, but because they were so poorly done, I couldn’t enhance or improve them, and thus began the refactor/rewrite.

Using fancy tricks that are hard to read the next day is bad – True

Only use Spring Python if you need it. And only use what you need. If you really need to mark up transactions in your code, this is perfect. But if your tool doesn’t need that protection, forget about it! If your tool needs rigorous testing, utilize the IoC container to inject test doubles. But if you are writing a simple admin tool, then perhaps you don’t need any IoC at all. Same goes for all the features.

If you need IoC and you are used to reading XML-configured app contexts (perhaps because you used to write Java code a lot), then you can easily utilize the functionality of Spring Python. But if you have been a pythonista since day one, forget about it! Use the pure python, decorator based container, and don’t waste one minute on XML.

Using Problem A’s solution for Problem B without analysis is dumb – True

Just because an AOP transactional interceptor was what you needed for Problem A, doesn’t mean you need it for Problem B. And don’t assume that your strategy of IoC injection points for Problem A will map cleanly to Problem B. Again, TDD should help you expose the injection points. I can’t possibly predict where you need to inject dependencies. That is what software analysis is for.

Anyone can pick up and use a library. But using it for your needs (and not mine) is where judgment calls come in. Your experience in using tools to solve problems is needed to figure out what parts of Spring Python will serve your needs.

XML is the way to configure everything – False

This is a false assumption, and unfortunately an oft repeated one. Notice that there was no XML to be found in my samples? I stuck with pure python, using what I argue is similar to a DSL. Consider writing your code to have some fluent APIs, and the pure python IoC container can easily and efficiently wire your application together as needed. I have also done my best to update documentation to show configuring things either way. Whether or not the documentation is up-to-date, please remember, you can ALWAYS stick with pure python.

Java’s solutions are perfect for Python – False

I already wrote how Spring Python is not just a simple port. The idea is to grab higher level concepts that transcend languages, and see where they apply. Some perceive this as grabbing Java’s solutions and dropping them into Python. It’s not. I know a dozen or so computer languages (C, C++, Tcl, tk, expect, PERL, java, python, csh, bash, Forte 4GL, and more), and some problems I find occur no matter where I go. Some carry a similar solution, some don’t. The key factor is, many of these problems need a solution that looks similar from 10,000 feet, but when you look deeper, the details are very different. That is what Spring Python strives for: solving some of the same high level concepts, where the high level perspective looks similar, but the details use the power and idioms of python. If you perceive one of my ideas as being out-of-whack, please drop me a comment, and I will be happy to discuss it.

Dependency Injection should be used for every problem – False

You need look no further than at coily, our command line tool, to see that some solutions don’t require any Spring Python. Instead, good ole’ raw python does the trick. Use Spring Python sparingly, and when it improves your efficiency, NOT just for the sake of using. This is the type of critical judgment you should apply to any library you utilize. Use what you need to solve your TDD test cases, and when they pass STOP! Move away from the keyboard. You are done.

Conclusion

I will say I have presented at least a minimal case that I’m not really using Java idioms, but instead solving higher level problems. People may assume I am reusing the idioms considering my library was inspired by one written in Java. But you really need to peel back the layers and see what is happening. This is true for any library you use. Look at the documentation we provide and inspect the source code, and see if you can spot any anonymous inner classes, or getter & setters. You won’t find them. Instead, you find anonymous functions, python iterators, usage of standard python libraries, etc, and what I consider a slim-and-trim library of functionality. And on top of that, lots of documentation on how to use it along with a solid forum of support. Makes it more than just a random recipe posted somewhere, and subject to discovery by google.

Does Spring Support support Python 3.0?

At the end of my talk yesterday, one of the attendees pointed out that Python 3.0 was released that very same day. In joking fashion, he asked, “Do you support it?” I chuckled and did explain that currently we were supporting 2.4 and 2.5 because essentially, that is what I have personally ran the automated test suite against. I hadn’t been able to run it against 2.6 yet, but intend to do that at some time. I would like to cover as much as possible, but there will definitely be limits to what is possible with a single baseline. Of course this begs the question: what DO you do when you hit that limit?

We need to determine how much interest there is, because it will demand we maintain multiple branches of Spring Python. That gets costly. I discussed this point over lunch with Russ, and he agreed with me that our primary interest is to support production systems. I believe that most people are NOT jumping rapidly to Python 3.0, so…why should we?

This is the same problem Spring Java has to deal with in regard to supporting Java 1.3, 1.4, 5, and 6. Spring 2.5 is currently supporting Java 1.3-1.4 with most classes. They also have additional classes to support Java 5/6. However, Spring 3.0 is going to move up to Java 5 and drop support for the older JVMs. This gives them the space to remove some older code, make other code fully utilize generics, etc. Like always, if you buy a support contract, SpringSource will certainly support you on an older JVM. But, they can’t hold the entire framework up forever, so they are following what the industry is using, any many have moved up to Java 5.

Likewise, Spring Python is following the Python industry, and Python 3.0 is a little too new. Don’t get me wrong. We’ll check it out. Perhaps it works out of the box for us, because we haven’t used too much yet. Still, there are lots of features we want to get implemented that are way more pragmatic and useful than supporting Python 3.0.

Spring Python isn’t a simple port of the Spring Framework

While surfing across Google, I spotted a conversation where a couple guys were commenting on Spring Python. The only commentary was based on the Wikipedia entry, and showed no delving into official documentation or our website. Well, I can’t help but respond to the some of the assumptions that were made.

Spring-Python. A port of the Spring framework to Python. It’s like people haven’t read http://tinyurl.com/3jv3m

Uhh, I HAVE read Python is Not Java. In fact, I found it soon after I started this project, and took many of the points they were making to heart. If you look at the code, you will notice that a) there aren’t getters and setters all over the place typically found in Java, and b) I cleaned up the code base to better conform with PEP-0008. I have had other pythonistas look at my code and give me feedback so I can make this project as pythonic as possible.

That is, of course, a little unfair. They might be writing it in a Pythonic style. But still, aren’t there enough web frameworks?

Thanks for sticking up for me. I certainly appreciate that. And to clarify, yes, I am trying to writing this in a pythonic fashion. That is why I welcome other pythonistas to visit my code and seeing what I’m doing. Please send me patches or comments. However, please understand that Spring Python is NOT purely a web framework! This must one of the most frustrating things for the Spring folks. Spring MVC and Spring Web Flow may be web frameworks, but the whole framework isn’t just for web apps. In my daytime job, I use Spring all the time while NOT writing a web app! The stuff they have written is universal. Web app or not, there is good leverage inside their code, and I wanted the same power when writing python apps.

And reading the definitive source (Wikipedia, natch), I don’t see anything in there that isn’t done better in, say, Django.

Okay, how do I say this? Wikipedia is NOT the definitive source. While I admit that I wrote most of what’s in there, it is a NPOV, quick description of things. The definitive source is http://springpython.webfactional.com. I’m not trying to compete with Django or replace it. Again, Spring Python is NOT a web framework. I have written samples of how it can enhance your CherryPy app. I want to study the other python web frameworks, and see if Spring Python can help out.

Sure, but doesn’t everyone already have an ORM layer with a better description than “the SQL query and row-handling function”?

I have supported a particular application for several years, and it now has 200+ queries. I’m not about to migrate it to an ORM solution because I don’t have the time. I didn’t know that ORMs were the final answer to database access. It seems people are still debating the impedance mismatch between objects and relational databases. Sometimes, the technology-neutral language of SQL is the most portable, expressive way to communicate. Got an ORM? Do you enjoy using ORMs like SQLObject or SqlAlchemy? Great! Spring Python won’t get in your way. Frankly, I haven’t spotted ANYTHING where we can help them, either. But do you use pure SQL queries? We DO have something for you.

Reading from the database hasn’t required “a monotonous cycle of opening cursors”, etc. in a long time.

Again, if you are using an ORM, then Spring Python doesn’t have anything today. But if you are writing some pure SQL, you need to manage your cursors correctly. Have you read PEP-249 Python Database API Specification v2.0? The spec defines two method calls, one for connection objects and one for cursor objects: close. It clearly indicates that you can use this to close the connection before the object itself goes out of scope. If you are writing a tiny python app that exits after a bit, your connections and cursors will be garbage collected and these things will close automatically. But if you want to write any type of enterprise app that stays up ALL THE TIME, then you need to be correctly closing result sets.

I have worked on enterprise level Java apps, and one of the first pitfalls is failure to close cursors and connections when you are done. JdbcTemplate solves this problem for you. Python has the same issue to deal with, and DatabaseTemplate provides the same solution.

And I fully admit I’m being unfair. (Although having said that, XML config files are another thing I’m more than happy to drop.)

Thanks for being honest. I don’t mind someone putting in their $0.02. Regarding XML configuration files, I have two angles of approach: 1) make it easy for existing Java developers who have used Spring to migrate to Python and 2) support new Python users. The XML configuration option is to help support the Java developers. But I totally grasp the concept of avoiding XML situps. It’s why I rapidly wrote a pure python, decorator based IoC container after I had a working XML container. BTW, the key reason I started with XML was the fact that someone had already created one, and it helped me jump start things. Once again, in all tradition of Spring, the choice is yours.

XML may not be the most tasteful thing to pythonistas, but we still have to deal with it. For that, I gladly recommend Amara. We used it to rapidly rewrite our XML parsers. It works wonders.

I just ask that you visit our website, read our documentation, read the source code, and learn what this project is trying to accomplish before you make too many assumptions.