Category Archives: java

Something Java should never do

I’ve been working on a pull request on Spring HATEOAS for six weeks. That’s right, six weeks. It was a community contribution, and there’s a lot to sift through. Yesterday morning, I was slated to conduct the first review with the project lead. But that wasn’t going to happen, because Java decided to do something Java should never do.

Maven wouldn’t build my project while IntelliJ IDEA would.

I was alerted to this issue Thursday morning when I created a “run all profiles” script and it wouldn’t work. Huh? Every time I’ve pushed an updated commit to Travis CI, it builds perfectly. That’s right, a CI server, running Maven, built my branch with ease. But my Mac would not.

This made no sense. This is something Java should never do! Java doesn’t work on Linux but fail on a Mac. It must be my machine, right? So I log onto my wife’s MacBook Air, update Java to the latest version, grab the source, and fire off Maven. BOOM! It breaks just the same.

So…here comes the online review with Ollie. I pronounce my inability to build the system. The man with twenty years professional experience, who has written four tech books and one tech video, who has been fighting this for twenty four hours, pleads with his manager for clues. Ollie grabs the branch, tries to build, and SPLAT. It doesn’t work. He goes through the same questions I do, and finds no answers.

We jump to our OTHER CI server, Bamboo, and he commands it to build my branch. It works. Perfectly.

Score so far: Three Macs – 0  Two Linux CI servers – 2

“Ollie, this is something Java should never do!” I scream into my Google Hangout. (Okay, maybe I didn’t scream.)

He nods along. The review is busted. We both flip to our daily standup Hangout, and Ollie’s first words are, “Greg, I know what you did.” Heh.

We wrap that meeting up, and I proceed to dig through every Stackoverflow article I can find on Lombok, maven-compiler-plugin, and Java 6. Talk about a nebulous combination. WIth enough evidence that this is Lombok’s fault, I open a ticket.

Fun day, given it’s my son’s last day of school for the year. I’m irritated at having to stop work to fetch him. My brain keeps churning the whole time. The second I get home, I have him go play. Anything to get back to my keyboard and keep trying options. I flip through a myriad of options for maven-compiler-plugin: memory, forked JDK, use Java 8, hand configure annotation processors. Nothing works.

My three-year-old is trying to climb up on me, and I push him to the side. “I must find the answer!” Time to pick up our 7-year-old arrives, and I dash off. After getting home, she is asking me to write LOGO code. “No sweetheart, I need to try something first.” I usually stop ANYTHING to help her, but I was too obsessed with this issue.

At this point, I start to De-lombok the code. Maybe if I can flip Lombok off. I get 90% of the code converted, and IT STILL WON’T BUILD! What could this possibly be? Every debug/verbose flag I activate still shows me nothing about what is actually breaking. Just some missing symbols followed a completely corrupted class.

Then I get a strange idea. A curious insight. What if the first class that Maven claims about, a nested static class, was made top level? I command IntelliJ to perform this refactoring. Poof, the error message changes. So I make the next nested static class in the list of errors top level as well.

BANG! Reading “[INFO] BUILD SUCCESS” makes my jaw drop.

I cancel all my changes, and build again. Same failure. I then strategically pick one nest static class, and move it. Everything suddenly works. What the…? This is something Java should never do.

Looking at the Jackson2HalModule (the predecessor to my work on Jackson2HalFormsModule), I count the number of nested static classes and compare to this one. The HAL parser has eight or nine, the new work has over twelve. Somehow, this enclosing class has TOO MANY NESTED STATIC CLASSES!

Not wanting to move everything into public classes, I rig up HalFormsSerializers and HalFormsDeserializers, and split up the nested static classes into these two “namespaces”. Everything is humming. This change is mind boggling. Because this is something Java should never do.

So what is happening? Still not sure, but it implies that Lombok’s annotation processor must scan every class looking for its annotations. If the class is too deep, the scanner breaks and doesn’t recover, hence causing any nested classes further on to fail. And somehow, this state is contingent on the platform, because it works jim dandy on Linux while failing on the Mac.

I copy all these details into the Lombok ticket I opened. I hope I shined a light on an issue perhaps the committers can pinpoint and fix. But for now, my PR is buffed up and reviewable again. And as I go to bed, I can finally sleep in peace, having slain another dragon in the realm of open source hacking.

 

Learning Spring Boot 2nd Edition 80% complete w/ Reactive Web

This weekend I sent in the first draft for Chapter 2 – Reactive Web with Spring Boot (of Learning Spring Boot). Even though this is Chapter 2, it’s 80% of the book. That’s because I’m writing Chapters 2, 3, and 4 last, due to the amount they depend on Reactive Spring.

This may sound rather awkward given Spring Boot has yet to release any tags for 2.0. Pay it note, there is a lot of action in Spring Framework 5.0.0 such that it’s already had several milestones. A big piece of this book is getting a hold of those reactive bits and leveraging them to build scaleable apps. The other part is how Spring Boot will autoconfigure such stuff.

Thanks to Spring guru Brian Clozel, there is an experimental project that autoconfigures Spring Boot for Reactive Spring, and will eventually get folded into the Spring Framework. Bottom line: Reactive Spring is available for coding today, albeit not with every feature needed. But since the target release date is May, there will be time for spit and polish against the book’s code base.

And now, an excerpt from Chapter 2, for your reading pleasure:


Learning the tenets of reactive programming

To launch things, we are going to take advantage of one of Spring Boot’s hottest new features: Spring 5’s reactive support. The entire Spring portfolio is embracing the paradigm of reactive applications, and we’ll focus on what this means and how we can cash in without breaking the bank.

Before we can do that, the question arises: what is a reactive application?

In simplest terms, reactive applications embrace the concept of non-blocking, asynchronous operations. Asynchronous means that the answer is coming later, whether by polling or by an event pushed backed to us. Non-blocking means not waiting for a response, implying we may have to poll for the results. Either way, while the result is being formed, we aren’t holding up the thread, allowing it to service other calls.

The side effect of these two characteristics is that applications are able to accomplish more with existing resources.

There are several flavors of reactive applications going back to the 1970s, but the current one gaining resonance is reactive streams due its introduction of backpressure.

Backpressure is another way of saying volume control. The consumer controls how much data is sent by using a pull-based mechanism instead of a traditional push-based solution. For example, imagine requesting a collection of images from the system. You could receive one or a hundred thousand. To prevent sthe risk of running out of memory in the latter, people often code page-based solutions. This ripples across the code base, causing a change in the API. And it introduces another layer of handling.

For example, instead having a solution return a risky collection like this:

public interface MyRepository {
List findAll();
}

We would instead switch to something like this:

public interface MyRepository {
Page findAll(Pageable p);
}

The first solution is simple. We know how to iterate over it. The second solution is also iterable (Spring Data Commons’s Page type implements Java’s Iterable interface), but requires passing in a parameter to our API specifying how big a page is and which page we want. While not hard, it introduces a fundamental change in our API.

Reactive streams is much simpler – return a container that lets the client choose how many items to take. Whether there is one or thousands, the client can use the exact same mechanism and take however many it’s ready for.

public interface MyRepository {
Flux findAll();
}

A Flux, which we’ll explore in greater detail in the next section, is very similar to a Java 8 Stream. We can take just as many as we want and lazily waits until we subscribe to it to yield anything. There is no need to put together a PageRequest, making it seemless to chain together controllers, services, and even remote calls.


Hopefully this has whet your appetite to code against Reactive Spring.

Happy coding!

 

I owe so much to the @javaposse. Congrats on 10 fine years!

javaposseYears ago, I discovered the Java Posse. It was a phenomenal podcast. Every episode was chockful of deep technical discussions, modern issues and revelations in the Java community, and just good old fashioned fun. I thirsted for every episode.

These gentlemen were technical experts who also had loads of real world experience. They pointed me to new technologies, emerging technologies, and fostered my desire to start read blog sites. Back then, I had discovered Google Reader and slowly accrued over 100 blog sites to monitor. All of this constant reading and digesting, along with links from the Posse themselves has made me a better developer.

If I had relied on the dusty books on my shelf, my skills would never be where they are today. By constantly growing and polishing my skill sets beyond what my job demands, I also grew a hunger that I realized couldn’t be satisfied at my old company.

When my wife discussed changing jobs so we could relocate, I had been encouraged to be bold. Asking myself, “where do I REALLY want to work?” The answer I immediately felt: “SpringSource”. I made it a goal of mine. It took patience, but a year later, I was presented an offer. And here I am.

Reading just today on twitter that people were attending the final episode struck me very strongly. I knew it wouldn’t last forever. In the past couple of years, their subject material was shifting away from Java and into other areas. That was natural, because the members had moved into other areas of the vast Java community.

In the end, ten years of podcasting is an incredible story. Congratulations Dick, Tor, Joe, Carl, and Chet! Your production has touched more people than you can possible imagine.

 

Chapter 2 shows #SpringBoot + #java + #spring #mobile + #jquery. Fun!!!

learning_spring_boot_ch2I just shipped off chapter 1, “Quick Start with Groovy”. Last night, I dug through and updated the code samples along with outputs to Spring Boot 1.1.1.RELEASE, which went out late yesterday. It really is awesome having that effort minimized thanks to AsciiDoc! Basically, I edited the code directly, re-ran the bits where I had captured console outputs so I could recapture them, and made other suitable updates.

Now it’s time to dig in deeper on chapter 2, “Quick Start with Java”. The neat thing with Spring Boot is that what would be usually trimmed down to toy apps due to space constraints can really show off some slick stuff.

ch2_fodtIn this chapter, I am writing an app that will scan multiple GitHub repositories for open issues and print them out on a web page. It shows the name of the repository, a hyperlink to the issue, and the title. I could put more, but I think this gets the point across.

From there, I will ramp up things wicked cool by adding mobile support. I’ll plug in super simple automatic view switching between desktop and mobile views thanks to Spring Mobile. And I will add a mobile view with jQuery Mobile. Given the rise of mobile-based apps, I think this will be VERY appealing to my readers.

And once again, thanks to AsciiDoc, I don’t waste time on typesetting code. Instead, I WRITE the code, and add explanations and details, a lot like writing reference docs. Yum!!!

Learning valuable lessons in the trenches of production

computer_coffeeA key turning point in my career was when I was able to take over a non-functional Java Swing app. I had worked for years on a mission critical, 24×7 system written in a now non-existent language and runtime. Frankly, I was a bit tired of it. I had managed a total of seven employees over the years. Then I found one who was perfect to take over. He didn’t suffer from the burn out I was approaching, and he had new ideas. So I put him in charge. When one of our other devs announced he was leaving the company, I asked my manager if I could take over his app. It was a tiny, Java Swing application that the target users simply didn’t use to complete their daily job.

I found out why. This application had been under development for two years. Six devs had worked on it. I heard how one program manager had complained in strong language how too much money had been invested into that app, and it STILL didn’t accomplish anything. Let’s see:

  • non-functional: check!
  • Java Swing (who wants THAT?): check!
  • the current developer would be gone and not available to answer questions in two weeks: check!

A 100% basket case. And I was hungry to take it over.

Before I lifted a finger, I pledged that I wouldn’t commit a line of code unless that code was covered by an automated test. A simple pledge, and the only thing I went in with. I didn’t know how difficult it would be to uphold such a commitment, let alone how gratifying its outcome would be.

Current state of things

The app was used by the finance team to ingest itemized invoices in the form of spreadsheets from our subcontractors. While we had over three hundred, there were about six different formats. The first test case I wrote involved reading in an Excel spreadsheet with something like five thousand rows of data, storing it’s contents in the database, and doing some other post processing. That process probably took a minute. I knew that down the road, this would get costly.

But it was through this painstaking process that I started weeding out what was needed and what was a waste of time. The code had a layer of abstraction around the spreadsheet that frankly was useless. It was like having the ability to abstractly fetch a column of data, only this wasn’t needed. So once I had wrapped invoice processing in a handful of test cases, I went in and started ripping this out.

And then I discovered other patterns and anti-patterns at play. I started ripping out other things. Simplifying. Reducing. The app was beginning to function. I was interacting with the users and had begun implementing features they needed. I had only been on the code for about three months when the finance team leader decided it was time for a demonstration to program management

Acceptance by management

I could do mission-critical-never-goes-down in my sleep. The side effect was nothing would faze me. I wanted to demo this app in front of managers that had seen this app in the past, and worried that too much money had been spent. It was important to make it work right.

I knew what would be perfect. I wrote a single, length JUnit test case that basically walked through the app like a movie script. I spent an entire month on this script. (I have since learned this would be viewed as an acceptance test). Every day, I walked through the script and tried to add one step. And another. And another.

What this script was doing was forcing me to decouple the Swing UI layer from all business logic. My automated movie script was essentially driving the app in a headless fashion, and it was so cool.

Then came the day of demo. I printed my movie script on paper, and walked into the conference room. I would indeed drive the GUI using the movie script. Essentially, it told me each and every step. And I had 100% it would work flawlessly.

Suffice it to say, management was awestruck. I didn’t care that this made me look good. I just loved that the power of test automation had allowed me to build a flawless demo. I also loved that all this test driven development was empowering me to rip out junk, rewrite flawed patterns, and basically Do The Right Thing. It was liberating and sold me on the power of TDD.

Sure my entire coverage was something like 60%. But something I learned was that 60% vs. 80% is a really worthless comparison. If the team is spending 99% of their time using the 60% that works, then it’s effective coverage.

Welcome Spring and other OSS tools

As I grew the app, I knew I had the ability to pull in open source tools as needed. I reached for Spring, something I had read about previously in “Spring: A Developer’s Notebook”. I started to use it, but thanks to my TDD approach, I only used the parts from this bag-o-tricks that I needed. The feeling was nothing less than glorious.

After about six months of work, things were sizzling. I was having weekly meetings with the users. They would describe features they wanted. I would write a test case, and sometimes have another released out that week, or even the very next day. They had never seen anything like that.

Less is more

One day, my manager called me into his office. By “manager”, I mean the guy that I used to share a cubicle with, write code side-by-side, and whom was a victim of his own success. I wasn’t worried. Just curious.

One of his fun jobs is that he has to count lines-of-code for every app once a month and present it as part of his monthly report to senior management. He showed me the graph of my app going back to before I had touched it, all the way up to today. It had risen steadily, peaked a few months ago, and began to go down.

“None of the others apps is doing this,” he said. “No one else has actually reduced their lines of code.”

I hadn’t realized this until now. I was actually shrinking the application, and yet it did more than ever. It was loaded with new features. But I had managed to yank out useless abstractions, features nobody used, and simplified its architecture. The net effect was less code despite having added gobs of automated test code.

This is when I discovered that less is truly more. And the only way to discover this is through test automation. I even learned that less testing is more. The test cases were built up on the flow of “clean database, ingest invoice into database, examine database”. At one time, my test suite took an hour and a half to run. When I had to throw away an entire day’s worth of changes because I hadn’t run the test suite and broken half the system, I paused and went to clean up the tests.

I honed things to get them back down to thirty minutes. A decent price to pay for a solid application.

Users and developers, working together

I reached a point where we had a “summit” of sorts between myself and the users. They had some new features they needed that really went off on a tagent. We met in a conference room, and I fired up PowerPoint. We captured each use case as a separate slide. They could understand that. I would then convert each slide into a separate test method, using the language of the slides in my code. We spent a week honing the slides so there was concurrence.

I then spent may be a couple weeks implementing the features. And we were done with that go around! I showed the actually coded tests to the user’s team leader. She wasn’t technically savvy. But she could easily read my automated test because it was written in the identical language of the slides.

When I listened to a recent episode of the Java Posse where they were discussing Domain Driven Design, I was connecting with so many points being made.

  • Common set of terms
  • Agreed upon vocabulary

I had brought the system to a point where the users had an active participation in what was built. I perceived myself not as the focal point the way I had in the past. Instead, the business these people were doing was the focal point, and I was the support guy merely making it possible with a bit of software.

The future

It was at this point, when things were humming along solidly, that I received my offer from SpringSource. I told the financial team leader that I would be departing in a couple weeks. He smiled and said, “well the timing couldn’t be better.”

I asked him what that meant. He indicated that my work had made things great for the users, but they still had a terrible inefficiency. The labor cost of a half dozen accounts receivable staff was way over budget. There are companies that specialize in this exact work and even get bonus checks for finding overcharges. And were cheaper. They were going to decommission my application and subcontract the work to someone else.

I chuckled. All my effort would be scrapped because doing this in house just wouldn’t work.

I wasn’t upset. Instead, I was excited! Maybe that sounds strange, but the past year had been like a playground where I got to test out a lot of ideas I had been reading about in books, blogs, and listened to in podcasts.

I had grown so much. And it really prepped me for my new position. I felt like the past year combined with my mission critical experience gave me a great set of skills. I was more excited than ever to join my new team.

Having a ton of fun with Spring Data REST + Javascript

Today has been 100% fun coding with Spring Data #REST, javascript, and pics of cats. 🙂 —https://twitter.com/gregturn/status/448219842056052736

Since last fall, I have shifted to working full time on Spring Data REST. This is an amazing project. It takes Spring HATEOAS & Spring Data and combines them together. This way, your Spring Data repositories are exported with an automatically generated hypermedia interface.

What does this mean? Maybe it’s not obvious, but data stored in a persistent repository is now available to desktop web apps, smart phone hybrid apps (those that basically wrap a browser), and native iOS/Android apps.

I’ve spent several months getting up to speed on this project by fixing bugs, adding new features, and adding to the test suite. This has slowly but surely helped me understand how it works. I don’t have it all down, but when we reached 2.0.0.GA, and had the basic CRUD functionality working, I proceeded to write a handful of getting started guides. This way, other people could quickly see through simple examples how to add a hypermedia interface to their existing Spring Data-powered apps.

But for at least six months, I’ve had a vision for where I could take this project. I finally felt I had enough know how to start working on a sample application that would heavily use this hypermedia + some javascript to build a dynamic web app. (NOTE: There are pictures of cats!) That’s what I’ve been working on for the past couple weeks.

Thankfully, we have some super sharp javascript coders on our team. I’ve been able to get solid feedback on my newbie javascript code. And it’s been fun! Javascript has a neat, function-oriented perspective. Instead of focusing on classes, i.e. nouns, I have been writing in terms of verbs. When you click this, do that, then that. It’s a big shift. But I ran into familiar territory when I needed to splice some data together. One of my javascript colleagues told me about reduce(), and when I read about it, I recognized it was the same as foldLeft.

Ultimately, I didn’t use it, because it didn’t fit what I needed, but nonetheless, it was nice to run into javascript’s nice functional APIs.

I have been having an extra dose of fun this past week. My colleague commented, “hey @Greg. looks like you’re having a ton of fun! :)” That was the truth! I just didn’t realize it was so obvious.

I’ve always been a language junky. While I enjoy writing Java full time and am really good at it, I love the opportunity to write code in any language. (Okay maybe not COBOL, but I think you get the idea.)

Hopefully I’ll be able to offer a presentation at SpringOne on this exact topic. It’s not guaranteed, but I’ll see if I can get a talk approved. Now…back to coding javascript!

What is the best testing tool?

Someone posted to me a question through meetupcom, “Greg, what is the best testing tool?” I didn’t have room to reply. I posted my response inside the Nashville JUG Google Group we host, but I thought today, it would be better to capture it here.

Asking “what is the best tool” with no other context sounds like you are looking at the situation from the wrong point of view. Let me explain.

A few years ago, I inherited a java app with hardly any automated tests, had been worked on by 6 engineers before me for a couple of years, had been demoed in front of program management, and was non operational. It did stuff, but not much and was loaded with errors. I adopted an approach where I wouldn’t work on any bug/issue until I could write an automated test to reproduce the issue. This was painful; more than you can realize. The software was tightly coupled, lots of static singletons, i.e. global variables, and hard to isolate. Basically, my first test involved using JUnit to empty some database tables, programmatically ingesting a spreadsheet, and then inspecting the database for results. This test took 5 minutes to run. It probably ran through a big chunk of code.

The problem was the system, and no magic test tool was the answer. Changes in the beginning were slow, but as I gained momentum, I eventually reached over 60% test coverage. This may sound low, but within 3 months, I made a flawless demo before program management. They were impressed that it worked, let alone with no problems. The tool was starting to get used by the intended audience. A year later, reported bugs were fixed in one day, and never regressed. I reached a point where I worked for a whole day, and when the half the unit tests failed, I threw away all changes and went home for the day, depressed. I started fresh the next day and actually fixed the problem. Two days to fix it! With no automated test suite, that would probably have been released, and incurred a gob of new bugs to deal with.

I met weekly with the users and captured new feature requests, problems, and generally made this tool work really well. The customers were very happy. I was also empowered to rewrite whole sections that were slapped together hastily in the past, because I had the security of my automated test suite. I threw away code that wasn’t used and didn’t work. My boss showed me a chart where the total lines of code became less than when I first inherited, yet it did more than those other 6 people could squeeze out of it. That was a happy day!

What I’m saying is that I could have used JUnit, TestNG, ScalaTest (works on java too), or any other suite of tools like acceptance testing frameworks, load testing, etc. But what test tool was used wasn’t important. Adopting a strategy of making the whole thing subject to testing, staying test focused no matter how painful it is, paid off. At some times, the test suite took 1.5 hours to run. I spent three days speeding up the most expensive parts of the system, and cutting out certain tests, getting the same number of tests to run in 30 minutes. I came up with a comprehensive test suite and a smaller, smoke test that ran much faster. I also created a spreadsheet to track number of tests and total test time, along with a graph. As the test time grew, I would periodically halt development and polish up certain parts that made it too hard to run tests. I would run the test suite at least once a day to make sure things worked right.

This whole development period was some of the best coding I had done in a long time. Cranking out top quality code with a warm fuzzy green bar made me grin ear-to-ear. When I left that company, I cried a bit because I wouldn’t be working on that tool anymore.

In the 9th chapter of my book, Python Testing Cookbook, the first recipe captures a lot of what I wrote up above, with some more detail. To quote the recipe “Something is better than nothing,”

“Just don’t let anyone tell you that you are wasting your time building a long-running test case. An automated test suite that takes an hour to run and is exercised at least once a day probably instills more confidence than clicking through the screens manually. Something is better than nothing.” —Python Testing Cookbook, page 326

When I wrote chapter 9, I wanted to move beyond simple coded recipes, but instead general lessons I had learned in the realm of testing. These principles work, whether you are writing Python, Java, Scala, or anything else.

Creating SSL sockets in Java using out-of-the-box APIs isn’t good enough

Lately, I have had to work on a Java solution that involved locking down SSL. What do I mean by this? Quite simply, the list of default ciphers provided to Java’s SSLSocket/SSLServerSocket includes some really crazy choices. A few are low grade ciphers (40 bit and 56 bit), and some even have no encryption.

Essentially, if you are going to write any code with an SSLSocket you MUST override the default ciphers and put in a better choice of ciphers. Sadly, this means that for every app you write with SSL sockets, you must include some custom code to override these settings.

What do you call it when you write the same code over and over again? A code smell! Code smells require some form of refactoring to avoid. In this situation, you should delegate creating SSL sockets to a library dedicated to making SSL sockets easier to configure correctly. I believe such a library is needed.

Encryption always carries a sticky issue: not all countries support all the ciphers. What I mean is that it is illegal to use certain levels of encryption technology in certain countries. Basically, if you build an app and sell it to a customer located in a country that can’t use your default keys, they may need the ability to reconfigure to something weaker. You need to build your app so that it can use either property files, Spring application contexts, or other means to adjust the ciphers supported. This can really become a hassle if you have to implement it more than once.

Another issue is that your odds of making mistakes goes up every time you implement that solution. That is why libraries are really handy. For all this, I have decided to reach out to the creator of Jasypt (Java Simplified Encryption) to see if he feels it belongs in his library. If not, I think creating something on github would be in order.

Please don’t say “don’t do that”

I have been digging for a couple of days on how to build a Scala List inside Java. The blog entries are hard to find. Most talk about converting a java.util.List into a Scala List from inside Scala. That is not what I’m looking for, but what really infuriated me was reading a blog entry that told me to just not do it.

I’ll show you how to do what you want in a second, but first, and importantly, I need to explain that what you actually want to do is NOT convert a Java List to a Scala List. Why? It’s because, despite the name similarity, they’re actually very different things…http://grahamhackingscala.blogspot.com/2010/02/how-to-convert-java-list-to-scala-list.html

That is really annoying. Please don’t tell me what to do and not do, okay?

  • I know that there are gaping differences between java.util.List’s interface scala.collections.immutable.List’s concrete implementation.
  • I know that they are backed by different structures. 

That doesn’t change the fact that to embed Scala’s compiler into my Java project, I need to access scala.tools.nsc.Settings which requires that I enter certain parameters as a scala.collections.immutable.List.

This was really annoying, and sounded very similar to the answer I often see when people are seeking help on writing multi-threaded coding in Python. “Don’t write multi-threaded. You are probably doing it wrong.” Sorry, but eventually, most people have to write some multi-threaded code. That answer in the sounds a lot like saying “you will hit the GIL and have all kinds of problems, because Python wasn’t built for multi-threading. And we aren’t removing the GIL.” I find that approach equally annoying. It’s one reason I watch the blogs from PyPy, since it seems the only viable way to remove the GIL from Python.

To cut to the chase, it’s not that hard to build an immutable list in Java. Look below.

The key part is Nil.$colon$colon(“dce”). That is the same as saying “dce” :: Nil in pure Scala. The whole block is showing how to embed the Scala compiler (2.9.1) inside Java, configured with –usejavacp -Ystop-after:dce -debug, and then compiling a string of code (not a file).

All you need to remember is that 1 :: 2 :: 3 :: Nil translates to Nil.::(3.::(2.::(1))). Given that, you can probably figure out many Scala interop issues.

See how Spring Python works with Jython

I recently completed a patch that replaces the amara library with python’s default elementTree library. As much as I like amara’s API, its C-extension basis was unacceptable. I just merged the patch into the trunk, and verified it works with Jython 2.5.1.FINAL. When Spring Python’s 1.1.0.M2 release comes out, we will be Jython-compatible.

With this change, there were only a couple of tweaks to the rest of Spring Python.

What does this mean? How about looking at some code to see how we can start with a Java app, and migrate things over to Python. To kick this off, let’s start with an uber-contrived example Java app: a wiki engine. The wiki engine calls into a data access layer in order to look up some statistics on the site.

public interface DataAccess {

public void statistics();

}
public class MySqlDataAccess implements DataAccess {

public void statistics() {
System.out.println("MySQL: You are calling for some statistics.");
}

}
public class WikiService {

private DataAccess dataAccess;

public DataAccess getDataAccess() {
return this.dataAccess;
}

public void setDataAccess(DataAccess dataAccess) {
this.dataAccess = dataAccess;
}

public void calculateWikiStats() {
System.out.println("You are passing through a pure Java WikiService...");
dataAccess.statistics();
}

}

Let’s assume this is a Spring app. That means we need an XML application context. Let’s create one and call it javaBeans.xml.



xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">








We can start up our pure java app along with its context by writing jython script javabeans.py. It uses Spring Python’s SpringJavaConfig parser.

if __name__ == "__main__":
from springpython.context import ApplicationContext
from springpython.config import SpringJavaConfig
ctx = ApplicationContext(SpringJavaConfig("javaBeans.xml"))
service = ctx.get_object("wikiService")
service.calculateWikiStats()

We just need to compile the java code, and then run our script using jython.

gregturn@startrek:~$ javac *.java
gregturn@startrek:~$ jython javabeans.py
You are passing through a pure Java WikiService...
MySQL: You are calling for some statistics.
gregturn@startrek:~$

Okay, we’re off to a good start. We have tapped a running java system. For our next step, let’s write a pure python version of the data access code and put it into Py.py.

import DataAccess

class MySqlDataAccess(DataAccess):
def statistics(self):
print "PyMySQL: You are calling for some statistics."

class StubDataAccess(DataAccess):
def statistics(self):
print "PyStub: You are calling for some statistics."

Now, to really crank things up a bit, let’s migrate from the Spring Java XML configuration over to Spring Python’s pure Python container.

from springpython.config import PythonConfig, Object
from Py import MySqlDataAccess
import WikiService

class WikiProductionAppConfig(PythonConfig):
def __init__(self):
PythonConfig.__init__(self)

@Object
def data_access(self):
return MySqlDataAccess()

@Object
def wiki_service(self):
results = WikiService()
results.dataAccess = self.data_access()
return results

if __name__ == "__main__":
from springpython.context import ApplicationContext
ctx = ApplicationContext(WikiProductionAppConfig())
service = ctx.get_object("wiki_service")
service.calculateWikiStats()

The top half of our mixed.py script contains an IoC container, except there is no XML whatsoever. Instead, its raw python. You can see how we are also using our python version of MySqlDataAccess.

gregturn@startrek:~$ jython mixed.py
You are passing through a pure Java WikiService...
PyMySQL: You are calling for some statistics.
gregturn@startrek:~$

This shows that we are still using the pure java WikiService class, and it calls into our pure python MySqlDataAccess.

Another useful feature that Spring Python provides is easy access to Pyro, the Python Remote Objects library. This library is for making RPC calls from Python-to-Python. Spring Python makes it easy to link clients and services together with Pyro.

First, let’s take our wiki service, and export it with a server script.


from springpython.config import PythonConfig, Object
from springpython.remoting.pyro import *

from Py import MySqlDataAccess
import WikiService

class WikiProductionAppConfig(PythonConfig):
def __init__(self):
PythonConfig.__init__(self)

@Object
def data_access(self):
return MySqlDataAccess()

@Object
def wiki_service(self):
results = WikiService()
results.dataAccess = self.data_access()
return results

@Object
def exported_wiki_service(self):
return PyroServiceExporter(service=self.wiki_service(), service_name="wiki", service_port=9000)

if __name__ == "__main__":
from springpython.context import ApplicationContext
print "Starting up exported WikiService..."
ctx = ApplicationContext(WikiProductionAppConfig())

We made some slight tweaks to the IoC container by adding exported_wiki_service. It targets wiki_service, and links it with a name and port to expose it as a Pyro service. Spring Python by default will instantiate every object, so in this case, it creates the PyroProxyExporter. Because this isn’t the caller, we don’t need to fetch anything from the context.

Next, let’s code a client to call our service.


from springpython.config import PythonConfig, Object
from springpython.remoting.pyro import *

class WikiProductionAppConfig(PythonConfig):
def __init__(self):
PythonConfig.__init__(self)

@Object
def wiki_service(self):
results = PyroProxyFactory()
results.service_url="PYROLOC://localhost:9000/wiki"
return results

if __name__ == "__main__":
from springpython.context import ApplicationContext
ctx = ApplicationContext(WikiProductionAppConfig())
service = ctx.get_object("wiki_service")
service.calculateWikiStats()

Now, let’s start up the server.


gregturn@startrek:~$ jython mixed_with_pyro_server.py &
[1] 4953
Starting up exported WikiService...

The script is backgrounded, because it is running a Pyro daemon advertising our service. Now, let’s launch the client script.


gregturn@startrek:~$ jython mixed_with_pyro_client.py
You are passing through a pure Java WikiService...
PyMySQL: You are calling for some statistics.
gregturn@startrek:~$

As you can see, our client makes remote to the exported WikiService, which happens to be a pure java service. The java service then calls our python version of MySqlDataAccess to do its statistics call. This goes to show how easy Spring Python makes it mix and match Java and Python.