A tale of two mediatypes

By Greg Turnquist

Greg is a member of the Spring team, an author of several books on Spring Boot, conference speaker, and the lead for Spring Data JPA.

January 18, 2018

I’ve spent the last several weeks picking up something I started back in 2015. Way back then, in the airport departing SpringOne, I started working on the third mediatype for Spring HATEOAS. There were already two: the original one based on pure Jackson and HAL.

This was Collection+JSON by Mike Amundsen. As a traveling consultant, speaker, and author, Mike has actually penned several mediatypes while also advocating for evolable, adaptable RESTful architectures. Working on this one, I made considerable progress. The progress was really about me learning how Jackson worked.

When your pull request gets turned down

You see, Spring HATEOAS doesn’t just crank out hypermedia. We have a collection of domain classes that represent a neutral format. Your Spring MVC controllers can actually be used to generate multiple forms of hypermedia. At least, that’s the idea, and in it’s formative six years of development, that premise hasn’t been dropped.

Spring HATEOAS

So after supposedly finishing the job and submitting it to the project lead, Oliver Gierke, I wondered why my work wasn’t getting merged. I chatted with him and learned of another effort afoot that was blocking its acceptance: Affordances. Another group of people were working on a very deep, extensive enhancement to Spring HATEOAS, and Oliver didn’t want to complicate things by having yet another mediatype in the baseline.

Two years later, when I rejoined the Spring Data team, I picked up the mantle of Spring HATEOAS. Oliver’s responsibilities had grown including become the official manager for the Spring Data umbrella. One of my biggest undertakings was to start reading through this Affordances API handiwork and bring it forward. Doing so required months of reading, testing, experimenting, polishing, and coding.

When your pull request gets turned down…again!

My months of effort were clobbered by the inalienable fact that the code had suffered major feature creep. It had also been forked into a couple other branches. Despite months of massaging a pull request that had over 200 commits and getting all of its tests to pass, Oliver and I agreed to shelve that work in favor of starting over.

Yes, I proposed starting over and trying to implement the same concept from scratch. If you want a hint of how long this took, you can see it in the commit logs. Because we squash pull requests, this causes the timestamps to go back to the original, first commit. This image shows code merged on November 29th, yet the commit for the Affordances work dates July 13th.

Ultimately, I crafted a much lighter weight API that allows chaining multiple Spring MVC methods together. With that, Spring HATEOAS can unpack these related links into the right format for each mediatype. For example, HAL-FORMS looks at any POST or PUT methods, finds the input type, and extracts its properties.

Testing an API by using it with another mediatype

So my big test with Collection+JSON was to take this two-year-old effort, rebase it against all of Spring HATEOAS’s improvements, and see if a completely different layout of JSON could be derived from the same vendor neutral representations INCLUDING this new Affordances API.

Step one was to rebase this entire chunk of code. Since it had previously been written against Java 6 and Spring HATEOAS was now on Java 8, that alone was a bit of effort. Several contracts had been altered meaning not all the implementations worked. Had to hammer that out.

Then in the midst of revamping things and reading the spec in detail, I noticed something unfortunate: I had completely misread the format of data. I thought turning a POJO into a simple JSON object was okay:

https://gist.github.com/gregturn/9e05e3dd9d1bfcb3d0e5ad7468f7e4c0

It looked like Collection+JSON’s “data” field would let any type of nested domain object get turned into a simple JSON structure.

Turns out, Collection+JSON requires a very explicit array of key-value pairs:

https://gist.github.com/gregturn/3cffdb21c242c42f5351d791c06a3be5

This is not nestable unless done out-of-band.

Additionally, it required that I code up the ability to extract property names and values from objects for serialization, and go the other way for deserialization. This is kind of like serializing inside the serializer. Yech! But I got it done. So after several weeks, I managed to rewrite half if not more of my original code and get things working.

Next step–Affordances!

I had to hook into the Affordances API and generate custom things pursuant to this new mediatype. Two years ago, I was very aware of Collection+JSON’s “template” and “queries” sections. And I hadn’t a clue how to populate them. Until now.

This would be the moment of truth at how well I had designed this API for HAL-FORMS. Turns out, the API was quite well suited to handling this.

Coding the Affordances API, Oliver had recommended creating an AffordanceModel and AffordanceModelFactory to allow mediatype-specifics to be neatly encapsulated behind a incredibly simple interface until the mediatype’s serialization code could unpack it. Turns out that was a very good decision.

Debugging the code, I could see that a model was now being generated for both Collection+JSON as well as HAL-FORMS, and I only need one of these for any given situation. Having this complex, detailed code is most important, and handily doesn’t leak into unnecessary areas.

I had allowed some Spring MVC specifics to leak into my HAL-FORMS handlers. That was a no-go, because the whole idea is to have things be tech neutral at this point.

One of my last tests was to have one Spring MVC controller wired up, and have it serve queries for two different mediatypes.

With this in place, I began to engage in classic jürgenization. This has taken me several days. I discovered a missing pair of units (serialize/deserialize custom extension of ResourceSupport) and hence missed a critical corner case. Crisis averted, I have pushed off the final set of commits to Travis CI and submitted it to Oliver for review.

Using one mediatype to develop another

Having built HAL-FORMS, I enjoyed leveraging all the lessons learned to build up Collection+JSON. It gave me a starting point for tests that needed to be written, a nice repeatable pattern of serializers and deserializers that also had to be tackled.

And having just recently performed the work for HAL-FORMS, I was ecstatic that when I solved a particular problem for Collection+JSON, I was able to refactor HAL-FORMS to use the same solution, ensuring consistency between the two.

Every time I work on another mediatype, the whole project gets stronger, better, more reliable. That’s why I can’t wait to circle and resume my efforts with UBER Hypermedia. Most of the work is done. Or at least, that’s how it appeared when I was last there!

But I wanted to let it rest, and circle back. WIth three mediatypes implemented (HAL, HAL-FORMS, and Collection+JSON), UBER will be a real sizzler. After that, SIREN, and then possibly XHTML.

See you then!

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *