Author Archives: Greg Turnquist

The Power of REST – Part 1

I was kind of shocked when I saw Yet Another Posted Solution to REST. I sighed and commented, and drew the ire of many. So I figured this might be a good time to gather some thoughts on REST.

The article starts by criticizing REST and offering their toolkit as the solution. What’s really funny, is that the problems they ding are not RESTful issues.

REST requires lots of hops

Let’s start with this one:

As you might notice, this is less than ideal. When all is said and done we have made 1 + M + M + sum(Am) round trip calls to our API where M is the number of movies and sum(Am) is the sum of the number of acting credits in each of the M movies. For applications with small data requirements, this might be okay but it would never fly in a large, production system.

Conclusion? Our simple RESTful approach is not adequate. To improve our API, we might go ask someone on the backend team to build us a special /moviesAndActors endpoint to power this page. Once that endpoint is ready, we can replace our 1 + M + M + sum(Am) network calls with a single request.

This is the classic problem when you run into when fetching a 3NF (3rd normal form) data structure served up as REST.

Tip #1: REST doesn’t prevent you from merging data or offering previews of combined data. Formats like HAL include ability to serve up _embedded data, letting you give clients what they need. Spring Data REST does this through projections, but you can use anything.

In fact, server-side providers will probably have a better insight into exactly the volume of traffic fetching such data before clients. And through the power of hypermedia, can evolve to add links to the hypermedia without breaking existing clients. Old clients can do multiple hops; new clients can proceed to consume the new links, with full backwards compatibility.

REST serves too much data

If you look closely, you’ll notice that our page is using a movie’s title and image, and an actor’s name and image (i.e. we are only using 2 of 8 fields in a movie object and 2 of 7 fields in an actor object). That means we are wasting roughly three-quarters of the information that we are requesting over the network! This excess bandwidth usage can have very real impacts on performance as well as your infrastructure costs!

Just a second ago, we complained that the REST API was serving up too little data, forcing us to take multiple hops. Now we are complaining that it serves too much data and is wasting bandwidth.

The example in the article is a bit forced, given we are probably talking a couple tweets worth of data. It’s not like they are shipping 50MB too much. In fact, big volume data (images, PDFs) would best be served as links in the hypermedia. This would let the browser efficiently fetch a linked item once, and lean on the browser’s cache.

But I sense the real derision in the article is because the endpoint isn’t tailored to the client’s precise demands. No, the real example here is to illustrate a query technique on the client.

Just put SQL in the client already!

Wouldn’t it be nice if we could build a generic API that explicitly represents the entities in our data model as well as the relationships between those entities but that does not suffer from the 1 + M + M + sum(Am) performance problem? Good news! We can!

With GraphQL, we can skip directly to the optimal query and fetch all the info we need and nothing more with a simple, intuitive query.

So now we get to the real intent of the article: introduce a query language. Presumably solving REST’s straw man “problems” (which it doesn’t).

If you want to write a highly detailed query, just open a connection the data store and query directly. That’s what query languages are for. Why invent something that’s weblike, but really just Another Query Language?

What problem are you solving?

GraphQL takes a fundamentally different approach to APIs than REST. Instead of relying on HTTP constructs like verbs and URIs, it layers an intuitive query language and powerful type system on top of our data. The type system provides a strongly-typed contract between the client and server, and the query language provides a mechanism that the client developer can use to performantly fetch any data he or she might need for any given page.

This query technology may be quite handy if you must write intense, focused queries. If cutting a couple text-based columns makes that much difference, then REST may not be solution you seek. (Of course, at that point why not just have your JavaScript frontend open a SQL/MongoDB/Neo4j connection?)

What does REST solve? REST solves the brittle problem that arose with CORBA and SOAP.

REST makes it possible to evolve APIs without forcing you to update every client at once.

Think about that. When web sites make updates, does the web browser require an update? And why?

It’s no light feat of accomplishment. People were being beaten up left right as APIs would evolve. Updates were tough. Some clients would get broken. And availability is key for web scale business. So adopting the tactics that made the web resilient into API design sounds like a keen idea to try.

Too bad not enough people actually KNOW what these concepts are, and press on to criticize REST while offering “fixes” that don’t even address its fundamentals. The solution served in the article would put strong domain knowledge into the client, resulting in tight coupling. REST doesn’t shoot for this.

Am I making this assessment up?

This “virtual graph” is more explicitly expressed as a schema. A schema is a collection of types, interfaces, enums, and unions that make up your API’s data model. GraphQL even includes a convenient schema language that we can use to define our API.

Agreed. Continuing with more tight coupling instead of letting server side logic remain server side would align with SOAP 2.0, in my opinion. And it’s something I don’t much care for.

To dig in a little more about how REST makes it possible to evolve APIs with minimal impact, wait for my next article, The Power of REST – Part 2.

Something Java should never do

I’ve been working on a pull request on Spring HATEOAS for six weeks. That’s right, six weeks. It was a community contribution, and there’s a lot to sift through. Yesterday morning, I was slated to conduct the first review with the project lead. But that wasn’t going to happen, because Java decided to do something Java should never do.

Maven wouldn’t build my project while IntelliJ IDEA would.

I was alerted to this issue Thursday morning when I created a “run all profiles” script and it wouldn’t work. Huh? Every time I’ve pushed an updated commit to Travis CI, it builds perfectly. That’s right, a CI server, running Maven, built my branch with ease. But my Mac would not.

This made no sense. This is something Java should never do! Java doesn’t work on Linux but fail on a Mac. It must be my machine, right? So I log onto my wife’s MacBook Air, update Java to the latest version, grab the source, and fire off Maven. BOOM! It breaks just the same.

So…here comes the online review with Ollie. I pronounce my inability to build the system. The man with twenty years professional experience, who has written four tech books and one tech video, who has been fighting this for twenty four hours, pleads with his manager for clues. Ollie grabs the branch, tries to build, and SPLAT. It doesn’t work. He goes through the same questions I do, and finds no answers.

We jump to our OTHER CI server, Bamboo, and he commands it to build my branch. It works. Perfectly.

Score so far: Three Macs – 0  Two Linux CI servers – 2

“Ollie, this is something Java should never do!” I scream into my Google Hangout. (Okay, maybe I didn’t scream.)

He nods along. The review is busted. We both flip to our daily standup Hangout, and Ollie’s first words are, “Greg, I know what you did.” Heh.

We wrap that meeting up, and I proceed to dig through every Stackoverflow article I can find on Lombok, maven-compiler-plugin, and Java 6. Talk about a nebulous combination. WIth enough evidence that this is Lombok’s fault, I open a ticket.

Fun day, given it’s my son’s last day of school for the year. I’m irritated at having to stop work to fetch him. My brain keeps churning the whole time. The second I get home, I have him go play. Anything to get back to my keyboard and keep trying options. I flip through a myriad of options for maven-compiler-plugin: memory, forked JDK, use Java 8, hand configure annotation processors. Nothing works.

My three-year-old is trying to climb up on me, and I push him to the side. “I must find the answer!” Time to pick up our 7-year-old arrives, and I dash off. After getting home, she is asking me to write LOGO code. “No sweetheart, I need to try something first.” I usually stop ANYTHING to help her, but I was too obsessed with this issue.

At this point, I start to De-lombok the code. Maybe if I can flip Lombok off. I get 90% of the code converted, and IT STILL WON’T BUILD! What could this possibly be? Every debug/verbose flag I activate still shows me nothing about what is actually breaking. Just some missing symbols followed a completely corrupted class.

Then I get a strange idea. A curious insight. What if the first class that Maven claims about, a nested static class, was made top level? I command IntelliJ to perform this refactoring. Poof, the error message changes. So I make the next nested static class in the list of errors top level as well.

BANG! Reading “[INFO] BUILD SUCCESS” makes my jaw drop.

I cancel all my changes, and build again. Same failure. I then strategically pick one nest static class, and move it. Everything suddenly works. What the…? This is something Java should never do.

Looking at the Jackson2HalModule (the predecessor to my work on Jackson2HalFormsModule), I count the number of nested static classes and compare to this one. The HAL parser has eight or nine, the new work has over twelve. Somehow, this enclosing class has TOO MANY NESTED STATIC CLASSES!

Not wanting to move everything into public classes, I rig up HalFormsSerializers and HalFormsDeserializers, and split up the nested static classes into these two “namespaces”. Everything is humming. This change is mind boggling. Because this is something Java should never do.

So what is happening? Still not sure, but it implies that Lombok’s annotation processor must scan every class looking for its annotations. If the class is too deep, the scanner breaks and doesn’t recover, hence causing any nested classes further on to fail. And somehow, this state is contingent on the platform, because it works jim dandy on Linux while failing on the Mac.

I copy all these details into the Lombok ticket I opened. I hope I shined a light on an issue perhaps the committers can pinpoint and fix. But for now, my PR is buffed up and reviewable again. And as I go to bed, I can finally sleep in peace, having slain another dragon in the realm of open source hacking.

 

Marketing your book with a launch group

I’ve recently been working on building up a new aspect of marketing – forming a launch group. A launch group is a close knit circle of people that help you get the word out when your book, as they say, hits the stands.

I have two fronts in progress: Darklight and Learning Spring Boot 2nd Edition.

One is my speculative fiction title that I’ve sent off to a tentative publisher. The other is my latest tech book slated to go out this September.

Getting the word out

When it comes to marketing, it’s key to get the word out. In fact, it’s so important, that I learned of an author that used Russian pirate book sites to actually distribute his novel. What had been maybe 1000 downloads per year, turned into one million downloads a year, resulting in him carrying #1 ranking on Amazon.

(I’m not ready to pull the lever on THAT strategy yet. But if you can write a fleet of books, it’s a path to consider.)

When it comes to writing, people have a rather dated concept of marketing. Perhaps some of us can remember an author visiting our hometown. We’d load everyone up in the car and drive to the bookstore to get in line, buy a copy, and have the author sign it. Sorry, but those days are long gone. Book signings draw such little attendance these days, that authors tend to lose money carrying them out.

If you’re writing a book, tech or not, you may assume your publisher will advertise and publicize it. Nein! The only marketing dollars that are getting spent are on authors that don’t need it. In other words, unless your name is Stephen King or Lee Child, publishers aren’t spending two nickels on you. You (an unestablished author) are flat out not worth the risk of that investment.

Marketing is up to the author. Each of us has to get the word out on our works. And one of those avenues can be putting together a collection of people to help share it in various circles.

Launch group, assemble!

As I said, I’m putting together two core teams of people. For each team, I have a super secret Facebook group where I share in progress work, advanced copies ahead of public release, and will host contests from time-to-time when I need help with things like character names, plot tips, and more. On the day of release, I turn to my inner circle to help blog/tweet/facebook about the title.

If you’re interested in being a part of The Undergrounders and fighting the forces that oppose Darklight‘s release, you can sign up right here.

If you’re interesting in being a part of The Turnquist Techies and helping spread the word of Spring Boot, you can sign up for that group right here.

The myth of polymorphism

I remember reading about polymorphism for the first time. I was in high school, and boy it sure looked cool! Too bad I didn’t realize that the myth of polymorphism was a bunch of poppy cock.

You see, polymorphism never seems to be presented in its real state. Instead, we get this goofy, toy-app type presentation. Does Shape -> Rectangle -> Square ring a bell?

The fallacy of geometrical shapes being polymorphic

One of the simplest ways people seem to introduce polymorphism is using geometric shapes. We all know that a Square is a Rectangle. We covered that nicely in geometry, right? Problem is, geometry doesn’t equal software.

When discussing things in light of geometry, the reason we value this relationship is because we are looking at things like angles, parallelism, vertices, and intersections. Hence, squares carry all the attributes of rectangles. They simply have the same width and height.

But software isn’t geometry. The things we construct we must also interact with. The shapes must afford us operations to grab them, interrogate them, draw them, and manipulate them. A rectangle has two attributes: width and height. A square has one: width.

If we grabbed a square, set its width, then set its height, the assumptions of what would happen is unclear. Should a square morph into a rectangle? Or should setting the height induce the side effect of also updating the width? Either way is bad form. Hence, its best to break apart this faulty geometric relationship and realize that squares are NOT rectangles.

The fallacy of inheriting behavior

So shaking off the trivial example of geometric shapes, another common example is to talk about the glorious ability to reuse code. With polymorphism, it will be SO easy to code something once, and then reuse it across dozens if not hundreds of classes! And with late binding options, gobs of 3rd party libraries can go forth and reuse your code.

The problem is, no programming language has adequately been invented to gather ALL the semantic nuances of code. As more and more classes extend a given class, they all either realize EXACTLY what the abstract parent class does and agree with it, or they discover some new wrinkle not quite handled. The API may be supported, but some underlying assumption is buried that requires an update.

As the tail of inheritance grows, maintainers are less likely to accept new changes to the shared code. The risk of breaking someone else grows, because everyone knows not the ENTIRE nature of the code can be captured.

Some of the avenues to remedy this involves opening up the API a bit more. Perhaps a private utility method is needed by a new extender. But opening it up introduces more maintenance down the road. Or more opportunities for others to abuse things that used to be kept tightly controlled.

History has proven that composition beats inheritance for sustainability. Raise your hand if maintenance, not new development, doesn’t encompass much of your work.

The alternative are more sophisticated languages where you can capture more of the concepts. Yet these languages come across as too complex to many, arguably because CAPTURING all semantics is inherently challenging. And more often than not, we don’t KNOW all these semantics on the first round.

The myth of polymorphism vs. the reality

One thing that has emerged is programming to interfaces. Interfaces provide a nice contract to work against. Naturally, we can’t capture everything. But at least every “Shape” can  institute the defined methods. In Java, interfaces can be combined, allowing multiple behaviors to be pulled together.

So when it comes to abstract and refactor, think about extracting interfaces when possible and delegating solutions.

Strangely enough, despite my consternation with static variables, I’ve come to appreciate static methods in the right scenario. Factories that support interfaces can be quite handy dandy. But a well modeled, inheritance hierarchy is to hard to accomplish. If possible, try to avoid multiple layers, and see if it’s not possible to extract share code into various utility classes.

And when the entire hierarchy is nicely contained INSIDE your framework, and not as an extension point to consumers, the fallacies of polymorphism can be contained. But watch out! It takes much effort to put this together. Never think it will be easy.

Of course, I could be entirely wrong. If you have some keen examples, please feel free to share!

Happy coding.

Why take a break?

Good developers take breaks. But why? Last night, I attended a men’s session where two very different people talked on stage about very different breaks. The first was a pastor who was granted a six month sabbatical. The second was someone that was fired, and instead of lunging for the next job, spent time evaluating things carefully. As I listened, it made me reflect upon my own discoveries about taking breaks.

Why do we need breaks?

To be honest, it’s quite easy to burn yourself out. Maybe when you’re a bit wet behind the ears and single, not so much. I once worked extensive hours (despite not getting paid overtime), and I don’t regret that.

But today I can’t do the same thing. I have three kids and many other responsibilities that preclude me from working ALL THE TIME. Well, a big difference between now and then, is that back then, I needed a lot of “tinkering” to figure things out. I was short on experience, but long on drive. And working extra hours helped me learn a lot of valuable lessons.

Today, the valuable lessons are taking breaks, and letting my subconscious continue chugging away. Even today, I’m working on a major review/refactor/rebase task that has already cost me two weeks and will probably cost a couple more. Yet I don’t dread this effort. In fact, I’m not trying to rush it. Because the hours I’m away from my laptop (like now), my mind is still noodling things out. And when my fingers rejoin the keyboard, I feel like my knowledge of this branch is growing at a comfortable pace. If I tried burning ten hours a day, I’d burn out and torch it all.

But by letting my family pull me away, and focus on other important things, I am stopped from burning out. And I can avoid spinning my wheels in the mud of frustration. In the past five years, I feel as if my productivity has actually increased, because I am working smarter not harder.

What good do breaks do?

Let’s just jump right in and state something critical – your work isn’t everything. When you pass from this world, people are less likely to remember the code you wrote, and instead remember the impact you made on your children, your spouse, and your community.

I remember a graphic that came out of Microsoft showing how you could access every aspect of your job from everywhere – a kids soccer game, waiting for an appointment, during breakfast, etc. 37Signals released a mocked up variant with the same style, only they dubbed it Work Can Wait (which you can see linked her). The thing basically stresses the importance of having a life.

But in addition to balancing work and life, I’ve discovered that my work actually flourishes when I get away from it. I once sat in on a talk by Dr. Venkat Subramaniam, who described that classic software eureka moment of realizing the answer to your problem you battled all day. The next morning, in the shower. I’ve experienced that a fair number of times, moreso since I started taking more breaks.

I enjoy, on occasion, merging a Github pull request after hours. But I don’t go out of my way to do that. It’s usually something small, like a typo in a guide. I enjoy being able to shuffle that off my plate in a bit of idleness. But pulling myself AWAY from the keyboard sometimes forces my mind to assess what just happened, and what’s coming next. Seeing that I have less time to spend, my brain hammers out certain details to make better usage of our next keyboard session. And I sincerely believe I code with higher quality because of this.

Try it. You might be surprised.

I’ll close by sharing a reflection I received from Mark Fisher, founding member of the Spring team. He once mentioned that if he had been forced to work nose to the grindstone, and hadn’t taken all the various walks, and simple “think about it” moments, Spring Integration may never have happened. Not everything is a burn-baby-burn coding moment.

Happy coding!

Fun @SpringData team meeting in Deutschland, storified from a bunch of tweets

I had a great team meeting last with my five other co-workers. I wanted to share the majority of it purely in tweets, with a couple comments thrown in here and there. Enjoy!

As my cohorts arrive by other means:

And this draws concern from some!

https://twitter.com/Hameiste/status/849263376375140352

https://twitter.com/olivergierke/status/849490414835044352

https://twitter.com/olivergierke/status/849611590047604737

https://twitter.com/olivergierke/status/849868582901121024

https://twitter.com/olivergierke/status/849933811374796801

That evening, we trekked over to the Berlin-Brandenburg JUG meeting where Christoph and Mark talked about Reactive Spring Data, and I managed to give away a copy of Learning Spring Boot:

Finally glomming onto the fact that Germany dubs the “ground floor” as the 0 floor, I really appreciated being able to do pointer arithmetic based on my floor number!

A nice little event in the midst of our team meeting:

https://twitter.com/gregturn/status/850813991030898688

https://twitter.com/gregturn/status/851059503944540160

https://twitter.com/gregturn/status/851190002872717312

And for those of you asking, no I have yet to discover the location of my luggage. ¯\_(ツ)_/¯

UPDATE: The airline found my bag and returned it yesterday. Eventual consistency! 😄👏

Guten Tag Deutschland. Ich bin da!

I just arrived in Germany for our big Spring Data summit. Our team is having a get  together to do some planning and scoping of work for the next year. And I couldn’t be more excited.

I arrived at 7:00 am local time, and waited two and half hours to discover the airline had lost my bag. But pay it no mind. I packed the critical in my carry on so with paperwork filed, it’s no sweat. The fun part (so far) was having the leverage my German to interact with my taxi driver, who spoke hardly any English.

Something i was able to contemplate while traveling, is that one of the biggest things I missed while working on Spinnaker was the jürgenized code base of Spring. The clean, elegant nature of all the portfolio projects really gives me a warm fuzzy.

Its hard to explain if you haven’t seen clean code before. But having a strongly enforced clean, consistent coding policy eases the mind. When we write code, we need to understand it. Sifting through inconsistent spacing, sloppy names, tabs AND spaces, and other aspects of code slows down the brain, making it hard to digest the functional aspects.

My father-in-law once shared that driving while it’s raining tires you out faster. He once had to drive at night through a lot of rain and it wore him out. The same can be said for sifting through sloppy, inconsistent code.

So this week I celebrate being back amongst my Spring Data mateys as he continue hammering out what has to be one of my favorite umbrella projects. Applying the concepts of the Spring programming model to Data management.

Cheers!

API evolvability is key in a system of systems

Heh. That tweet was a follow up to Oliver’s latest presentation on the topic “API evolvability is key in the system of systems”. So what does this stuff mean, and why is it important?

API evolvability is one of the core underpinnings we get when we use the constraints laid out in Roy Fielding’s dissertation. And what is that exactly? In essence, the web grew FAST and is a major success. Many of the reasons are that is constrains us in certain ways, thus forcing certain things forward. Web sites are easy to evolve WITHOUT evolving the web browser. The value of this cannot be stated enough.

For decades, developers have faced the issue of evolving APIs. The concept of REST is take some of the underpinnings of the web and take them into API-land. Evolving APIs with similar constraints should make it possible to scale such changes and avoid some of the brittle, backwards-compatible breaks we have suffered for years, which can lead to slow downs in evolution.

How does this tie in with Spring?

I recently moved back to the Spring Data team. I had been on that team before, working on Spring Data REST. A key library that supports it is Spring HATEOAS. In Spring HATEOAS, we have an API meant to build hypermedia-powered RESTful APIs. And due to the incredible popularity of ALL Spring Data projects, Spring HATEOAS hasn’t received enough love.

No more!

Since jumping on this fab team, I have started putting together a list of proposed updates, enhancements, and polishings. Next week, I’ll be in Germany with the rest of the Spring Data team having a coordinated meeting. Among the many topics, Oliver and I will be hammering out this plan to propel Spring HATEOAS forward to be benefit of the community as well as the Spring Data REST community.

And one thing that Oliver has invested much time in speaking at various conferences about DDD, REST, and API evolvability. I have a strong desire to create some sort of code repository, a playground if you will, where people can checkout things and see how to evolve an RESTful API using Spring HATEOAS and the concepts of REST.

That way, we aren’t just preaching a high falutin’ concept, but showing how it can be done.

And I also imagine that in putting together such a repository, we can also find the gaps and holes in Spring HATEOAS. I suppose it’s risky putting this all out in a blog post, but I just wanted to share some of the ideas floating around in my brain as we ramp up support for Spring HATEOAS.

Laptop on a work table with DIY and construction tools all around top view hobby and crafts concept

Learning Spring Boot 2nd Edition delayed – Find out how to get a FREE advanced copy

Having taken a week off after turning in the last chapter, I geared up and started reading the whole thing, top-to-bottom. Some bits were written before Spring Boot 2.0 was even available on Github, so there is much code that needs updating.

That’s when I got word from my published that due to Boot’s schedule, they are delaying the release. Didn’t say when. We haven’t hammered that out. But this makes me feel good. Encourages me to really polish things up and ensure I can get the right message out there.

In the meantime, I’m putting together my Launch Team.

  • get an advanced reader e-copy of my book
  • be a part of a secret Facebook page and get sneak peeks
  • learn more about my writing journey as it happens

Then the Turnquist Techies is for you!! I will send out FREE advanced reader e-copies to the members of my launch team. There will be a secret Facebook page and a private newsletter just for this team.

What do I ask?

I am encouraging the members of the team to participate in this journey with me by sharing their honest opinion in reviews, tweeting and posting about what they are reading, and the like. For the most part, being a part of the Turnquist Techies is about having fun, reading, connecting, and learning more about the technical writing process.

How do I sign up?

Just click on join The Turnquist Techies and fill out this form.

What do you do when you’re traveling to Germany in two weeks?

At Pivotal I work on the Spring Data team, and our fearless leader is having us all converge in Berlin in just two weeks.

Short of being über awesome, what do you do? Well, considering I’ve studied German off and on since high school, I thought it time to get back into my tools to freshen up my speaking.

For starters, need to use this cue card to remember when to say “bitte”! Heh, that’s a joke. You throw it out about every 3-4 sentences, because it means EVERYTHING.

Anki

But seriously, my favorite app is Anki. To call it a “flashcard app” is a gross understatement. This app uses “spaced repetition”, a concept going back at least to the 1930s from Professor C. A. Mace.

Spaced repetition takes a deck of cards, and as you answer each one, it polls you on whether you did/didn’t know it. Cards you knew are delayed as to when they show up again. Cards you didn’t are reviewed sooner.

There are all kinds of studies backing up the efficacy of such study techniques. In fact you can spend hundreds of dollars on language learning courses based on this. Or you can do what I did (cue dramatic music leading into next section.)

Duolingo

I learned of DuoLingo.com back in 2013, when one of my teammates started tweeting about learning Spanish. Digging in, I found a really cool website/iPhone app that had free, 5-minute lessons on many languages including German.

So I dug in!

It was nothing short of awesome. Each lesson is concentrated. I enjoyed how many verbs I could still conjugate even after being away from German for twenty years.

Then a major breakthrough – after learning of Anki, I discovered someone had built a deck out of DuoLingo and shared it with the community.

Slack + German

If that wasn’t enough, just this week someone launched a German JVM Slack group. Duh! Who wouldn’t sign up for that?

To wrap things up, I find myself roaming around doing chores or work, mumbling little phrases in German. I have written my high school German teacher. And I’ve heckled my teammates.

It’s going to be awesome!