Category Archives: pythontestingcookbook

Valuable things I have learned while writing three books for @PacktPub

I recently had a friend of mine ask me about the viability of writing a book for his open source project. He had questions about how to submit a proposal as well as the money involved. I wrote him a detailed response, but decided to post some of my lessons learned here.

Don’t write a book for the money

This is definitely a worn out expression. I have seen people get this starry gleam in their eye, Sorry, but that only happens when you write a best selling, fictional trade novel. Not everyone can be J.K. Rowling. In fact, there are actually a very small number of authors who make it big. The real name of the game in writing is to write LOTS of books, such that over time, people that like your material end up buying all your titles.

When writing technical books, you still need to sell a bunch to really make any money, but we are forever cursed by books going out-of-date in just a few years. Trade novels last a lot longer.

That may sound a tad depressing. So why write book? Because it can open doors you never knew would happen. When you are a published author, it can impact interviews, speaking engagements, consulting opportunities, and other things. It is always a major +1 to your CV to have written a title. Even more if you get several under your belt. It also increases your overall name recognition and personal brand.

Understand what you’re getting into

Before expending any effort in writing, I made sure we had a contract in place that we both agreed to. I’ve read article where people keep writing chapter after and yet have not signed an agreed upon contract. I don’t work that way. You are taking on risk by doing that. Until a contract is in place, the publisher can bring onboard other writers.

I have historically preferred to write my titles by myself. It lets me control the content 100% and not have to deal with interacting with another author, one who might/might not be as motivated as me. As a nice side effect, it also ensures that I get all the money. Since we’re already talking about a small pile of money, it feels counter production to split it up amongst two or three people.

Packt’s contracts include an option of first refusal for your next two titles. I understand Packt’s desire to get first dibbs on future work. It make business sense, so I had no issue with this in my first two titles. But on the third one, I had decided they had made enough money on me. I asked that this clause be removed to liberate me for future works. I have no hard feelings nor have I felt any malice from anyone at Packt.

Be sure to understand that the contract includes deadlines and exactly how much money you’ll be paid and when. If you sign the contract, you are agreeing to everything stated. Understand what you are agreeing to.

It’s your title. Act accordingly

Every publisher out there will probably tell you how glorious their marketing and sales team is. But for you to write a book and make no mention on Twitter/FaceBook/Google+, nor to write a peep on your blog site is ridiculous. You should be blogging all the time as you write and beyond publishing. Don’t assume that your publisher will automatically sell 1000 copies of your book and be fired up to order another batch!

Another aspect of “acting accordingly” is to remember this will reflect first and foremost on YOU. If you have a certain voice or a style of presentation when you speak at conferences, then by all means, try to write that into your manuscript. Your editors may try to edit that out. If you give them the upper hand, they can reduce your effuse writing into boring humdrum. I’ve seen editors edit out some of my sassy writing. My response? I edit it back in! They don’t usually push back too hard.

It’s important that the publisher have a certain level of quality. But there needs to be enough of YOU in whatever you write that makes it stand out.

Go and read other people’s technical books. Part of this inspiration for me came from a couple other technical books I read. The style these author’s wrote knocked me out of my chair. I LOVED the small jokes, clever class names, and funny material they injected into their writing. And it told me I could do the same, as long as it wasn’t offensive or obnoxious. Again, my editors pushed back, but I paid keen attention to EVERY edit, and undid several of these. I wanted my book to be fun.

Making your book fun is no one’s job but yours.

Take credit for your work – give credit to others

In my latest title, I started each chapter with a quote from twitter. “Spring Boot” was so hot, that I started collecting tweets as I wrote. I would launch a chapter, and include a link to that user’s twitter handle. Their comments were inspirational. I hope they picked up new followers.

At the same time, I gave a presentation that pre-dated release of my book. I mocked up a logo of the book with the title in it, and included that in my profile slide at the beginning. This was a PR moment I wouldn’t pass up. I wanted everyone in that audience to know a book was coming, and hopefully they all bought a copy!

That same week, my colleagues and I trekked to a JUG meeting. Someone in the group asked, “Is there a book on Spring Boot coming?” Everyone turned to me, and I happily gave them the estimated release date. More free PR. I also was sure to cc more than a handful of tweets to that group’s twitter handle. Hopefully they each bought a copy! (Not likely, but one can dream, right?)

Finally, I wrote my last title using asciidoctor thanks to Dan Allen. This was by far the best option. It allowed me to focus on quality content and not idiotic Word Processor issues.

Knuckle down and write

Okay, all these suggestions up to this point are nice and all, but it’s all for naught unless you put in the hours to actually write your book. I read someone’s blog article where they thought the deadlines were crazy since they only planned to write on the weekends.

Sorry, but I don’t have much sympathy there. I wrote each of my books while we also had a newborn baby. I quickly adopted the style of writing every night after everyone had gone to bed. This way, I could focus with no distractions, and then take the weekend off. Writing every night from 10pm-midnight can be exhausting when you mix it with feeding a baby at 2 AM, but this ensured that I got my book finished and out the door. It also cut down on the risk of my technology moving away from me too quickly and making the material invalid, or avoiding massive rewrites due to mega-changes.

Another thing you have to be ready to do is carry your book all by yourself. You can’t depend on certain reviewers to fix things for you. Instead, you need to be ready to proof read in the event your book reviewers don’t actually give you any feedback until the last minute. It’s sad, but this still reflects on you.

There were times when I hit bugs in my sample code, and had to simply pack up for the night. There were other times when I had no choice but to work an extra two hours to hammer out a problem that no one else could help me with, either in my asciidoctor backend, or in my sample code. After all, no one was going to pick this up and fix it, right? Leaning on pull requests, etc. is no excuse for not writing your book.

I demoed Spring Social GItHub in my latest book, and had to go to press using version 1.0.0.BUILD-SNAPSHOT because the project lead was simply too backed up to fix the milestone release already published which broke my book’s sample code. Them’s the breaks.

So by now, if I haven’t smashed your dream of writing a technical book too badly, I wish you only the best in going out, submitting a proposal, and snagging a contract.

Finding your voice when writing a technical book

learning-spring-boot-ch3Something that I have been polishing for some time is my writing voice. The voice is more than just words, grammar, and speaking in a me-to-you, versus us-with-us style. It’s the way we blend words and language together to provide our writing with a certain personality. One of the things that won me over before I got past page one of Killing Floor, the first Jack Reacher novel, was the voice.

In that page, the dialog was 1st person, coming from Reacher himself. It was calm, clear, and crazy. The man was being arrested. Why? Unknown. And yet, Reacher didn’t tremble a bit. He had no fear. Again, why? I would panic! On top of that, Reacher calmly moves such that no one ELSE in the diner is put at risk. What?!? I had a clue who he was, because I had seen the movie. But subtracting that, you DON’T know who he is, and this type of uncharacteristic reaction to three cops with loaded firearms made me hunger to turn the page. Again and again and again. The voice grabbed me and held onto me through the whole book. And to the next book. (I’m not on #10).

Since I started writing this blog six years ago, my writing style has matured. I have slowly found the way I want people to share the thoughts I am putting down. I want it to be fun and not a cold lecture. I want to invite people in, and hunger to read my next post. And my next one after that.

When I wrote my first book, Spring Python 1.1, I was new to the idea of writing a book. I struggled with being concise. Boiling ideas and concepts into simple, well crafted sentences. I wanted the reader to enjoy what they were seeing and be able to think of ways they could use the code to meet their own needs. I didn’t think much about my voice, but instead more about the content.

When I embarked upon writing Python Testing Cookbook, I was much more confident. I had a big vision of what I wanted to tackle, how I wanted to tackle it, and what was important to me. I also had a better sense of time management and what it took to get things done. And I had a clearer picture of Packt’s shortfalls that I needed to watch to make sure the release was top

So as I began writing Learning Spring Boot, I had much more creative energy. I had even launched a separate financial blog two years ago and reached a rhythm of blogging two posts/week. To top it off, last summer, two other devs and I had been tapped to write over fifty getting started guides. I spent all summer writing technical stuff. We worked hard to find a common voice the three of us could share with the intention we would spread it to others to decentralize the writing in the long run.

All that writing I felt had improved my writing on several fronts. I had more self confidence to write in the voice that I chose. I had also honed my blogging voice. I was interested in the long run of people being able to read my book, and then visit my blog, and find some familiar writings. Or the other way around. And I had learned how to write for different audiences with different voices.

Here’s a sample from chapter nine of Python Testing Cookbook:

Testing isn’t a cold, mechanical process. It’s an exciting, fiery area of development. Test bitten developers can’t wait to share it with others. If you look for ways to spread the fire of automated testing, eventually others will warm up to it and you will find yourself talking about new testing techniques together. –Greg L. Turnquist, Python Testing Cookbook, p.330

If you read Scheduling Tasks amongst Spring’s Getting Started guides, you won’t find this same voice. You’re not supposed to. You’re supposed to find an entire collection of guides with a somewhat uniform style that show you how to solve common problems quickly. Making these guides consistent, uniform, and simple took an arduous amount of effort.

But if you switch to reading this blog, you’ll find a different style entirely. It’s more personal, because this is my personal blog site. I want to write enticing articles that you’ll enjoy.

And in Learning Spring Boot, here’s a sneak peek of some of my prose from Chapter Two: Quick Start with Java:

For production-ready support, it’s nice that Actuator provides us handy data we can consume. The scripts we used to consume these metrics might seem a bit crude. But in the real world, we need tools that let us quickly try something out before investing in bigger, more complex, and often more expensive solutions.

If our manager wanted a quick read on the last twenty four hours of data, we could easily take `metrics.groovy` and adjust it to have a rolling time limit. We could then serve up the content using a little Spring MVC. Cash in on Spring MVC’s WebSocket support, and we could have the screens update dynamically. Program managers love eye candy, right?

There’s no telling what requirements we’ll get from sysadmins and managers. Spring Boot’s easy-to-consume endpoints provide us the tools we need in both our main application and the support tools we’ll build up to manage things. –Greg L. Turnquist, Learning Spring Boot

I am receiving feedback from a couple of my colleagues as well as an editor. I have to sift through this material and decide what is an undeniable improvement (faulty words, typos, bad grammar) vs. stylistic changes. And when it comes to stylistic changes, I have to figure out which things will improve my story and which things impede my voice. I want to polish my voice and nothing helps better than honest, objective feedback from others.

So if I’ve held your attention this far,  I hope that perhaps you are also thinking of writing. I wish you the best of luck, and never to be afraid of asserting yourself to define your own voice.

What is the best testing tool?

Someone posted to me a question through meetupcom, “Greg, what is the best testing tool?” I didn’t have room to reply. I posted my response inside the Nashville JUG Google Group we host, but I thought today, it would be better to capture it here.

Asking “what is the best tool” with no other context sounds like you are looking at the situation from the wrong point of view. Let me explain.

A few years ago, I inherited a java app with hardly any automated tests, had been worked on by 6 engineers before me for a couple of years, had been demoed in front of program management, and was non operational. It did stuff, but not much and was loaded with errors. I adopted an approach where I wouldn’t work on any bug/issue until I could write an automated test to reproduce the issue. This was painful; more than you can realize. The software was tightly coupled, lots of static singletons, i.e. global variables, and hard to isolate. Basically, my first test involved using JUnit to empty some database tables, programmatically ingesting a spreadsheet, and then inspecting the database for results. This test took 5 minutes to run. It probably ran through a big chunk of code.

The problem was the system, and no magic test tool was the answer. Changes in the beginning were slow, but as I gained momentum, I eventually reached over 60% test coverage. This may sound low, but within 3 months, I made a flawless demo before program management. They were impressed that it worked, let alone with no problems. The tool was starting to get used by the intended audience. A year later, reported bugs were fixed in one day, and never regressed. I reached a point where I worked for a whole day, and when the half the unit tests failed, I threw away all changes and went home for the day, depressed. I started fresh the next day and actually fixed the problem. Two days to fix it! With no automated test suite, that would probably have been released, and incurred a gob of new bugs to deal with.

I met weekly with the users and captured new feature requests, problems, and generally made this tool work really well. The customers were very happy. I was also empowered to rewrite whole sections that were slapped together hastily in the past, because I had the security of my automated test suite. I threw away code that wasn’t used and didn’t work. My boss showed me a chart where the total lines of code became less than when I first inherited, yet it did more than those other 6 people could squeeze out of it. That was a happy day!

What I’m saying is that I could have used JUnit, TestNG, ScalaTest (works on java too), or any other suite of tools like acceptance testing frameworks, load testing, etc. But what test tool was used wasn’t important. Adopting a strategy of making the whole thing subject to testing, staying test focused no matter how painful it is, paid off. At some times, the test suite took 1.5 hours to run. I spent three days speeding up the most expensive parts of the system, and cutting out certain tests, getting the same number of tests to run in 30 minutes. I came up with a comprehensive test suite and a smaller, smoke test that ran much faster. I also created a spreadsheet to track number of tests and total test time, along with a graph. As the test time grew, I would periodically halt development and polish up certain parts that made it too hard to run tests. I would run the test suite at least once a day to make sure things worked right.

This whole development period was some of the best coding I had done in a long time. Cranking out top quality code with a warm fuzzy green bar made me grin ear-to-ear. When I left that company, I cried a bit because I wouldn’t be working on that tool anymore.

In the 9th chapter of my book, Python Testing Cookbook, the first recipe captures a lot of what I wrote up above, with some more detail. To quote the recipe “Something is better than nothing,”

“Just don’t let anyone tell you that you are wasting your time building a long-running test case. An automated test suite that takes an hour to run and is exercised at least once a day probably instills more confidence than clicking through the screens manually. Something is better than nothing.” —Python Testing Cookbook, page 326

When I wrote chapter 9, I wanted to move beyond simple coded recipes, but instead general lessons I had learned in the realm of testing. These principles work, whether you are writing Python, Java, Scala, or anything else.