How to create your own OpenOffice backend w/ @asciidoctor /cc @kubern @ysb33r

asciidoctorMy work on creating an asciidoctor backend for Packt Publishing (i.e. OpenOffice) seems to have caught several people’s attention. Today I spotted an exchange of tweets asking about OpenOffice in general. I figured a more detailed approach to doing that would better fit in a blog posting.

Starting with the basics

For starters, this whole thing is based on creating an FODT document, not an ODT one. The “F” stands for Flat and it is basically an XML document. Your mission, should you choose to accept it, is to create an XML document with all the right parts. That’s the reason my backend is written in SLIM.

SLIM is basically XML with every bit of unnecessary stuff removed. No closing tags. No angled brackets. Nested elements are denoted by indentation, hence giving it a more YAML-like feel. Checkout the following fragment for generating different section levels.

Working backwards

My recommendation is to start from a document you know and love. I started with Packt’s template they sent me. I performed a “Save As” and picked “Flat ODT” to get it into XML format. I then wrote a sample asciidoctor document with all the things I wanted. I suggest you start small and not get into complex layouts (yet). Remember: Minimum Viable Production (MVP) is the key to short, iterative success. From there, I started running my backend against this thing and comparing it with the outcome.

Got xmllint? If not, you should. Your output will be compressed XML. xmllint is the key to viewing your FODT output. Got giant monitor? .It can help dealing with stuff scrolling off the screen as you inspect your handiwork.

A key thing to understand. Asciidoctor splits up each “element” of asciidoc into a separate fragments and looks for the corresponding stuff in your backend. Got a “section”? Look for slim/packt/section.fodt.slim.

  • slim is the template engine
  • packt is the name of the backend (hence all your fragments will be there)
  • section.fodt.slim is convention-over-configuration for generating an FODT-styled chunk of XML everytime asciidoctor hits a “section”.

The top level thing for you document is, duh, document.fodt.slim. And this is also where you’ll find the root FODT XML <office:document>.  Way at the bottom is a common pattern, “=content”. This says, continue processing further asciidoc tokens and insert results at this level.

Iteration is the key to success

To run your backend against your asciidoc input:

asciidoctor -T /path/to/your/backend-root/slim -b packt your.adoc

My backend doesn’t cover every possible element of asciidoc because I didn’t need them. With asciidoctor, you can start with just document.fodt.slim & helpers.rb and when it hits an unknown token, it will tell in simple terms. Go and code <foo>.fodt.slim, and run again. Eventually, you can piece out your entire document. And in the process, you’ll become quite fluent with both FODT and asciidoctor.

Don’t give up!

You will undoubtedly hit some bumps. I sure did. One was getting my code fragments to apply a different style to the last line of any block of code. I probably invested two weeks of effort in this entire backend while writing Learning Spring Boot, but the payoff was enormous. If you are trying to convert a giant slew of documents, try to avoid after the fact edits. And if possible, get the core stuff working, like sections, paragraphs, bold/italics, bullets and admonitions. That stuff really motivated me to push forward.

Oh yeah…

Last time I checked, asciidoctor has special processing rules for double underscrores (__). Several of my style names from Packt had them, and it caused havoc. Once I tracked it down, I simply did a global replace, reducing them to single underscores. DON’T USE DOUBLE UNDERSCORES FOR ANYTHING!

Good luck!

Checking out editors

idea_sublime_atomIf there is something software developers do, it’s get real picky about tools for editing software. That’s why there are strong, almost religious debates about vi vs. emacs. I recently got fed up with TextMate for various reasons, so I decided to investigate more than my usage of vi.

I’ve included a screenshot of all three, side-by-side on my 39″ monitor. You guess which is which.

Sublime Text 3

This one I have found to be pretty neat. It’s fast, snappy, and opening up a project is a single command from my shell: subl /path/to/project

I quickly got moving when I learned there were only two key commands: Cmd-P and Cmd-Shift-P. The first is used to find files. You start typing fragments of a filename, and it quickly narrows your selection. Pick it, and it’s open in no time flat. That nicely matches Java’s one-class-per-file convention. Cmd-Shift-P is the Command Palette, which lists every available command. You then start typing fragments, and your list of commands shrinks. Then you can arrow-down and pick it, or hit the hotkey. Having a single way to find any command or file saves me from going to my trackpad and using the menus.

In the JavaDocs, HTML tags are nicely highlighted. Sometimes the color choices of Java syntax seem a tad awkward. But I rarely alter settings because I always fear either losing them, or going to someone else’s machine and not having them. Perhaps I might be able to stash them in Dropbox and soft link them in?

Another killer feature is the right-hand miniaturized layout of the entire file. Hover over it, and you quickly see where you are.

I’m still pretty new, so I haven’t learned how to open multiple tabs on the same file yet. Nonetheless, this tool is already productive for me!

Atom

After spending almost a whole day adding new asciidoctor content, my colleague Roy Clarkson hinted that I should check out Atom. I downloaded it last night and took a peek.

Nicely, Atom has the same keystroke to pull up the Command Palette. This feature sure must be popular. Right off the bat, I noticed the font size was a bit big. While it appealed to fuzzy vision, it was too impractical for the ground I needed to cover. Felt like I was peeking through a tube to see the code. So I pulled up the Command Palette and shrink the font size about five times so it approached the size of Sublime.

Opening the same file in the same project, I noticed that the color palette of syntax highlighting was a bit easier on my eyes. The contrast between bits of code didn’t seem wide enough, while Sublime’s contrast was a bit much.

I read an article pointing out how Atom is built on top of Chrome, JS, and Node, and conveniently comes with the JS toolset. Another article heavily criticized Atom’s shaky foundations which can start to magnify performance issues as you get into bigger projects. I’ve already seen a bit of this when trying to find files. Sublime just seems snappier.

IDEA

To compare things in full, I switched IDEA to it’s Darkula theme and opened the same file in the same project. A third syntax highlight color palette. Great. This dark-theme on IDEA is softer on the eyes, but some of the color clashes kind of turn me off. I’ve tweeted before that I don’t get it. Yet, at the same time, I use green-on-black for Terminal and totally dig the white/color-on-black of the other editors. Perhaps if I commit to a couple days, I might get into this?

What does IDEA have that the others don’t? IDEA is a full fledged IDE. Pick a function, then find everywhere that it’s used. Editors aren’t designed to do that.

This Command Palette from the editors has encouraged me to learn how to do that in IDEA. Apparently, Shift-Shift is the way to pull “Search Everywhere”. I am trying to do more of this. I have already learned how to find files and classes as well as pull up refactoring toolbars. These are all handy features. But to speed up the action, I need to jump to the command palette. From there, you can see commands as well as learn their hotkeys. More time on the keyboard is the ticket to efficient coding.

Results?

So far, IDEA isn’t going anywhere. I need this type of stuff. I just have to keep improving my efficiency at using it. As for an editor, Sublime is ahead of Atom right now. But like my bio says, I’m always looking for the right tool.

Mustering up the courage to watch “Doctor Strange” from 1978

Doctor_Strange_Vol_2_56You might not know this, but as a kid I collected comic books. My favorite was “Doctor Strange”. It makes it kind of neat because there aren’t a whole lot of books that he stars in. Suffice it to say, I probably own the bulk of those! True Big Bang Theory nerd showing through, ehh?

Well I have become excited since discovering that this rising era of super hero movies now has a movie coming next year starring the famous Stephen Strange. Doing some googling, I was shocked and surprised that CBS crafted a movie/pilot back in 1978 (with intentions to create a TV show). Shocked that someone went out on a limb for this back in 1978. And surprised that in all my collecting years, I had never heard of it.

Discovering that the entire movie is available is available for download from YouTube, I sought it out, loaded it up on my IPad, and watched it last night.

It took QUITE a bit of courage to sit through a movie that was only 90 minutes long, and yet, the first two-thirds have little magic or action. It was quite interesting to see Jessica Walter carry the bulk of acting talent in this show. She portrayed Morgan Le’Fay, Arthur’s half sister. In the comic books, she is somehow banished to the astral plane for eternity. In the movie, she is sent to crash the party.

I found it quite awkward that the reputed Ancient One, Earth’s previous Sorcerer Supreme, was portrayed by an Englishman who doesn’t show a lot of magic. I kept checking the cast on Wikipedia, only to discover that wizard was NOT the Ancient One, but some other dude they invented! It turns out, the Ancient One makes a vocal appearance right at the end.

Speaking of the end, this movie could cut out half of the opening, and STILL be sloooooowwwww. All the real action happens in the last ten minutes apart from Stephen rescuing Clea from “the higher planes”. Hearing the voice of “Kang” from Buck Rogers (Peter Ansara) and seeing the chick from “Twiki is Missing”, it seems like the cast of Buck Rogers in the 25th Century was making an appearance.

undergroundersAs a big fan of Doctor Strange, this comic book has served as one of my many sources of inspiration for Darklight, I’m glad that I unearthed this vintage creation. I would say it had a bigger budget than Doctor Who ever did (they had a horse in the movie, while K9 was a hunk of cardobard), and yet, couldn’t pull off the immortal power of that BBC-powered show. Yet, to know the entire genre of Doctor Strange, I felt it my duty to watch this snoozer.

To properly cleanse myself, I will have to turn back to the OTHER Doctor Strange movie I carry on my iPad: Doctor Strange (2007). That is all.

Doctor_Strange_video_cover

Announcing the winner of our #LearningSpringBoot contest…. /cc @PacktPub

Over the past month, we have had several entries. We have rallied together an international set of both contestants and judges. And here are the results!

The contestants

First of all, let’s list the applications that were submitted (in alphabetical order).

1. Mercury

This app is used for piping various types of traffic over different systems. It leverages Spring Boot’s conditional settings, allowing the user to customize as needed. It also has a hypermedia-based RESTful API.

2. Polaromatic

This application shows cute eye candy. It ingests files and manipulates them to look like “Polaroid” snaphots and dynamically displays them on the screen. It has a Spring Boot CLI script to fetch pictures from Flickr. It is Groovy all the way (all the way to the web templates). It uses websockets, Spring Integration, has nice Spock test cases, and

3. VotesApp

This application is a bot that listens in on chat sessions to count votes for/against spontaneous activities. That way, they can quickly get a tally of who all is going/not going on the fly. It has a pluggable architecture, comes with a video demo on an Android smartphone, uses Project Reactor, and leverages Spring’s profile support.

The results

Each of these apps was top notch. The contestants showed real skill. As judges, it was very hard because each team went in a different direction. Spring Boot offers a lot of concise and flexible power, and each of the entries used it well, just differently.

The winner **drum roll**

And the winner is…..Polaromatic! If you want to see, the winner has a running copy here. Visit it and simply wait a few seconds to see the outcome. The team behind Polaromatic has won a free copy of Learning Spring Boot.

Thanks for all who entered!

Valuable things I have learned while writing three books for @PacktPub

I recently had a friend of mine ask me about the viability of writing a book for his open source project. He had questions about how to submit a proposal as well as the money involved. I wrote him a detailed response, but decided to post some of my lessons learned here.

Don’t write a book for the money

This is definitely a worn out expression. I have seen people get this starry gleam in their eye, Sorry, but that only happens when you write a best selling, fictional trade novel. Not everyone can be J.K. Rowling. In fact, there are actually a very small number of authors who make it big. The real name of the game in writing is to write LOTS of books, such that over time, people that like your material end up buying all your titles.

When writing technical books, you still need to sell a bunch to really make any money, but we are forever cursed by books going out-of-date in just a few years. Trade novels last a lot longer.

That may sound a tad depressing. So why write book? Because it can open doors you never knew would happen. When you are a published author, it can impact interviews, speaking engagements, consulting opportunities, and other things. It is always a major +1 to your CV to have written a title. Even more if you get several under your belt. It also increases your overall name recognition and personal brand.

Understand what you’re getting into

Before expending any effort in writing, I made sure we had a contract in place that we both agreed to. I’ve read article where people keep writing chapter after and yet have not signed an agreed upon contract. I don’t work that way. You are taking on risk by doing that. Until a contract is in place, the publisher can bring onboard other writers.

I have historically preferred to write my titles by myself. It lets me control the content 100% and not have to deal with interacting with another author, one who might/might not be as motivated as me. As a nice side effect, it also ensures that I get all the money. Since we’re already talking about a small pile of money, it feels counter production to split it up amongst two or three people.

Packt’s contracts include an option of first refusal for your next two titles. I understand Packt’s desire to get first dibbs on future work. It make business sense, so I had no issue with this in my first two titles. But on the third one, I had decided they had made enough money on me. I asked that this clause be removed to liberate me for future works. I have no hard feelings nor have I felt any malice from anyone at Packt.

Be sure to understand that the contract includes deadlines and exactly how much money you’ll be paid and when. If you sign the contract, you are agreeing to everything stated. Understand what you are agreeing to.

It’s your title. Act accordingly

Every publisher out there will probably tell you how glorious their marketing and sales team is. But for you to write a book and make no mention on Twitter/FaceBook/Google+, nor to write a peep on your blog site is ridiculous. You should be blogging all the time as you write and beyond publishing. Don’t assume that your publisher will automatically sell 1000 copies of your book and be fired up to order another batch!

Another aspect of “acting accordingly” is to remember this will reflect first and foremost on YOU. If you have a certain voice or a style of presentation when you speak at conferences, then by all means, try to write that into your manuscript. Your editors may try to edit that out. If you give them the upper hand, they can reduce your effuse writing into boring humdrum. I’ve seen editors edit out some of my sassy writing. My response? I edit it back in! They don’t usually push back too hard.

It’s important that the publisher have a certain level of quality. But there needs to be enough of YOU in whatever you write that makes it stand out.

Go and read other people’s technical books. Part of this inspiration for me came from a couple other technical books I read. The style these author’s wrote knocked me out of my chair. I LOVED the small jokes, clever class names, and funny material they injected into their writing. And it told me I could do the same, as long as it wasn’t offensive or obnoxious. Again, my editors pushed back, but I paid keen attention to EVERY edit, and undid several of these. I wanted my book to be fun.

Making your book fun is no one’s job but yours.

Take credit for your work – give credit to others

In my latest title, I started each chapter with a quote from twitter. “Spring Boot” was so hot, that I started collecting tweets as I wrote. I would launch a chapter, and include a link to that user’s twitter handle. Their comments were inspirational. I hope they picked up new followers.

At the same time, I gave a presentation that pre-dated release of my book. I mocked up a logo of the book with the title in it, and included that in my profile slide at the beginning. This was a PR moment I wouldn’t pass up. I wanted everyone in that audience to know a book was coming, and hopefully they all bought a copy!

That same week, my colleagues and I trekked to a JUG meeting. Someone in the group asked, “Is there a book on Spring Boot coming?” Everyone turned to me, and I happily gave them the estimated release date. More free PR. I also was sure to cc more than a handful of tweets to that group’s twitter handle. Hopefully they each bought a copy! (Not likely, but one can dream, right?)

Finally, I wrote my last title using asciidoctor thanks to Dan Allen. This was by far the best option. It allowed me to focus on quality content and not idiotic Word Processor issues.

Knuckle down and write

Okay, all these suggestions up to this point are nice and all, but it’s all for naught unless you put in the hours to actually write your book. I read someone’s blog article where they thought the deadlines were crazy since they only planned to write on the weekends.

Sorry, but I don’t have much sympathy there. I wrote each of my books while we also had a newborn baby. I quickly adopted the style of writing every night after everyone had gone to bed. This way, I could focus with no distractions, and then take the weekend off. Writing every night from 10pm-midnight can be exhausting when you mix it with feeding a baby at 2 AM, but this ensured that I got my book finished and out the door. It also cut down on the risk of my technology moving away from me too quickly and making the material invalid, or avoiding massive rewrites due to mega-changes.

Another thing you have to be ready to do is carry your book all by yourself. You can’t depend on certain reviewers to fix things for you. Instead, you need to be ready to proof read in the event your book reviewers don’t actually give you any feedback until the last minute. It’s sad, but this still reflects on you.

There were times when I hit bugs in my sample code, and had to simply pack up for the night. There were other times when I had no choice but to work an extra two hours to hammer out a problem that no one else could help me with, either in my asciidoctor backend, or in my sample code. After all, no one was going to pick this up and fix it, right? Leaning on pull requests, etc. is no excuse for not writing your book.

I demoed Spring Social GItHub in my latest book, and had to go to press using version 1.0.0.BUILD-SNAPSHOT because the project lead was simply too backed up to fix the milestone release already published which broke my book’s sample code. Them’s the breaks.

So by now, if I haven’t smashed your dream of writing a technical book too badly, I wish you only the best in going out, submitting a proposal, and snagging a contract.

GitHub isn’t your CV, and GitHub commits don’t show your talent

I previously wrote about how GitHub isn’t your CV, and went on the even close my LinkedIn profile because of how little value it actually provided.

GitHub’s eye candy

Lately, I noticed people posting things like diagrams of burnout based on a dwindling number of commits to GItHub. Once again, it seems like GitHub likes to invent metrics and measures that seem to me little more than eye candy. Yet I fear managers and engineers using them as hiring metrics.

As my best example of such useless eye candy, I will point out myself. I am a self dubbed editor-at-large for Spring’s Getting Started Guides. The side effect is that if you go check out all the guides, you’ll find my name spread across almost all of them. I can probably claim 80%+ of the commits as well. It makes my GitHub profile look crazy with bursty rates of activity. For a full comparison, compare myself to my colleague Roy Clarkson. GitHub is showing that I have over 2600 commits for the pat year while Roy has 610. I fear that if Roy and I were somehow applying for the same position, someone might use that crazy metric to make a decision!

The truth is, Roy is a project leader for both Spring Mobile and Spring for Android. Roy has tackled countless problems, and being a one-man-band for two projects, has tackled an armada of issues with no one but himself. Passing on him due to a silly metric would be foolish.

The reason I have so many commits under my belt is that I wrote a tiny script that made it easy to roll the guides whenever a new version of Spring Boot comes out. It looks like this:

#!/usr/bin/env python

import os
import re
import sys

if len(sys.argv) < 2:
    print "Usage: %s " % sys.argv[0]
    sys.exit(1)

with open(sys.argv[1]) as input:
    with open(sys.argv[1] + ".new", "w") as output:
        content = input.readlines()

        for line in content:
            if "1.1.9.RELEASE" in line:
                output.write(re.sub("1.1.9.RELEASE", "1.1.10.RELEASE", line))
                pass
            else:
                output.write(line)

os.rename(sys.argv[1]+".new", sys.argv[1])

That 22-line app has saved me hours upon hours of effort and allowed our guides to stay up-to-date with the latest releases of Spring Boot. It’s the reason that when Phil tweets about a newly released version of Spring Boot, I usually tweet about the guides being updated within the hour. My teammates have come to assume my answer to a given problem is, “I can write a script for that.” And this app is nowhere to be found in my GitHub profile.

All code requires context to be properly evaluated

That is the kind of code and context I would carry into an interview. It shows that I know more than one language. I know that not everything is a Java application. It shows that I also understand the dividends paid in saving engineering time. And it shows that I understand the value of keeping documentation up-to-date.

The unfortunate side effect is that GitHub now shows me with MANY days of having bursts of over 50 commits. The thing is, if you were to hire me, there is no guarantee that I would be knocking out 50 commits on key software components. (Unless I write a script that starts doing document updates the same as I do now.)

Hiring others isn’t easy

I have actually hired people in the past. Long ago, when I received my first promotion, my manager explained that I had reached a level where interviewing candidates would be part of my annual evals. I took that on full steam. I interviewed dozens of people. I quickly learned that the one hidden question to ask yourself is, “would I want to work on the same project, side-by-side, with this person?” As can be expected, not everyone was perfect.

But nothing in a resume or any other publicly visible metric tipped me off about the best nor the worst people that worked directly for me. One person took over the mission critical app I had led for years. He had new ideas I would never have thought up. Another was a copy-and-paste developer that couldn’t seem to learn new lessons. He didn’t last six months. There was no way to know this short of actually working with them for a few months.

People, please don’t pick pre-fabricated metrics and eye candy as your criteria to hire and fire. Use real talent and judgment. And be ready to ask yourself, if this person doesn’t work out, what will be your course of action?

One week left for #LearningSpringBoot contest!

3021OS_mockupcover_normalIt’s not too late to join the Learning Spring Boot contest! Spring Boot has made it super simple for people to create new apps rapidly. You still have plenty of time!

I’ve seen a few tweets from contestants. Are you secretly working on an entry? Tweet your github repo, and I’ll DEFINITELY star it so you can score some extra points with the judges.

Just visit the contest site, fork it, create your app, and submit a pull request by January 17th 11:59pm CST (UTC-6).

I’ve already seen cool traffic a la twitter. I would also suggest posting links/announcements at https://www.facebook.com/springsource as well!

Good luck!