[email protected] – The Ultimate #NoSQL Database
Over the past year, I have dove head first into use Scrivener for my writing efforts. The thing is amazing!
Scrivener is a writer’s tool, built by writers for writers. It costs about $30+. I couldn’t put my finger on what was so cool about it until I read this.
tl;dr – Scrivener puts your story/case/project into a database, Microsoft Word puts what you’re doing into a typesetter. Typesetters optimize for printout, databases optimize for reading/writing/updating information.
NoSQL data stores have gained big popularity over the past ten years. Why? Their charm is being schema-less.
schema-less – data not required to adhere to a certain structure
For years, people have adhered to SQL, the codified and accepted standard for grouping data to a strong structure. SQL comes loaded with lots of promises, which it indeed delivers. What is that?
If you define the structure of your data upfront, and observe other related practices (like 3rd Normal Form, i.e. 3NF; ACID, …), your data store will…
- have maximum efficiency in storage by not duplicating data
- have maximum efficiency in maintenance by not accidentally updating data in one place but forgetting to update in another place
- get ALL the results when you query the database
- ensure ALL inputs are committed to the database or none
These sound great, until you reach the era we have entered. People have discovered that all the guarantees of schema-driven data have costs. And costs that are proportional to your volume of data can catch up and cripple you.
We have discovered that not ALL data needs this amount of guarantee. Different data stores optimize in other ways, solving different problems. And thus was born the schema-less data store revolution.
Scrivener as a NoSQL data store
How does Scrivener work? Out-of-the-box, it has a hierarchical nature. You can create folders within folders with folders. Each folder can have metadata about the folder itself, and it can contain leaves as well.
Click on a folder and you can view/edit all its leaves at once. Click on a leaf, view and edit a single leaf.
Folders and leaves can be converted from one to the other. The only difference is that folders are also containers, able to hold more folders/leaves.
The content can be text (our primary medium as writers) or other types (PDF, images, videos, …), meaning folders don’t have to just contain your story. Use it to capture your research, character notes, whatever!
Breaking out of the box
When you first install Scrivener, it comes with a pre-written manuscript and a tutorial. You walk through it, learning how to use the tool. It’s really quite clever and brought me up to speed, fast!
But at some point, you need to break out of conventions and learn how to use the tool. I first did so when I needed to sift through an extensive critique from my editor.
In the span of a 2-hour phone call, I had written down two pages of notes in a LibreOffice document last year. Some points, high level; some points, specific to a single sentence. I imported that document into Scrivener and took it apart, using another structure.
I split up the collection of notes into individual leaves, all contained in a single folder. This way, as I addressed each comment, I could flag it as complete inside scrivener (I used checkbox icon to indicate this).
I put the scene-specific notes in a sub-sub-folder. To tackle the fact my editor had a different version of my manuscript, I dug that copy out of email and put in there as well. Using that, I tracked down every page specific comment and found its current scene.
Scrivener lets you put links between scenes (kind of like a MongoDB DBRef).
In a nutshell, I laid out my own structure, and then bent it as needed. Instead of bumping into it, like one often does with schema-based data stores, Scrivener accommodated my needs.
Spring Data Scrivener?
As a member of the Spring Data team, I’m truly amazed at how this release train of projects has leaped over the balkanized landscape of query languages. Use them when needed, but offloading typical queries to a framework is great!
I may have to keep my eye on the potential for writing apps that can query Scrivener manuscripts. It would lean on exactly what people are putting in their projects.
Until then, I hope you poke your nose into Scrivener and see how it’s perhaps the most user friendly NoSQL data store put out there to solve a very popular problem.