Yes. I wrote a book. It hasn't been printed yet but now that I have sent it to my publisher I have time to kill. So I thought: Why not reflect on the last one and a half year I spent writing the book and share my experience. If you want to write a book you maybe you can get an idea of what the journey is like.
Here's the backstory; One day in September 2016 I got an email from someone claiming to be an assistant acquisitions editor at Manning publications. According to the email they were interested in publishing a book on data quality and wanted to know if I'd be interested in talking to them. My first thought was: "email scam", then after some thought I decided to reply which 1.5 year later ended in a complete book.
User oriented book writing
Manning publications has a really great approach to book writing. I can't compare it to other publisher because this is my first book and first foray in book writing (except for my Ph.D. thesis). Manning's approach is very user oriented and the whole process is designed to output a book that suits all kind of readers.
First thing I had to do after the initial call with Manning with was define my intended audience (data scientists) and what I wanted to teach that audience in my book. I scoped the book, weighed the difficulty of each chapter to see if the book really matched my audience. It was a kind of a requirement analysis for my book. As I worked on the book I fleshed out the lesson in each chapter, reweighed the chapters, and constantly aimed all my writing at the intended audience.
During the writing process the book was regularly reviewed by a focus group consisting of my intended audience. A group that gave me really valuable feedback and allowed me to better focus my book on what they found helpful and avoid any confusion or boring bits. This was often frustrating because I had to rethink, restructure and rewrite all of my chapters at least twice before I found the best approach to teach data quality without being limited by preconceptions of the intended audience.
Then once I had sent Manning a majority of the chapters my book went into something called Manning early access program or MEAP for short. This program publishes an early release of the book and allows the general public to provide feedback and help me further improve the book for all possible readers. As part of the MEAP Manning put up a book forum where I could converse with readers and adapt the book accordingly but I found private conversations with readers outside the forum most helpful.
Manning provided me with a development editor and a technical editor who helped me frame the book and keep me on track. My development editor was awesome and patient while I internalized Manning's learning system -- the way they have found out helps people learn best. That way I adapted my style to how my readers would read my book. I think this shift in how I write and structure my content was the most difficult thing to overcome. I had written a lot of text before, but unknowingly I wrote only for me and people like me. Obviously, I'm not everybody and Manning had a way to structure book to fit a wider audience.
For example I didn't like repeating explanations in each chapter but what I didn't realize is that while I read books from page 1 to the last page. Others fly into chapters in random order, based on what interests them. If I had never repeated things I said in previous chapters those readers would miss context and not get the same experience. So the trick was to write so that the book was approachable to both groups.
Given my experience in data quality and usability, which is all about understanding what users need in addition to software development which is also about understanding user requirements, it's almost embarrassing to admit how hard it was for me to realize this.
Balancing work, life, and writing
The second most difficult part of the book writing process was fitting the book writing into my life. I'm a father of two daughters. When I started writing the book my two daughters were 2 and 4 years old. My wife was finishing her Ph.D. and had recently started a full-time job. I had a full-time job as the head of IT and dissemination at Statistics Iceland. We had also planned our summer vacation trip in 2017. That plus all of my other interests had to be balanced somehow in addition to all the crazy things I decided to do also, like buying a new apartment and moving, facing a hard disk breakdown and restoring from backup to a new laptop, participating in the parent association of my daughters' preschool, undertaking a major strategy shift at work and whatnot. In retrospect I don't know what I was thinking.
So how did I write the book? First thing: You know how authors tend to thank their family and spouses for the support. I see that in a new light. It's not something they do out of courtesy to their spouses. If you have a family you need to have a supporting spouse. We had to plan how to split the weekends, who got to work in the morning and who got to work in the afternoon (as I said my wife was finishing her Ph.D. on the side). Our family life suffered. We often discussed how awful it was not to see each others for days on end. If one of us was picking up the daughters from preschool, cooking, reading to them, and getting them to bed; the other one was sitting in a coffee shop typing on a laptop.
Coffee shops were great. They became home away from home. A place where I could focus. If I was at home I was easily disturbed which caused me to lose focus. In coffee shops I was rarely disturbed and kept focus for hours. I especially like one coffee shop that was opened until late so I was dose up on coffee during the day and shift into beer in the evening. I wrote the book in a lot of different places that kept my me constantly interested while revising the book.
The writing rhythm I found most suitable was to spend a week or two to research further the chapter I was going to write; find interesting stories, dig into literature, material, create example programs for the chapter and other preparation work. Then take a weekend to bash out surrounding words. I then sent the first rough draft to my awesome development editor who read it and helped me finalize the chapter for user review with lots of back and forth editing and comments. This rhythm didn't always work -- some chapters were more difficult to plan and write than others so they needed more preparation -- but that's the setup that worked best for me and my family.
Git statistics
I wrote all of my book in Asciidoc, a typesetting text-based format to write books. I used Git to version control the book and pushed it to a Gitlab server hosted by Manning. Looking at the Git statistics and Gitlab charts reveals how I wrote the book.
Looking at the whole git repository I changed 1208 files, inserted 113137 lines and deleted 42037 lines (a line is is no more than 80 characters). To put those insertions and deletions into into context, the main chapters of the book are in total 9347 lines (70077 words). Some of the insertions and deletions are things like svg diagrams I created for the book and kept in Git, but it's still pretty clear that a lot of the work was rewriting, deleting, and adding text.
The setup of writing most of the content during the weekend is clear if we look at commits broken up by weekday. Majority of the 157 commits (in total) I made on Sundays (almost 40). Those were the big text blocks I hammered out after preparing for a week or longer. The Thursday commits are made up of two types of commits, when I finalized code for the chapters before the weekends or the chapter structure and when I made changes after getting edits from my editor.
Looking at the commit hours it's also pretty clear that I'm a family man working a full-time job (my timezone is UTC+0 so those are my working hours in the graph below). I seem to work best in the evenings. I start committing work around 4PM and continue until 2AM in the night. Then I sleep for a few hours and head into work. The small amount of commits around noon are from when I took the morning shift on weekends.
Was it worth it?
Oh yes it was. Even if I worked late, after my family went to sleep, sacrificing extra curricular activities I learned so lot, not only about how to teach and write, but also about my area of expertise. I had to come up with creative solutions to make my examples simple enough to be useful in a book.
I'm very happy with the outcome. The book is still in MEAP (now when I write these words) so it hasn't been printed yet but all the chapters have been published and it's going through the last review. If you're interested, you can buy it -- The Art of Data Usability on Manning's website (ebook available now, printed book/pbook once it's been published).
If I was able to do this with everything that was going on in my life at the same time, you can too! So if you have an idea for a book somewhere in your stomach there's nothing stopping you from just starting (although I do recommend that you have fewer things going on in your life than I had -- you're more likely to keep your sanity that way).
I hope this post is inspiring, even if it was mostly about me. I'm more than happy to answer any questions you may have in the comments to help you and encourage you to get your book written.
Top comments (5)
Congratulations!
I've a friend who has published many books. In the field of technical books, it is truly a labor of love. I don't think there are any rags-to-riches J.K.Rowling stories in the technical book genre.
I appreciate that you've detailed out your process and tools, and how it impacted your personal and family life. (What editor did you use? Vim? Emacs? BBEdit? UltraEdit? Atom? TextMate? Brackets? Sublime? Other...?)
Sorry that your hard drive took a dive. I've just had that same experience last month -- fortunately I was able to recover all but two files. Thanks to Alsoft's DiskWarrior... a wonder tool. I still get anxiety attacks from the recent harrowing experience.
Thank you for sharing this, and educating me on what authors mean by thanking their loved ones in the dedication.
Thanks for the kind reply.
I don't expect this to be a rags-to-riches story. I'm not in this for the money, I'm in this for the experience. For the love as you say.
The crash of my old laptop's hard drive was an great example of dogfooding. My book is all about understanding what the needs of data are and identifying quality attributes you want to be sure you deliver as expected. One of those attributes is recoverability -- being able to recover your data in the event of data loss. So being able to recover all my stuff through backups struck really close to home (and I was able to recover it using the exact same backup software I use in an example in the book).
I used two editors. I'm an Emacs user so all my code and extras were done in Emacs. However because of a built-in Asciidoc preview extension for Atom, I decided to write the book in Atom. So Atom for writing, Emacs for code.
What happened to the book, btw? Looks like it's not getting released with Manning. Will it be available through some other publisher?
You're right. In the end it didn't seem to be a good fit with Manning's readers. The rights have turned back to me.
In a way that's good because then I can go in the direction I originally wanted to go with the book, but it's also bad because now I have to intrinsically find the motivation to modify it and find a different publisher.
Do you still have a draft copy of the book, I would like to read the original copy.