Leanpub, d3.js, scruffy graphs and predicting the future

One of the neat things about using computers is learning new tricks.

Mostly the tricks are not new to the World, but they're new to me, so it's like the excitement of invention with a small side order of resigned disappointment.

But this doesn't make the journey any less interesting, so the following is a tale of doin' stuff for no particular reason other than wanting to learn new tricks and findin' stuff out that you didn't know you wanted to know, but it's kinda interesting anyway.

I like Leanpub. Its a great idea run by a motivated team who are providing an excellent service. If you're thinking that you might want to write a book, check them out. They make the process easy.

I have a book on Leanpub called 'D3 Tips and Tricks'. Hell, I have two books. The others imaginatively called 'Leaflet Tips and Tricks', but that's still a significant work in progress. You can download them for free if you want (d3.js is free, and I'm not clever enough to contribute to the code, so why not information?). D3 Tips and tricks is over 580 pages long, so it's amassed a bit of information since I started writing.

It turns out that d3.js is a pretty popular JavaScript library. People like what it can do and want to learn how to use it. So a book on d3.js that you can download for free from Leanpub is a an attractive proposition with little to no downside. It gets downloaded a fair bit and that's great. That's what it's there for. Help people out if they are looking for some information.

You have to have goals in life right? Whether it's something to aspire to or something you want to achieve (and make sure you know the difference) they're good to have.

When I started writing D3 Tips and Tricks the goal was to learn about d3.js and to take usable notes for myself that might be useful for others.  That worked well for me and (hopefully) well for those downloading the book. So that goal got ticked off as achieved.

Then I started to take notice of the number of people who were downloading the book. Let's get things straight, it's not a landslide, we're not talking `50 shades of SVG' (although Mike Bostock's probably got that covered with some of the work he's done on palettes). So numbers of downloads of the book has been part of my goals.

First I looked forward to just moving up the list of Lifetime Number of Copies Sold. One at a time. Slowly clawing my way upward. I kept little notes. Tried to get a feel for how the numbers around me were moving.

Then I realized that I might have a chance to break into the big time. Get on the front page so to speak. That meant getting into the top 10. I don't remember how many downloads that needed at the time, but today it would take about 5500. Time passed and that goal was met. Then I dared to dream that I might get onto the top shelf. Into the top five most downloaded books on Leanpub.

This was difficult going. books up there have a lot of downloads under their belt and a lot of momentum. But more time passed and D3 Tips and Tricks stumbled onto the top shelf.

Currently It's at number four and I'm finding it difficult to give myself goals to relate to the number of people who download the book since there's not too much wiggle room to improve. But then I realized that the question I had to ask myself was how much wiggle room do I have?

So in the interests of learning a new trick, I worked out how to scrape the page of a Leanpub book to extract the number of downloads that it had and I started to record the number of downloads of the top 10 books on Leanpub every day. Using that information I started to develop a scruffy graph to show how D3 Tips and Tricks was doing in relation to it's peers.

This graph shows how many more or less books are sold than D3 Tips and Tricks of the books near the top of the most downloaded list.

It's tricky to read the text on the right because of the compressed nature of the books, but here's the key to identifying who is who;

bt = Backbone Tutorials
tnbb = The Node Beginner Book
hon = Hands-on Node.js
d3tat = D3 Tips and Tricks
htdwyl = How to do what you love and earn what you're worth
watir = Web Application Testing in Ruby
lh = Lisp Hackers
taopwi = The ABC of PDF with iText
qgffoad = Quick guide for Firefox OS App Development

There's very little to see there really other than to notice that D3 Tips and Tricks is doing fairly well compared to most, but 'The ABC of PDF with iText' (the blue line) is doing really well.

This graph didn't really help me with my goals, but then I realized that I really needed to look into the future to see what might become of all of these books and their interactions.

So I made a nice graph of trends.

That provided me with a better view of the downloading landscape and I can see that there are some significantly different gradients in there.

The ABC of PDF with iText has some serious upwards movement and has been carving up the competition at a ferocious rate. By the looks of it D3 Tips and Tricks won't be too far away from being a notch on it's belt.

But when? How will this play out? Well, the lines are fairly consistent, if we take the length of time that the data has been captured for and extrapolate using the number of books downloaded in that time we can see how the rankings and downloads will vary into the future.

Now we're talking. There's some interesting interactions.

This gave me a chance to play with a new trick I had just learned with d3.js to invert the position of the mouse on the screen into a representative date, so I was able to make a graph that provided a read out of the date that something would happen and the number of books downloaded when you hovered your mouse over it.

Sure you could do this manually, but I wouldn't have learned anything then!

So, feel free to go here to see a graph of the upcoming action.

From this, here are the predictions for the next few months.

• 24 August 2014:  Quick guide for Firefox OS App Development will pass Lisp Hackers at around 6000 downloads.
• 25 October 2014: Web Application Testing in Ruby will pass How to do what you love and earn what you're worth at around 8380 downloads.
• 2 February 2015: D3 Tips and Tricks will pass Hands-on Node.js at around 17480 downloads.
• 2 March 2015: Quick guide for Firefox OS App Development will pass How to do what you love and earn what you're worth at around 8380 downloads.
• 7 March 2015: The ABC of PDF with iText will pass Hands-on Node.js at around 17970 downloads.
• 14 March 2015: D3 Tips and Tricks will pass The Node Beginner Book at around 18590 downloads.
• 26 March 2015: The ABC of PDF with iText will pass The Node Beginner Book at around 18770 downloads.
• 5 April 2015: The ABC of PDF with iText will pass D3 Tips and Tricks at around 18770 downloads.

(Edit 22-Jul-2014: I originally typo'd the first two predictions (swapped the passing books) :-( so they read correctly (per the graph) now)

From this D3 Tips and Tricks can hope for a climb to the lofty heights of 2nd place, but this will be short lived as The ABC of PDF with iText will force it back to 3rd after only a couple of weeks.

Clearly none of this is written stone. There are so many variables I haven't considered that it makes my head spin. What sort of variations over time will each book face. Will there be a bolter from below (the astute among you will note that Laravel 4 Official Documentation doesn't feature on the graph. I can't remember why it's not there. It's possible that it wasn't reporting the number of readers when I developed the scraper)?

But what would be the point of doing all this graphical stuff if you weren't having a little fun :-).