----------------------------------------------------------
To learn some of the capabilities of dc.js and crossfilter we will start with a rudimentary template and build chart examples as we go.
The template we'll start with will load d3.js, crossfilter.js, dc.js jquery.js and bootstrap.js. We will be including bootstrap as it provides lots of nice capabilities for fine tuning layout and styling as laid out in the chapter on using bootstrap. Since bootstrap depends on jquery, we have to load that as well.
We'll also load cascading style sheets for bootstrap and dc.js.
The template will load a csv file with earthquake data sourced from New Zealand's Geonet site over a date range that covers a period of reasonable activity in July 2013.
In it's bare bones form we will present only a data table with some values from the csv file. When we begin to add charts, we will see this table adjust dynamically.
We'll move through the explanation of the code in a similar process to the other examples in the book. Where there are areas that we have covered before, I will gloss over some details on the understanding that you will have already seen them explained in other sections.
Here is the full code (an expanded version of which can be downloaded from here with all the associated
*.js and *.css files);<!DOCTYPE html> <html lang='en'> <head> <meta charset='utf-8'> <title>dc.js Experiment</title> <script src='js/d3.js' type='text/javascript'></script> <script src='js/crossfilter.js' type='text/javascript'></script> <script src='js/dc.js' type='text/javascript'></script> <script src='js/jquery-1.9.1.min.js' type='text/javascript'></script> <script src='js/bootstrap.min.js' type='text/javascript'></script> <link href='css/bootstrap.min.css' rel='stylesheet' type='text/css'> <link href='css/dc.css' rel='stylesheet' type='text/css'> <style type="text/css"></style> </head> <body> <div class='container' style='font: 12px sans-serif;'> <div class='row'> <div class='span12'> <table class='table table-hover' id='dc-table-graph'> <thead> <tr class='header'> <th>DTG</th> <th>Lat</th> <th>Long</th> <th>Depth</th> <th>Magnitude</th> <th>Google Map</th> <th>OSM Map</th> </tr> </thead> </table> </div> </div> </div> <script> // load data from a csv file d3.csv("data/quakes.csv", function (data) { // format our data var dtgFormat = d3.time.format("%Y-%m-%dT%H:%M:%S"); data.forEach(function(d) { d.dtg = dtgFormat.parse(d.origintime.substr(0,19)); d.lat = +d.latitude; d.long = +d.longitude; d.mag = d3.round(+d.magnitude,1); d.depth = d3.round(+d.depth,0); }); // Create the dc.js chart objects & link to div var dataTable = dc.dataTable("#dc-table-graph"); // Run the data through crossfilter and load our 'facts' var facts = crossfilter(data); // Create dataTable dimension var timeDimension = facts.dimension(function (d) { return d.dtg; }); // Setup the charts // Table of earthquake data dataTable.width(960).height(800) .dimension(timeDimension) .group(function(d) { return "Earthquake Table" }) .size(10) .columns([ function(d) { return d.dtg; }, function(d) { return d.lat; }, function(d) { return d.long; }, function(d) { return d.depth; }, function(d) { return d.mag; }, function(d) { return '<a href=\"http://maps.google.com/maps?z=12&t=m&q=loc:' + d.lat + '+' + d.long +"\" target=\"_blank\">Google Map</a>"}, function(d) { return '<a href=\"http://www.openstreetmap.org/?mlat=' + d.lat + '&mlon=' + d.long +'&zoom=12'+ "\" target=\"_blank\"> OSM Map</a>"} ]) .sortBy(function(d){ return d.dtg; }) .order(d3.ascending); // Render the Charts dc.renderAll(); }); </script> </body> </html>The first part of the code starts the html file and inside the
<head> segment loads our JavaScript and css files
<!DOCTYPE html> <html lang='en'> <head> <meta charset='utf-8'> <title>dc.js Experiment</title> <script src='js/d3.js' type='text/javascript'></script> <script src='js/crossfilter.js' type='text/javascript'></script> <script src='js/dc.js' type='text/javascript'></script> <script src='js/jquery-1.9.1.min.js' type='text/javascript'></script> <script src='js/bootstrap.min.js' type='text/javascript'></script> <link href='css/bootstrap.min.css' rel='stylesheet' type='text/css'> <link href='css/dc.css' rel='stylesheet' type='text/css'> <style type="text/css"></style> </head>
It's important to note here that the order of loading the files is important. The
jquery-1.9.1.min.js file must be loaded before the bootstrap.min.js file or it just won't work.
From here we move into the section where we set up our page to load our bootstrap grid layout for the table.
<div class='container' style='font: 12px sans-serif;'> <div class='row'> <div class='span12'> <table class='table table-hover' id='dc-table-graph'> <thead> <tr class='header'> <th>DTG</th> <th>Lat</th> <th>Long</th> <th>Depth</th> <th>Magnitude</th> <th>Google Map</th> <th>OSM Map</th> </tr> </thead> </table> </div> </div> </div>
It might look a little complicated, but if you have a look through the bootstrap chapter (where we cover using the bootstrap grid layout), you will find it no problem at all.
The important features to note are that we have declared an ID selector for our table
id='dc-table-graph'and we have set a series of headers for the table; DTG, Lat, Long, Depth, Magnitude, Google Map and OSM Map.
We have also included some bootstrap styling for the table by including the the
class='table table-hover' portion of the code. With that styling included our table looks like this;
We will be adding to this grid layout section as we add in charts which will want their own allocated space on our page.
The next section of the file starts our JavaScript and d3.js.
// load data from a csv file d3.csv("data/quakes.csv", function (data) { // format our data var dtgFormat = d3.time.format("%Y-%m-%dT%H:%M:%S"); data.forEach(function(d) { d.dtg = dtgFormat.parse(d.origintime.substr(0,19)); d.lat = +d.latitude; d.long = +d.longitude; d.mag = d3.round(+d.magnitude,1); d.depth = d3.round(+d.depth,0); });
We load our csv file with the line
d3.csv("data/quakes.csv", function (data) {. I have deliberately left this file in its raw form as received from Geonet. Its format looks a little like this;
FID,publicid,origintime,longitude,latitude,depth,magnitude,magnitudetype,status,phases,type,agency,updatetime,origin_geom
quake.2013p550753,2013p550753,2013-07-23T18:41:11.707,174.4298,-41.5313,7.9883,2.2425,M,automatic,27,,WEL(GNS_Primary),2013-07-23T18:43:15.672,POINT (174.42978 -41.531299)
quake.2013p550747,2013p550747,2013-07-23T18:38:02.481,174.414,-41.5181,11.6797,1.7892,M,automatic,11,,WEL(GNS_Primary),2013-07-23T18:39:25.37,POINT (174.41398 -41.518114)
quake.2013p550725,2013p550725,2013-07-23T18:26:30.229,175.5516,-40.0264,8.75,3.4562,M,automatic,21,,WEL(GNS_Primary),2013-07-23T18:29:46.305,POINT (175.55155 -40.026412)
We then declare a small function that will format our time correctly (
var dtgFormat = d3.time.format("%Y-%m-%dT%H:%M:%S");). This follows exactly the same procedure we took when creating our very first simple line graph at the start of the book.
T> However, there is a slight twist... Observant readers will notice that while we have a function that resolves a date/time that is formatted with year, month, day, hour, minute and second values, I don't include an allowance for the fractions of seconds that appear in the csv file. Well spotted. The reason for this is that in spite of initially including this formatting, I found it caused some behaviour that I couldn't explain, so I reverted to cheating and you will note that in the next section when I format the values from the csv file, I truncate the date/time value to the first 19 characters (
d.origintime.substr(0,19)). This solved my problem by chopping off the fractions of a second (admittedly without actually solving the underlying issue) and I moved on with my life.
As mentioned. the next section goes through each of the records and formats them correctly. The date/time gets formatted, the latitude and longitude are declared as numerical values (if they weren't already) and the magnitude and depth valued are rounded to make the process of grouping them simpler.
data.forEach(function(d) { d.dtg = dtgFormat.parse(d.origintime.substr(0,19)); d.lat = +d.latitude; d.long = +d.longitude; d.mag = d3.round(+d.magnitude,1); d.depth = d3.round(+d.depth,0); });
The next section in our code sets up the dimensions and groupings for the dc.js chart type and crossfilter functions
// Create the dc.js chart objects & link to div var dataTable = dc.dataTable("#dc-table-graph"); // Run the data through crossfilter and load our 'facts' var facts = crossfilter(data); // Create dataTable dimension var timeDimension = facts.dimension(function (d) { return d.dtg; });
The first line assigns the variable
dataTable to the dc.js dataTable chart type (var dataTable = dc.dataTable("#dc-table-graph");) and assigns the chart to the ID selectordc-table-graph.
Then we load all of our data into crossfilter (
var facts = crossfilter(data);) and give it the name facts.
Lastly we create a dimension from our data (
facts) of the date/time values.var timeDimension = facts.dimension(function (d) { return d.dtg; });The last major chunk of code is the piece that configures our data table.
dataTable.width(960).height(800) .dimension(timeDimension) .group(function(d) { return "Earthquake Table" }) .size(10) .columns([ function(d) { return d.dtg; }, function(d) { return d.lat; }, function(d) { return d.long; }, function(d) { return d.depth; }, function(d) { return d.mag; }, function(d) { return '<a href=\"http://maps.google.com/maps?z=12&t=m&q=loc:' + d.lat + '+' + d.long +"\" target=\"_blank\">Google Map</a>"}, function(d) { return '<a href=\"http://www.openstreetmap.org/?mlat=' + d.lat + '&mlon=' + d.long +'&zoom=12'+ "\" target=\"_blank\"> OSM Map</a>"} ]) .sortBy(function(d){ return d.dtg; }) .order(d3.ascending);
Firstly the width and height are declared (
dataTable.width(960).height(800)). Then the dimension of the data that will be used is declared (.dimension(timeDimension)).
Separate sections of the table can have a header applied. In this case the entire table is given the grouping 'Earthquake Table' (
.group(function(d) { return "Earthquake Table"})), but several examples online give the date.
The
.size(10) line sets the maximum number of lines of the table to be displayed to 10.
Then we have the block of code that sets what data appears in which columns. It should be noted that this matches up with the headers that were declared in the earlier section of the code where the div's for the table were laid out.
The portion of this block that has a '*little bit of fancy*' are the two columns that set links that allow a user to click on the designation 'Google Map' or 'OSM Map' and have the browser open a new window containing a Google or Open Street Map (OSM) map with a marker designating the location of the quake. I won't mention too much about how the links are made up other than to say that they are pretty much a combination of the latitude, longitude and zoom level for both. Please check out the code for more.
Lastly we sort by the date/time value (
.sortBy(function(d){ return d.dtg; })) in ascending order (.order(d3.ascending);).
The final part of our JavaScript renders all our charts (
dc.renderAll();) and then closes off the initial d3.csv call.// Render the Charts dc.renderAll(); });
The final part of our code simply closes off the
<script>, <body> and <html> tags.
There we have it. The template for starting to play with different crossfiltered dc.js charts.
Thanks a bunch. Learning a lot from your d3 adventures.
ReplyDeleteVery nice article, thank you for writing it. I added it to my list of crossfilter resources... http://robertjliguori.blogspot.com/2016/12/interactive-data-visualization-with.html
ReplyDeletesabungayam.co
ReplyDelete