Grouping and summing data using d3.nest

The following post is a portion of the D3 Tips and Tricks book which is free to download. To use this post in context, consider it with the others in the blog or just download the the book as a pdf / epub or mobi .

----------------------------------------------------------

Grouping and summing data (d3.nest).

Often we will wish to group elements in an array into a hierarchical structure similar to the GROUP BY operator in SQL (but with the scope for multiple levels). This can be achieved using the d3.nest operator. Additionally we will sometimes wish to collapse the elements that we are grouping in a specific way (for instance to sum values). This can be achieved using the rollup function.

The example we will use is having the following csv file consisting of a column of dates and corresponding values;

date,value
2011-03-23,3
2011-03-23,2
2011-03-24,3
2011-03-24,3
2011-03-24,6
2011-03-24,2
2011-03-24,7
2011-03-25,4
2011-03-25,5
2011-03-25,1
2011-03-25,4

We will nest the data according to the date and sum the data for each date so that our data is in the equivalent form of;

key,values
2011-03-23,5
2011-03-24,21
2011-03-25,14

We will do this with the following script;

d3.csv("source-data.csv", function(error, csv_data) {
 var data = d3.nest()
  .key(function(d) { return d.date;})
  .rollup(function(d) { 
   return d3.sum(d, function(g) {return g.value; });
  }).entries(csv_data);
...
});

We are assuming the data is in the form of our initial csv file and is named source-data.csv.

The first thing we do is load that file and assign the loaded arrar the variable name csv_data.

d3.csv("source-data.csv", function(error, csv_data) {

Then we declare our new array’s name will be data and we initiate the nest function;

 var data = d3.nest()

We assign the key for our new array as date. A ‘key’ is like a way of saying “This is the thing we will be grouping on”. In other words our resultant array will have a single entry for each unique date value.

  .key(function(d) { return d.date;})

Then we include the rollup function that takes all the individual value variables that are in each unique datefield and sums them;

  .rollup(function(d) { 
   return d3.sum(d, function(g) {return g.value; });

Lastly we tell the entire nest function which data array we will be using for our source of data.

  }).entries(csv_data);

You should note that our data will have changed name from date and value. This is as a function of the nestand rollup process. But never fear, it’s a simple task to re-name them if necessary using the following function (which could include a call to parse the date, but I have omitted it for clarity);

data.forEach(function(d) {
 d.date = d.key;
 d.value = d.values;
});

The description above (and heaps of other stuff) is in the D3 Tips and Tricks book that can be downloaded for free (or donate if you really want to :-)).