Tuesday, 18 February 2014

How to import data from a csv file with spaces in the header under d3.js

The following post is a portion of the D3 Tips and Tricks book which is free to download. To use this post in context, consider it with the others in the blog or just download the the book as a pdf / epub or mobi .
----------------------------------------------------------

How to use data imported from a csv file with spaces in the header.

When importing data from a csv file that has headers with spaces in the middle of some of the fields there is a need to address the data slightly differently in order for it to be used easily in your JavaScript.
For example the following csv data has a column named ‘Date Purchased’;
Value,Date Purchased,Score
12345,2011-03-23,99
22345,2011-03-24,100
32345,2011-03-25,99
42345,2011-03-26,100
This is not an uncommon occurrence since RFC 4180 which specifies csv content allows for it and d3.js supports the RFC;
Within the header and each record, there may be one or more fields, separated by commas. Each line should contain the same number of fields throughout the file. Spaces are considered part of a field and should not be ignored.
When we go to import the data using the d3.csv function, we need to reference the ‘Data Purchased’ column in a way that makes allowances for the space. The following piece of script (with grateful thanks to Stephen Thomas for answering my Stack Overflow question) appears to be the most basic solution.
d3.csv("sample-data.csv", function(error, data) {
    data.forEach(function(d) {
        d.date = parseDate(d['Date Purchased']);
    });
...
});
In the example above the ‘Date Purchased’ column is re-declared as ‘date’ making working in the following script much easier.

The description above (and heaps of other stuff) is in the D3 Tips and Tricks book that can be downloaded for free (or donate if you really want to :-)).

4 comments:

  1. Hi, I want to make the following var englishArray get from CSV file.
    var englishArray =[ "I am pen.","He is OK","I am a pen","I am OK","I am Kevn"];

    I tried as follows;
    d3.csv("data.csv", function(error, data) {
    data.forEach(function(d) {
    d.date = parseDate(d['value']);
    var englishArray = d.date;

    });

    });

    It doesn't work. Can you give me some advices?

    ReplyDelete
    Replies
    1. Hi Kevin, I think that you my be getting your code tangled up a bit there. You have what appears to be a date value being parsed, but it appears to be a text value. I think that you might need to take a step back and start at an earlier point in the book to get a better grounding before this point in data manipulation. Conversely, try googling for JavaScript data manipulation and see what that brings up. Although I seriously think that you might benefit from going back to a simpler task before getting to this one. Good luck

      Delete
  2. I'm working on a chart in d3.js I've added some code for a tooltip and I don't know how to call the actual data into the tip. I can get the name of the column from the csv but not the actual data of the csv.

    The function for the tooltip is
    d3.tip.v0.6.3.js
    var tip = d3.tip()
    .attr('class', 'd3-tip')
    .offset([-10, 0])
    //This is the internal html for the tooltip
    .html(function(d) {
    return(d.name + " " + /*something goes here*/);
    });



    The data looks like this...
    Category,Statistics,Psychology,Programming,Mental health,Aboriginal education
    Other,1,3,1,4,2
    Lifelong learner,20,39,36,44,48
    Professional,40,18,29,20,20
    Academic,5,3,10,3,14
    Research ,15,4,2,13,3
    Student,19,33,22,16,13

    ReplyDelete
    Replies
    1. Hi James,

      Because I don't have any experience with d3.tip I can only really recommend that you have a look at this portion of the book which deals with tool tips. https://leanpub.com/D3-Tips-and-Tricks/read#leanpub-auto-adding-tooltips.
      Your data looks fine. I suspect that you may be jumping ahead of yourself slightly, so the link above may help to give context to the problem. Best of luck

      Delete