DEV Community: Pratibha KS

Do you know, you can create custom charts with no code?

Pratibha KS — Sat, 03 Apr 2021 05:10:41 +0000

Let me introduce you to Charticulator.

Charticulator is an open source and free to use tool from Microsoft Research team, which helps you to create reusable custom chart layouts with no code. This tool is implemented as an HTML5 application and uses technologies including React, Typescript and Web Assembly. This is a browser based tool, which means your charts stay in your browser as long you don't delete your browsing history. Its interface is easy to use and intuitive. You can export your built charts as Image files (JPEG, PNG, SVG), HTML, Charticulator template. The best part is, it lets you export your visual as Power BI Custom Visual which can imported as a template into Power BI and can be used as a normal chart by plugging in data of your choice (of course, your data should support the chart design).

Below image is an example from Charticulator gallery. Here is the demo showing how this chart was built. There are many examples in the gallery which would give you inspiration to ideate your own chart designs.

So far so good, but why do we need a separate tool to build custom charts? And, what are custom charts?

If you work with data visualizations, you will be familiar with the limitations of BI tools available. Mostly, these tools only provide predefined chart layouts like bar charts, scatter plots etc. What if you have a dataset which doesn't fit into any of the predefined standard charts? In other words, you cannot explain your data with any of the existing charts. You will need to create custom chart which lets you present insights from your data. Custom charts are novel charts which are not readily available for use and has to be built from scratch. Unfortunately, creating these charts is not a straight-forward process and only few tools helps you in this process and they require a lot of coding, which means you have to code your chart yourself.

D3.js is one such visualization library which lets you code your own visual. Unfortunately, there is a steep learning curve involved with D3.js. What if you have more than one chart design in mind and you want to test the designs with your data to understand which one works best? Then, D3.js is not a go-to tool as it involves lot of coding and it takes time to prototype all your chart designs. Especially, when you are pressed for time and you need a quick prototype. For these scenarios, Charticulator comes in handy.

If you would like to learn more about this project, please read their paper.

Sortable Bar Chart with D3.js

Pratibha KS — Sat, 14 Nov 2020 03:40:04 +0000

This chart is made based on Mike Bostock's example code: https://observablehq.com/@d3/sortable-bar-chart#chart

I rewrote the code implementing my own logic. Let's quickly understand the data behind this code before we dive into the steps of creating the chart.

This is a small dataset with Alphabets and their frequency of usage in words.

Here is the link to the dataset.

Components to be added:

Add a container DIV which holds the SVG element and its contents
Add a drop down select box which let's you select the options to sort the bars
Finally, add an SVG element where you draw your chart

Here is the complete code for creating this chart. I will explain each section as and when required.

HTML File: index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <script src="https://d3js.org/d3.v5.min.js"></script>
    <link rel="stylesheet" href="style.css" />
    <title>Sorted Bar Charts</title>
  </head>
  <body>
    <h1 style="text-align: center">Sortable Bar charts</h1>
    <p style="text-align: center">
      This chart implements bar chart sorting with D3.js.
    </p>
    <div id="selectMe"></div>
    <div id="chartContainer"></div>
    <script src="script.js"></script>
  </body>
</html>

If you observe in this HTML file, I have created two DIVs. One holds the drop down and other holds the SVG element.

CSS File: style.css

.label {
  margin-left: 15px;
}

#selection {
  font-size: 19px;
  margin-left: 15px;
}

This CSS file is styling the drop down and its label.

JavaScript File: script.js

const w = 1000;
const h = 500;

const margins = { top: 50, left: 100, bottom: 100, right: 100 }
const innerWidth = w - margins.left - margins.right;
const innerHeight = h - margins.top - margins.bottom;

const url = "https://gist.githubusercontent.com/learning-monk/96c71857dcfc348ecbcda5cb4a90163a/raw/69fa895f7b5f2d99d236762969c46b1a9c47604a/alphabet.csv";

d3.csv(url).then((data, error) => {
  if (error) throw error;
  // console.log(data);

  data.forEach(d => {
    d.letter = d.letter;
    d.frequency = +d.frequency;
  });

  // Create a select dropdown
  const mySelection = document.getElementById("selectMe");

  d3.select(mySelection).append("span").append("p").attr("class", "label").text("How should these bars sorted?").style("font-weight", "bold").style("color", "red").style("font-size", "25px");

  const selectItems = ["Alphabetically", "Ascendingly", "Descendingly"];

  // Create a drop down
  d3.select(mySelection)
    .append("span")
    .append("select")
    .attr("id", "selection")
    .attr("name", "tasks")
    .selectAll("option")
    .data(selectItems)
    .enter()
    .append("option")
    .attr("value", d => d)
    .text(d => d);

  // When the page loads, the chart which sorted alphabetically loads by default
  document.addEventListener("DOMContentLoaded", myChart()); 


  // Chart changes based on drop down selection
  d3.select("#selection").on("change", function() {
    const selectedOption = d3.select(this).node().value;
    if (selectedOption == "Ascendingly") {
      data.sort((a,b) => {
        return d3.ascending(a.frequency, b.frequency)
      }) 
    } else if (selectedOption == "Descendingly") {
      data.sort((a,b) => {
        return d3.descending(a.frequency, b.frequency)
      })
    } else if (selectedOption == "Alphabetically") {
      data.sort((a,b) => {
        return d3.ascending(a.letter, b.letter)
      })
    }
    myChart();
  })

  function myChart () {
    // Append SVG to this DIV
    const chartDIV = document.createElement("div");

    // Create scales
    const xScale = d3.scaleBand()
    .domain(data.map((d) => d.letter))
    .rangeRound([0, innerWidth])
    .paddingInner(0.05);

    const yScale = d3.scaleLinear()
      .domain([0,d3.max(data, d => d.frequency)]).nice()
      .range([innerHeight, 0]);

    const xAxis = d3.axisBottom().scale(xScale);

    const yAxis = d3.axisLeft().scale(yScale);

    const svg = d3.select(chartDIV)
      .append("svg")
      .attr("viewBox", [0,0,w,h]);

    const mainG = svg
      .append("g")
      .attr("transform", `translate(${margins.left}, ${margins.top})`);

    const g = mainG
      .selectAll("g")
      .data(data)
      .enter()
      .append("g")
      .attr("transform", `translate(15,0)`);

    g.append("rect")
      .attr("class", "bars")
      .attr("x", d => xScale(d.letter) - innerWidth/data.length/2)
      .attr("y", d => yScale(d.frequency))
      .attr("width", innerWidth/data.length-1.5)
      .attr("height", (d) => innerHeight - yScale(d.frequency))
      .attr("fill", d => d.frequency == d3.max(data, d => d.frequency) ? "#f4c430" : "green")
      .append("text")
        .attr("x", 5*3)
        .attr("y", (d,i) => i*5)
        .text(d => d.frequency);

    mainG
      .append("g")
      .call(xAxis)
      .attr("transform", `translate(0, ${innerHeight})`);

    mainG
      .append("g")
      .call(yAxis);

    // This code will redraw charts based on dropdown selection. At any point in time, chartContainer DIV only contains one chart. The charts are recycled.
    const showChart = document.getElementById("chartContainer");
    while (showChart.firstChild) {
      showChart.firstChild.remove();
    }
    showChart.appendChild(chartDIV);

  }

});

This D3.js code can be broadly divided into blocks as shown below.

Load Data
Transform data
Create a drop down with values
Create scales
Create SVG
Add chart elements to SVG
Append SVG to Chart container DIV
Write functions to render charts for Load and Change events.

Most of the steps mentioned above are common for D3.js charts. So, I am not going in depth of each step. However, I want to particularly highlight the Load and Change events associated with this chart.

When the chart loads, a default chart is displayed ie., bars in alphabetical order.

document.addEventListener("DOMContentLoaded", myChart());

myChart() is a function which is template to draw chart.

When a user changes the selection from the drop down, charts based on the selection gets rendered. This is when Change event kicks in.

   d3.select("#selection").on("change", function() {
      const selectedOption = d3.select(this).node().value;
      if (selectedOption == "Ascendingly") {
        data.sort((a,b) => {
          return d3.ascending(a.frequency, b.frequency)
        }) 
      } else if (selectedOption == "Descendingly") {
        data.sort((a,b) => {
          return d3.descending(a.frequency, b.frequency)
        })
      } else if (selectedOption == "Alphabetically") {
        data.sort((a,b) => {
          return d3.ascending(a.letter, b.letter)
        })
      }
     myChart();
   })

This piece of code sorts the underlying data based on user selection and that data is passed on to myChart() to draw the chart.

Recycling happens at the end of the code where SVG along with its chart elements gets changed based on user selection. Similar to blackboard where the previous notes is erased and fresh notes is drafted.

Hope this article helped you in understanding the logic behind sorting bars in a bar chart.

Feel free to reach out to me if you have any questions.

Analyzing suicide rates with D3.js

Pratibha KS — Thu, 24 Sep 2020 09:49:36 +0000

In this article, I am going to create a simple Stacked bar chart using D3.js (a JavaScript library for data visualization) to understand suicide rates by Region and Gender for the period 1985-2016.

Here is the link to the dataset.

You might ask me, why use D3.js for creating such simple charts, as much easier tools are available. You are right. D3.js is an overkill for such a simple chart. But, the reason behind using D3.js is, I like to create charts from scratch with code, and D3.js helps me learn intricacies of building charts unlike the commercial tools which gives me ready-made templates to work on.

My goal here is to understand how each piece is put together to create charts in D3.js.

Now, let's get started with building our stacked bar chart. By the way, you can choose any other chart which suits the question at hand. I have chosen stacked bar chart as I can analyze suicides by region and gender in one chart. Also, bar charts are good for analyzing categorical data. Feel free to experiment with other relevant chart types.

On a side note, I will be using Visual Studio code (a code editor). Feel free to use code editor of your choice and comfort.

Github project link

The first step is to create a blank HTML file and load D3.js library. Let's name this file index.html.

HTML file

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <meta http-equiv="X-UA-Compatible" content="ie=edge" />
    <title>Stacked bar chart</title>
    <!-- Load D3.js library -->
    <script src="https://d3js.org/d3.v5.min.js"></script>
    <link rel="stylesheet" href="style.css" />
  </head>
  <body>
    <div id="tooltip" class="hidden">
      <p>Region: <span id="region"></span></p>
      <p>Gender: <span id="gender"></span></p>
      <p>Suicides: <span id="suicides"></span></p>
    </div>
    <script src="stackedbar.js"></script>
  </body>
</html>

Ok, let's understand the details of this file before moving forward.

We are loading D3.js library
Load external CSS file (yet to be created)
Load a JavaScript file with all the code that is required to create this chart
Created a div element for the tooltip (more on this later)

Now, let's create JavaScript file which will host the code for creating the data visualization we want to develop. Let's name this file stackedbar.js.

JavaScript file

// set svg width and height
const width = 900,
  height = 600;
const margin = { top: 100, right: 100, bottom: 100, left: 100 };
const innerWidth = width - margin.left - margin.right;
const innerHeight = height - margin.top - margin.bottom;

//Load data
d3.csv("https://raw.githubusercontent.com/learning-monk/DataVisualizationProjects_JS/master/DataViz_with_D3js/Stacked%20bar%20chart%20-%20suicides%20by%20region/suicide_rates_1985-2016.csv")
  .then((finalData) => {
    finalData.forEach((d) => {
      d.Region = d.Region;
      d.female = parseFloat(d.female);
      d.male = parseFloat(d.male);
    });

    // set up stack method
    const stack = d3.stack().keys(["female", "male"]);
    // Data, stacked
    const series = stack(finalData);

    // set-up scales
    const xScale = d3
      .scaleBand()
      .domain(finalData.map((d) => d.Region))
      .range([0, innerWidth])
      .padding(0.1);

    const xAxis = d3.axisBottom().scale(xScale);

    const yScale = d3
      .scaleLinear()
      .domain([0, d3.max(series, (d) => d3.max(d, (d) => d[1]))])
      .rangeRound([innerHeight, 0]);

    const yAxis = d3.axisLeft().scale(yScale);

    //Easy colors accessible via a 10-step ordinal scale
    const colors = d3
      .scaleOrdinal()
      .domain(series.map((d) => d.key))
      .range(
        d3
          .quantize((t) => d3.interpolateSpectral(t * 0.8 + 0.1), series.length)
          .reverse()
      )
      .unknown("#ccc");

    //Create SVG element
    const svg = d3
      .select("body")
      .append("svg")
      .attr("viewBox", [0, 0, width, height]);

    const mainG = svg
      .append("g")
      .attr("transform", `translate(${margin.left},${margin.top})`);

    // Add a group for each row of data
    const g = mainG
      .selectAll("g")
      .data(series)
      .enter()
      .append("g")
      .style("fill", (d) => colors(d.key))
      .attr("transform", `translate(0,0)`);

    // Add a rect for each data value
    g.selectAll("rect")
      .data((d) => d)
      .enter()
      .append("rect")
      .attr("x", function (d, i) {
        return xScale(d.data.Region);
      })
      .attr("y", function (d) {
        return yScale(d[1]);
      })
      .attr("height", function (d) {
        return yScale(d[0]) - yScale(d[1]);
      })
      .attr("width", xScale.bandwidth())
      .on("mouseover", function (d) {
        //Get this bar's x/y values, then augment for the tooltip
        let xPosition =
          parseFloat(d3.select(this).attr("x")) + xScale.bandwidth() * 2;
        let yPosition = parseFloat(d3.select(this).attr("y")) + innerHeight / 2;

        // Update the tooltip position and value
        d3.select("#tooltip")
          .style("left", xPosition + "px")
          .style("top", yPosition + "px")
          .select("#region")
          .text(d.data.Region);

        d3.select("#tooltip")
          .select("#gender")
          .text(d[0] === 0 ? "Female" : "Male");

        d3.select("#tooltip")
          .select("#suicides")
          .text(d[1] - d[0]);

        //Show the tooltip
        d3.select("#tooltip").classed("hidden", false);
      })
      .on("mouseout", function () {
        //Hide the tooltip
        d3.select("#tooltip").classed("hidden", true);
      });

    // draw legend
    const legend = mainG
      .append("g")
      .attr("class", "legend")
      .attr("transform", "translate(" + (innerWidth - 110) + "," + 20 + ")")
      .selectAll("g")
      .data(series)
      .enter()
      .append("g");

    // draw legend colored rectangles
    legend
      .append("rect")
      .attr("y", (d, i) => i * 30)
      .attr("height", 10)
      .attr("width", 10)
      .attr("fill", (d) => colors(d.key));

    // draw legend text
    legend
      .append("text")
      .attr("y", (d, i) => i * 30 + 9)
      .attr("x", 5 * 3)
      .text((d) => d.key);

    mainG
      .append("g")
      .call(xAxis)
      .attr("transform", `translate(0,${innerHeight})`);

    mainG.append("g").call(yAxis);

    mainG
      .append("text")
      .attr("class", "chart-title")
      .attr("y", -40)
      .attr("x", innerHeight / 2)
      .text("Suicides by Region and Gender (1985-2016)")
      .attr("font-family", "arial")
      .attr("font-size", 24);
  })
  .catch((error) => {
    console.log(error);
  });

I am not planning to provide detailed explanation of each line of code in this article, as comments are provided at the right places to follow along. I am planning to start a separate tutorial series which covers D3.js specific topics. There, we can discuss how to create chart elements in detail.

I would encourage you to experiment with the code provided and come-up with your own versions of the visualization, which is, in my opinion a nice way of learning.

I will only take you through a few important points which will help you understand the code much easily.

This visualization is divided into 4 parts:

Main Chart
Chart title
Legend
Tooltip

SVG acts as a canvas which holds all the data visualization elements. The Main chart consists of bars for each gender which are stacked one over the other. The x-axis has Region and the Y-axis shows the count.

We are appending chart elements to this SVG element. Also, we are grouping common elements (example: x & y axis) and adding these groups to the SVG element. The advantage of creating groups is, maintaining and styling these chart elements is easier.

Tooltip: We can use a simple built-in tooltip for the chart. But for this chart, we have created a custom tooltip with a div element. Also, we are positioning the tooltip at a place of our choice.

Let's create a css file (style.css) which will style the chart elements. This file is only styling tooltip for now.

CSS file

#tooltip {
  position: absolute;
  width: 200px;
  height: auto;
  padding: 10px;
  background-color: lightblue;
  -webkit-border-radius: 10px;
  -moz-border-radius: 10px;
  border-radius: 10px;
  -webkit-box-shadow: 4px 4px 10px rgba(0, 0, 0, 0.4);
  -moz-box-shadow: 4px 4px 10px rgba(0, 0, 0, 0.4);
  box-shadow: 4px 4px 10px rgba(0, 0, 0, 0.4);
  pointer-events: none;
}

#tooltip.hidden {
  display: none;
}

#tooltip p {
  margin: 0;
  font-family: sans-serif;
  font-size: 16px;
  line-height: 16px;
}

Analysis

From the chart it is clear that suicide rate among Males is high compared to Females.
Region wise, Europe tops the list with most suicides for the period 1985-2016.

This chart only gives us a preliminary view of the data which answers the question we started off with, which region has highest suicide rates and how is this data distributed by gender?

There is scope for more analysis. You can find a lot of other data points connected to this dataset in this file.

Feel free to conduct your own analysis and share your work.

Who is a Data Visualization Engineer?

Pratibha KS — Wed, 23 Sep 2020 14:47:22 +0000

To answer simply, Data Visualization Engineer is someone who knows how to present data and story tell to drive the point home.

Let’s discuss what skills one should have to enjoy being in this role:

When you look at data especially messy data, your eyes should lit up and your fingers should start working already to clean and process data at hand.
When someone explains a problem, you kick start the process by doing research. The more research you do the better is your grip on the problem at hand. The better you understand the problem, the better is your presentation of solutions via Data Visualization.
If the question at hand is from a domain which is completely new to you, you should feel excited. Here is an opportunity for you to learn something new. Talk to people from that particular domain, do research and understand what is required to know about the domain to solve the problem. You should have good networking skills.
Data is not always available ready-made. If the data is not readily available, use your skills like web scrapping (with proper permissions) to get the data. Develop strong data acquisition skills. Sometimes, you have to design innovative solutions to acquire data.
If the data is not in the format you expected, pre-process it. Your programming skills should come in handy.
If the data is considerably big (that doesn’t fit in a spreadsheet) and you want to store it in a database, you should know how.
Sometimes, your data is distributed across different sources, you should have strong data engineering skills to get all of that data in one place.
Once your data is in the database, you want to conduct preliminary analysis to understand the data. Your strong SQL skills should help. You could also use Python, R or any other programming language of your choice to conduct Exploratory Data Analysis. Understand your data very well.
With the knowledge at hand, you should decide how you want to present your data and the solutions. Which tool to use. Always choose what suits the data. Develop knowledge and hands-on skills with different tools available including charting libraries, open source and commercial tools. You should feel excited to learn new tools.
Now, you have decided on the tool, using your creativity, design how you want to present your data and the solutions. You should have a good understanding of Data Visualization principles. Have a mental picture of the final product. If you are comfortable using pen and paper to sketch the visualization, do that. Remember, data visualization is an iterative process. There is no one-all-solution. You can tackle the problem from different angles.
Your eye-for-detail will help you in finding insights which are sometimes hidden.
Choose appropriate charts to present the data. Data Visualization need not be always charts, it could be a comic or an infographic as well. You should have a good understanding of visual vocabulary.
Decide on the medium of presentation. This should be driven by the question as to whom you are presenting it to. Presentation could be on a power point or an excel spreadsheet as well. There is no rule that the presentation medium has to be fancy. The success of the presentation depends on the key takeaways from the data.
Ok, we have come to the final stage of our skill list. This is the most important skill to have ‘Story-telling’. This one skill will set you apart as a skilled professional. All the effort you have put in acquiring, processing, designing data will be of no use if you fail to show the audience the key takeaway from the data. Be a good Storyteller.

Note: This article is purely based on my knowledge and experience. These skills by no means comprehensive. There will always be a new skill to learn. To the many experts out there who would like to add any other skill I might have missed, please feel free to do so.