DEV Community: Learners Just that

Maps with D3 and Svelte

AnupJoseph — Mon, 15 Nov 2021 07:30:35 +0000

This blog is fourth in a series of (unofficial) course notes for the Data Visualization with React and D3 series by Curran Kelleher. Read the introductory blog post here.

The next chart in the series is a simple map of the world. In this chart Curran actually uses a Topojson file, then converts it into Geojson and then plots it. I am not planning to do that as it seems complicated and also for some reason the topojson library does not play nice with Node on my computer and I was too lazy to actually debug it 😬. So instead I are going to use a simple geojson I found on D3 Graph Gallery.

I am going to start from a fresh Svelte project instead of carrying on from the last. To do and install d3 along with its:

npx degit sveltejs/template world-map
cd word-map
npm install
npm install d3

And to run the app itself:

npm run dev

Let clean everthing in the App.svelte. The dataset for the map is available here. Let's use the d3 json method to load in the data and use it.

import { json } from  "d3";

let  dataset  =  [];
json("https://raw.githubusercontent.com/holtzy/D3-graph-gallery/master/DATA/world.geojson"
).then((data)  => {
    dataset  =  data.features;
});

I am only using the features array in the geojson and this contains everything we need to draw the map. Let's start by making an SVG in the markup and adding a width and height property to it.

<script>
    const  width  =  1200,
    height  =  600;
</script>
<main>
    <svg  {width}  {height}>
    </svg>
</main>

If you console.log the dataset at this point (and well go deeper and deeper into the json tree) you'll notice that the geojson contains the boundary information using a construct called geometry. The only problem with this is that, this construct is not a svg path and is instead of course a co-ordinate you can plot. To solve this problem D3 provides a powerful geographic path generator, d3.geoPath which can take in a given GeoJSON feature or geometry to generate an SVG path.

D3 also provides a lot of geographical projections as well to transform how the map looks like. I am going to split out this logic to drawing the paths themeselves into a different component called Marks. So let import the new component into App.svelte like so:

import Marks from "./Marks.svelte"

And pass the dataset to the component like so:

<Marks  {dataset} />

Now of course this component does not exist, so lets go about making it real. We'll also import the projections and path generator,

<script>
    import { geoPath, geoNaturalEarth1 } from  "d3";
    export let  dataset  =  [];
    const  projection  =  geoNaturalEarth1();
    const  path  =  geoPath(projection);
</script>

Afterwards its really easy. We just need to iterate over the dataset and pass the data to the path function:

{#each  dataset  as data}
    <path  d={path(data)} />
{/each}

And now we have it, a perfectly horrible world map,

I actually kind of like the stark, edgy nature of the map. I was going to style this, but this is actually rather nice.

So of course the next part will be dedicated to showing off a cool svelte feature I just found. Let first remove all the fill colour from the map and add a stroke.

<style>
    path {
        fill: none;
        stroke:darkgreen;
    }
</style>

Svelte has great support for styling. One thing I like in particular are the transition directives. These are tools for using motion more effectively and gracefully in your app and generally making it look smoother.
Lets import the draw function from svelte/transitons and add it to the path tag.

<script>
    import { draw } from  "svelte/transition";
</script>

<path  transition:draw={{ duration: 5000, delay: 0 }}  d={path(data)} />

These transition directives actually pack in a lot of flexibility. We can pass in custom easing function to control the speed of the transition and we can even pass in custom CSS and JS to control the transtitions.

import { quadInOut } from  "svelte/easing";

<path
    transition:draw={{ duration: 5000, delay: 0, easing: quadInOut }}
    d={path(data)}
/>

Personally I think that's just awesome 🚀. Here's the final code.

Well that's it for today. Hope you have a nice day!

Line Chart with Svelte and D3

AnupJoseph — Wed, 20 Oct 2021 04:44:09 +0000

This blog is second in a series of (unofficial) course notes for the Data Visualization with React and D3 series by Curran Kelleher. Read the introductory blog post here.

The next chart type in the course is a line-chart based on San Francisco temperature data, available here. We will use the scatter plot code from the last chart here. Some interesting things about the dataset , its temporal in nature so the chart type must be able to show that change properly.

The row function in the last post is note very useful here. Lets convert the temperature column in to numbers. Also one of the columns is a timestamp so let's use the JS Date constructor to handle that.

const  row  =  function  (data)  {
    data.temperature  =  +data.temperature;
    data.timestamp  =  new  Date(data.timestamp);
    return data;
};

onMount(async  ()  => {
    dataset  =  await  csv(
"https://gist.githubusercontent.com/curran/60b40877ef898f19aeb8/raw/9476be5bd15fb15a6d5c733dd79788fb679c9be9/week_temperature_sf.csv",
    row).then((data)  => {
        return  data;
    });
});

I want to plot temperature on the Y-axis and timestamp on X-axis.Since X-axis is a timestamp we are going to use scaleTime from D3. I am going to modify the scales accordingly

$: xScale  =  scaleTime()
    .domain(extent(dataset, (d)  =>  d.timestamp))
    .range([0, innerWidth]);

$: yScale  =  scaleLinear()
    .domain(extent(dataset, (d)  =>  d.temperature))
    .range([innerHeight, 0]);

Let's just pass the temperature and timestamp points to these scales in the scatter plot logic we had before. The Axis doesn't need to be changed. Let's change the circle logic as follows:

<circle
    cx={xScale(data.timestamp)}
    cy={yScale(data.temperature)}
    r="5"
    style="fill:firebrick"
/>

Now we just need to connect these points. And it just so happens that D3 has a function to do exactly that. The line function in D3 gives a path provided a set of points. We can also specify the curveNatural to the line function.

$: line_gen  =  line()
    .curve(curveNatural)
    .x((d)  =>  xScale(d.timestamp))
    .y((d)  =>  yScale(d.temperature))(dataset);

Let's add this line as a path element to the svg <g> element. And style it as well so that it shows up as a line and not a closed path.

<path  d={line_gen} />
<style>
    path {
        fill: transparent;
        stroke: rgb(18, 153, 90);
    }
</style>

Yeah, I am not a fan of those red dots either. The line probably needs to be a bit thicker as well. Similarly we need to change the format of the ticks in the bottom axis. There are a few other stylistic changes as well and I am going to do them in a single sweep. This is the final chart produced:

And here's the code for the same:

If you want a simple line chart or use a line chart as base for something else, then this is what you probably want. I actually want to go a bit deeper and achieve a sort of hand drawing like effect.

The property we need is stroke-dasharray and stroke-dashoffset. Think of it like so, instead of drawing a continous path, we are drawing a dashed path. However the dashes themselves are pretty big, as big as the path required for this data or more infact. Okay, rather than reading through that terrible explanation you can read through Cassie Evans fantasic, interactive explanation here.

I am setting the stroke-dasharray to a nice large number.stroke-dashoffset is set to 0.

path {
    fill: transparent;
    stroke: rgb(18, 153, 90);
    stroke-width: 2.5;
    stroke-linejoin: round;
    stroke-dasharray: 4400;
    stroke-dashoffset: 0;
}

Visually there's absolutely no difference. However the underlying changes allow us to create the animation. Let's do just that:

path {

    fill: transparent;
    stroke: rgb(18, 153, 90);
    stroke-width: 2.5;
    stroke-linejoin: round;
    stroke-dasharray: 4400;
    stroke-dashoffset: 0;
    animation: draw 8.5s  ease;

}
@keyframes  draw {
    from {
        stroke-dashoffset: 4400;
    }
    to {
        stroke-dashoffset: 0;
    }
}

Initially the stroke-dashoffset is set at 4400. Hence the dash is outside the SVG as each dash is of 4400 as well. As the transition continues the dash slowly becomes visible in the SVG viewBox.

Now I actually wanted ever so slightly more than this. I wanted to have the circles we had earlier to appear on the path as the line moves ahead. I was unable to recreate the effect perfectly, but the effect is still pleasing so here goes. I am using the fly transition from Svelte. Let's modify circle accordingly.

<circle
    cx={xScale(data.timestamp)}
    cy={yScale(data.temperature)}
    r="3"
    in:fly={{ duration: 5000, delay: i  *  15 }}
/>

And this is the chart produced by adding that code

Beautiful!! I really like the teasing effect of the dots, appearing seeming at once all together yet seperate.

Here's the final code for the chart -

That's all for today! Have a nice day!

Make a scatter plot with Svelte and D3

AnupJoseph — Fri, 15 Oct 2021 07:02:53 +0000

This blog is second in a series of (unofficial) course notes for the Data Visualization with React and D3 series by Curran Kelleher. Read the introductory blog post here.

The next chart in the series is a scatter plot based on the Iris Flowers Dataset. I figured we could reuse a lot of the code from earlier examples than rewiriting everything from scratch. A properly cleaned version of the dataset by Curran is available here.The dataset has four numerical columns namely sepal_length,sepal_width,petal_length,petal_width which we need to convert to numbers. Let's change the row and onMount functions to reflect this:

const  row  =  function  (data)  {
    data.sepal_length  =  +data.sepal_length;
    data.sepal_width  =  +data.sepal_width;
    data.petal_length  =  +data.petal_length;
    data.petal_width  =  +data.petal_width;

    return data;
};

onMount(async  ()  => {

    dataset  =  await  csv(
"https://gist.githubusercontent.com/curran/9e04ccfebeb84bcdc76c/raw/3d0667367fce04e8ca204117c290c42cece7fde0/iris.csv",
    row
    ).then((data)  => {
        return  data;
    });
});

in case you are wondering where this code came from, look up this gist

The scaleBand logic we used before doesn't make much sense in a scatter plot so we need to change that to scaleLinear. I am going to plot petal_width on X-axis and petal_length on Y-axis and so let's change the domain of xScale and yScale respectively. Again doesn't matter too much, so feel free to change the X and Y axes to your liking

$: xScale  =  scaleLinear()
    .domain(extent(dataset, (d)  =>  d.petal_width))
    .range([0, width]); 

$: yScale  =  scaleLinear()
    .domain(extent(dataset, (d)  =>  d.petal_length))
    .range([0, height]);

To make the dots for the scatter plot we can make use of the <circle> SVG tag. in the plotting logic let's replace the <rect> tag by circle and specify its attributes appropriately.

<circle
    cx={xScale(data.petal_width)}
    cy={yScale(data.petal_length)}
    r="5"
/>

Something you probably noticed here is that some dots appear to be cutoff from the SVG. The solution I can think of is to shift all the circles to the left. So I am going to wrap all the circles in a <g> apply the transform directive on it. Lets use the margins that we initialized way back before to translate it across:

<g  transform={`translate(${margin.left},${margin.right})`}>
    {#each  dataset  as data, i}
        <circle
        cx={xScale(data.petal_width)}
        cy={yScale(data.petal_length)}
        r="5"
        />
    {/each}
</g>

I am also going to reconfigure the scales so that we have more space to work with at the bottom of the page and left.

const  innerHeight  =  height  -  margin.top  -  margin.bottom,
innerWidth  =  width  -  margin.left  -  margin.right;

$: xScale  =  scaleLinear()
    .domain(extent(dataset, (d)  =>  d.petal_width))
    .range([0, innerWidth]);

$: yScale  =  scaleLinear()
    .domain(extent(dataset, (d)  =>  d.petal_length))
    .range([0, innerHeight]);

The Iris flowers in this dataset are of three different species. I think it makes sense to represent them with different colors. I am going to map an array of colors to the species using the scaleOrdinal function in D3.

const classSet = new Set(dataset.map((d) => d.class));
$: colorScale = scaleOrdinal()
    .domain(classSet)
    .range(["#003049", "#d62828", "#f77f00"]);

And then change the <circle> element as follows:

<circle
    cx={xScale(data.petal_width)}
    cy={yScale(data.petal_length)}  
    r="5"
    style={`fill:${colorScale(data.class)}`}
/>

I think I'll make this a (slightly) more fully fleshed out chart and add labels and axes. First lets add x and y-axis labels. Labels are ofcourse just <text> elements.
We add the Y-axis label as follows:

<text  transform={`translate(${-25},${innerHeight  /  2}) rotate(-90)`}
>Petal Length</text>

That cryptic transform essentially just shifts to the left of all the circles and then rotate it. The Y-axis label is added as follows:

<text  x={innerWidth  /  2  }  y={innerHeight  +  30}>Petal Width</text>

Let's add an X-axis and Y-axis. We could write our own Axis component but I saw a nice reusable axis component that I quite liked here. I am going to make a few changes there and use it.

<script>

    import { select, selectAll } from  "d3-selection";
    import { axisBottom, axisLeft } from  "d3-axis";


    export let  innerHeight;
    export let  margin;
    export let  position;
    export let  scale;



    let  transform;
    let  g;

    $: {

        select(g).selectAll("*").remove();

        let  axis;
        switch (position) {
            case  "bottom":
                axis  =  axisBottom(scale).tickSizeOuter(0);
                transform  =  `translate(0, ${innerHeight})`;
                break;

            case  "left":

                axis  =  axisLeft(scale).tickSizeOuter(0);
                transform  =  `translate(${margin}, 0)`;
    }
    select(g).call(axis);
}
</script>

<g  class="axis"  bind:this={g}  {transform} />

Finally lets import the axis component and add it in the <g> element like so:

<Axis  {innerHeight}  {margin}  scale={xScale}  position="bottom" />
<Axis  {innerHeight}  {margin}  scale={yScale}  position="left" />

Yeah the Y-axis is inverted 😬. Turns out I have doing this a bit wrong. For the record, I did wonder how such thin petals were so long. But then again what do I know about Iris flowers. Fixing this is easy enough. Let's change yScale as follows:

$: yScale  =  scaleLinear()
    .domain(extent(dataset, (d)  =>  d.petal_length))
    .range([innerHeight, 0]);

If you want a simple scatter plot then this is probably all you need. I actually went on to add some more (completely unecessary) styling to it. I wanted to see if for each species of the flower we could have different shaped-petals. Not Iris petals shapes of course but petals nonetheless.

So I gathered some petal shapes from a FrontendMasters workshop by Shirley Wu here, modified them ever so slightly and saved as paths.js

export const  petalPaths  =  [
    'M0 0 C5 5 5 10 0 10 C-5 10 -5 5 0 0',
    'M-3.5 0 C-2.5 2.5 2.5 2.5 3.5 0 C5 2.5 2.5 7.5 0 10 C-2.5 7.5 -5.0 2.5 -3.5 0',
    'M0 0 C5 2.5 5 7.5 0 10 C-5 7.5 -5 2.5 0 0'
]

Let's import the petalpaths and map them to species using D3 scaleOrdinal.

import { petalPaths } from  "./paths";
$: shapeScale  =  scaleOrdinal().domain(classSet).range(petalPaths);

Finally instead of plotting circles, we plot a <path> element and set the d attribute to shapeScale.

<path
    d={shapeScale(data.class)}
    fill={`${colorScale(data.class)}`}
/>

We wrap it in a <g> element and translate it to their respective position so that they dont overlap each other.

<g

    transform={`translate(${xScale(data.petal_width)},${
    yScale(data.petal_length)  -  5
    })`}
    >
    <path
    d={shapeScale(data.class)}
    fill={`${colorScale(data.class)}`}
    />
</g>

I love this plot!
One thing to notice here however is that the plot does loose some accuracy on adding the shapes. So if that's an important concern then best stay away from it. Anyway, I think I'll end here.
Here's the full code -

So that's it for today. Have a nice day!

Make a bar chart with D3 and Svelte

AnupJoseph — Sat, 09 Oct 2021 06:51:32 +0000

This blog is part of (unofficial) course notes for the Data Visualization with React and D3 series by Curran Kelleher. Read the introductory blog post here .

The first chart Curran explores is a simple horizontal bar chart. The data used for the chart is United Nations World Population Prospects data available here on their website. I am going to be using a (badly cleaned 😬) slice of the data . In and itself, the data doesn't matter too much here. So long as it has atleast one value which we can plot as bars, any data should be useful without too much change.

To kick of the project, let first make a simple Svelte app. Degit package is a simple way to start building apps with Svelte. Feel free to replace bar-chart with any project name you want.

npx degit sveltejs/template bar-chart
cd bar-chart
npm install

This will download a starter Svelte project, build out a folder structure and install the dependencies for Svelte. To use d3, we need to add d3 to our project:

npm install d3

To run the Svelte app use

npm run dev

I am going to make a folder called data and save the csv file there. To load data I are going to use the csv utility from d3.

<script>
  import { csv } from "d3";
  import { onMount } from "svelte";

  let dataset = [];
  onMount(async () => {
    dataset = await csv("data/population_data.csv").then((data) => {
      return data.slice(0, 10);
    });
  });
</script>

Okay there's more going on here than I promised. First off, I am going to take only a slice of 10 rows from the dataset, rather than taking everything. The onMount function is a lifecycle function from Svelte which runs when the component is first loaded. Now I don't think this is actually required here. Something like

csv("data/population_data.csv", row).then((data) => {
    dataset = data.slice(0, 10);
  });

works just fine. onMount just looked like a convenient place to do the data loading work.

Now I am only going to plot the population for only the year 2020. So I am going to select that as a separate column. d3.csv only provides the output as strings. We need to convert those into numbers. Fortunately the csv method has support for an accessor function which can be used to do the necessary preprocessing.

const row = function (data) {
    data["Population"] = +data["2020"] * 1000;  // + here is the unary conversion operator
    return data;
  };
onMount(async () => {
    dataset = await csv("data/population_data.csv", row).then((data) => {
      return data.slice(0, 10);
    });
  });

The population figures are in millions, so the multiply by thousand.
In D3 we generally use SVG to render our graphics. Let's define a height and a width for our SVG and provide it with a few margins as well.

const margin = { top: 20, bottom: 20, left: 20, right: 20 };
const width = 840,
    height = 60
![bar-chart-middle.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1633758156708/v8jXQ1221.png)
;

And finally let's plot the svg.

<main>
  <svg {width} {height}></svg>
</main>

In case you are wondering why we are not writing width={width} sort of syntax, then that's Svelte's shorthand for attributes

At the moment of course, there's nothing on the browser to see. Let's start changing that. Bar charts are essentially a bunch of rectangles arranged nicely. SVG has a rect block which can be used for this purpose. We need to create one reactangle for each element in our array.

<svg {width} {height}>
    {#each dataset as data, i}
      <rect width={width} height={10} />
    {/each}
  </svg>

This is Svelte's {#each} block which is essentially a forEach which can render html. For now we'll ignore the height and width of the rect. This is what we get at the moment

It might look like only one bar but the Element inspector will show that there are 10 of them overlapping each other. To shift the bars down and to resize the bars we need to use the concept of scales. More specifically we need the scaleBand and scaleLinear functions from d3.

const yScale = scaleBand()
    .domain(dataset.map((d) => d["Location"]))
    .range([0, height]);

const xScale = scaleLinear()
    .domain([0, max(dataset, (d) => d.Population)])
    .range([0, width]);

Let's now change the rect tag to reflect this updated scales.

<rect
        x={0}
        y={yScale(data.Location)}
        width={xScale(data.Population)}
        height={yScale.bandwidth()}
/>

There we go. The browser reloads and .... it does not work!!! You refresh again hoping that maybe Svelte is a bit slow to update but the browser remains the same empty blank sheet it was 10 mins ago. In despair you turn to the inspector and it says nope width cannot be "NaN".

This is a pretty surprising error as the code you wrote seems pretty reasonable. The thing I forgot here is that csv is being processed asynchronously. So when xScale and yScale are being processed they are working on empty arrays. The dataset array changes on the way and we need our xScale and yScale to react to that change. This seemed like a perfect case for using Svelte state . Let's change the const to $:.

$: yScale = scaleBand()
    .domain(dataset.map((d) => d["Location"]))
    .range([0, height]);

$: xScale = scaleLinear()
   .domain([0, max(dataset, (d) => d.Population)])
   .range([0, width]);

And that works. There we have it. A perfectly horrible looking bar chart.

Now while, I love the stark simplicity and gravity if this bar chart, I am being told that this is simply not enough for this to be a multi-million dollar NFT. Let's clean this up a bit, add some spacing between the bars and some color to the chart

$: yScale = scaleBand()
    .domain(dataset.map((d) => d["Location"]))
    .range([0, height])
    .paddingInner(0.15);
<style>
  rect {
    fill: #13293d;
  }
</style>

That is a bit better. Let's add some transitions because its so easy to do so in Svelte. grab the fly transition from Svelte.

import { fly } from "svelte/transition";

And modify the rect block as follows.

{#each dataset as data, i}
      <rect
        x={0}
        y={yScale(data.Location)}
        width={xScale(data.Population)}
        height={yScale.bandwidth()}
        in:fly={{ x: -200, duration: 1000, delay: i * 50 }}
      />
{/each}

Here is the whole code for the graph.

That's all for today. Hope you had fun reading this!!

Thanks to Curran Kelleher for creating this amazing course

P.S. If you want a more complete example with title and axes and styling, look here

Series Intro: Data Visualization With Svelte and D3

AnupJoseph — Sat, 09 Oct 2021 06:41:32 +0000

Curran Kelleher's Data Visualization With React and D3 is probably the most comprehensive course on D3 on the Internet. For some time I have been curious about D3 and data visualization in general. While I have worked with Python libraries like Matplotlib , Seaborn and Plotly and they are fantastic for most kinds of exploratory and scientifc plots they have limitations on the web particularly in terms of interactivity. D3, for the uinitiated is the library which powers the bulk of all the interactive and SVG based plots on the web. It also acts as underlying low-level library for a lot of higher level libraries including Vega(For a more comprehensive list look here ).

In the series Curran uses D3 with React. However I personally find React impenetrable magic.

Its also not very convenient to use with D3 as both access the DOM and accessing DOM directly in React is always a risky business. I found Svelte's style of thinking really refreshing and intuitive. Svelte is a radical new approach to building user interfaces. Its essentially a compiler which takes .svelte files , which are components including HTML, CSS and JS required for the component to run and turns them into fast, lighweight code which can be executed in the browser.

So I am using Svelte instead of React for building the components. Also this series is not a tutorial on either D3 or Svelte. At best this can be thought of as course notes for my self to follow along with Curran's course. The only reason this is a series is because Hashnode allows to organize as series.
As the series progress and more articles get added I'll add them to the end of this post.

P.S. I am not very good at either D3 and Svelte (Coming to think of it not very good at JS either 😢). The code will probably be full of errors, atrocious bad practices and horrible inefficiencies. Be kind enough to point out in the comments if you notice any such in a constructive manner.

That's all now. Hope you have a good day!