DEV Community: Oscar Leo

Matplotlib Tutorial: Life Expectancy & Genocide

Oscar Leo — Sun, 10 Mar 2024 16:22:58 +0000

Background

In this tutorial, I'm creating a horrifying data visualization about how life expectancy at birth changed in Bangladesh, Rwanda, and Cambodia during their genocides. I'm using data from the World Bank open data platform, which you should check out.

If you like this tutorial and visualization, make sure to sign up for my free newsletter, The Data Sensei.

Step 1: Imports

We don't need anything special for this tutorial

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

from matplotlib.patches import Rectangle
from matplotlib.lines import Line2D

Step 2: Styling

I always use Seaborn to create a style that suits the data. You can't use cheerful colors when you design a plot with data like what we have in this tutorial. Since I still want a bright chart, I only use grayscale colors.

font_family = "DejaVu Serif"
background_color = "#e5e5e5"
light_gray = "#858585"
dark_gray = "#454545"
text_color = "#000000"

sns.set_style({
    "axes.facecolor": background_color + "00",
    "figure.facecolor": background_color,
    "axes.edgecolor": text_color,
    "axes.grid": True,
    "axes.axisbelow": True,
    "grid.color": light_gray,
    "text.color": text_color,
    "font.family": font_family,

    "xtick.color": text_color,
    "ytick.color": text_color,
    "xtick.bottom": False,
    "xtick.top": False,
    "ytick.left": False,
    "ytick.right": False,

    "axes.spines.left": False,
    "axes.spines.bottom": False,
    "axes.spines.right": False,
    "axes.spines.top": False,
})

Step 3: Helper functions

When I create plots, I always create reusable functions, but in this tutorial, I only have one. It's a really simple data visualization, so I don't need more.

def color(v, threshold=38):
    return light_gray if v > threshold else dark_gray

Step 4: Shared styles

Defining shared styles you reuse in several places outside the main functions is a good idea. It's always annoying when you have to change things like color and font sizes in multiple locations, and it's easy to introduce a design bug that you don't find in time.

BAR_STYLE = {"width": 1, "edgecolor": text_color}
LINE_STYLE = {"zorder": -10, "color": dark_gray, "linewidth": 2}
YEAR_STYLE = {"va": "center", "fontsize": 28, "fontfamily": "PT Serif Caption"}
INFO_STYLE = {"ha": "left", "va": "top", "fontsize": 24, "linespacing": 1.8, "color": dark_gray}

Step 5: Loading data

I have created a preprocessed dataset so that you can focus on the Matplotlib code. The CSV has four columns, one for each country and the year of the values. The values represent life expectancy at birth, which means how long someone born that year can expect to live. As you will see, that value changes drastically over time.

data = pd.read_csv(
    "https://raw.githubusercontent.com/oscarleoo/matplotlib-tutorial-data/main/life_expectancy_genocide.csv"
)

Step 6: Data visualization

Now, it's time to create the actual chart. It's a straightforward chart where I use Rectangle to draw the bars, and everything else basically adds information using lines and text. I'm using Rectangle instead of something like seaborn.barplot() because it gives me more control, and I don't need subplots. With subplots, I find it more difficult to add custom styles.

There are some hard-coded values to decide the text position, but I've found that it's faster than trying to automate the placement. All charts require some custom design to get the right feel.

fig = plt.figure(figsize=(30, 30))
ax = fig.add_subplot()
ax.set(ylim=(-68, 3.7*100), xlim=(-2, 63))

# Add bars
for i, row in data.iterrows():
    ax.add_artist(Rectangle((i-0.5, 200), height=row.rwanda, facecolor=color(row.rwanda), **BAR_STYLE))
    ax.add_artist(Rectangle((i-0.5, 100), height=row.cambodia, facecolor=color(row.cambodia), **BAR_STYLE))
    ax.add_artist(Rectangle((i-0.5, 0), height=row.bangladesh, facecolor=color(row.bangladesh), **BAR_STYLE))

# Add country names
ax.annotate("Rwanda", (-0.5, 265), ha="left", va="bottom", fontsize=48)
ax.annotate("Cambodia", (-0.5, 165), ha="left", va="bottom", fontsize=48)
ax.annotate("Bangladesh", (-0.5, 65), ha="left", va="bottom", fontsize=48)

# Tick formatting
ax.tick_params(labelsize=28, pad=16)
ax.set_xticks([])
ax.set_yticks(
    ticks = [0, 20, 40, 60, 100, 120, 140, 160, 200, 220, 240, 260],
    labels = ["0", "20", "40", "60", "0", "20", "40", "60", "0", "20", "40", "60"]
)

# Add lines
ax.add_artist(Line2D([11, 11, 7], [0, -20, -20], **LINE_STYLE))
ax.add_artist(Line2D([16.5, 16.5, 21], [100, -20, -20], **LINE_STYLE))
ax.add_artist(Line2D([34, 34, 42], [200, -20, -20], **LINE_STYLE))

# Bangladesh info 
ax.annotate("1971", (6.5, -20), ha="right", **YEAR_STYLE)
ax.annotate(
    "Ethnic cleansing of\n"
    "Bengalis in East Pakistan during\n"
    "the Bangladesh Liberation War.", (3.8, -28), **INFO_STYLE
)

# Cambodia info
ax.annotate("1975-1978", (21.5, -20), ha="left", **YEAR_STYLE)
ax.annotate(
    "Systematic killing of Cambodian\n"
    "citizens by the Khmer Rouge under\n"
    "the leadership of Pol Pot.", (21.6, -28), **INFO_STYLE
)

# Rwanda info
ax.annotate("1994", (42.5, -20), ha="left", **YEAR_STYLE)
ax.annotate(
    "Hutu militias slaughter\n"
    "the Tutsi population\n"
    "in Rwanda", (42.6, -28), **INFO_STYLE
)

# Title and source
title = "Life Expectancy at Birth During Genocides"
ax.annotate(title, (-0.5, 320), ha="left", va="bottom", fontsize=64)
ax.annotate("Source: World Development Indicators", (-0.5, 343), ha="left", va="bottom", fontsize=32, color=dark_gray)

plt.show()

When you run this code, you get the following visualization.

Conclusion

Above, you see the final data visualization. It might not look beautiful at first glance, but remember that we have a sad message that we must keep in mind when creating the design. I'm happy that the chart doesn't contain little information but is easy to understand immediately. I think the color scheme is good, and the entire data visualization gives a sober newspaper feel.

How I Extract Data From My Medium Stories

Oscar Leo — Wed, 06 Mar 2024 15:59:35 +0000

Hi, my name is Oscar Leo, and I'm learning about technology, entrepreneurship, and online content creation.

It's a lot of fun, and if you agree, you should have a look at my free newsletter! 😄

In this post, I want to show you how I extract and use data from my stories on Medium.com.

Step 1: Go to stats and select a story

First, you choose a story of interest on your stats page. When you do, you only see the data from the current month, but you can change that using the monthly dropdown.

Next, you open the developer tools (for example, by right-clicking anywhere on the page and clicking select).

If you change the selected month and go to Network → GraphQL → Response, you can see the JSON data from the server.

I have a folder for each story I want to look at, and I create a JSON file for each month, as shown in the screenshot below.

It's a bit too manual, but it's good enough since historical data doesn't change.

Step 2: Reading the data in Python

I use Python to look at the data, and the first step is to utilize the following function that lists all JSON files in my story folder.



def get_files(story_folder):
    files = os.listdir(story_folder)
    files = [f for f in files if ".json" in f]
    files.sort()
    return files

I have one function to structure the earnings from the JSON data that looks like this.



def get_earnings(data):
    earnings = data["post"]["earnings"]["dailyEarnings"]
    earnings = [(
        str(pd.to_datetime(e["periodStartedAt"], unit="ms").date()), e["amount"]
    ) for e in earnings]

    return earnings

And one that works for all other stats.



def get_stats(data):
    stats = {}

    for d in data["postStatsDailyBundle"]["buckets"]:
        date = str(pd.to_datetime(d["dayStartsAt"], unit="ms").date())

        if date not in stats.keys():
            stats[date] = {"member": {}, "nonmember": {}}

        stats[date][d["membershipType"].lower()] = {
            "readersThatRead": d["readersThatReadCount"],
            "readersThatViewed": d["readersThatViewedCount"],
            "readersThatClapped": d["readersThatClappedCount"],
            "readersThatReplied": d["readersThatRepliedCount"],
            "readersThatHighlighted": d["readersThatHighlightedCount"],
            "readersThatFollowed": d["readersThatInitiallyFollowedAuthorFromThisPostCount"],
        }

    return stats

I call these three functions using the following primary function, which returns a dictionary with all the data for that story.



def get_story_stats(story_folder):
    files = get_files(story_folder)
    story_stats = {}

    for file in files:
        with open(story_folder + file) as f:
            data = json.load(f)[0]["data"]
            story_stats = {**story_stats, **get_stats(data)}
            for date, amount in get_earnings(data):
                story_stats[date]["earning"] = amount

    return story_stats

Here's what it looks like in my Jupyter Notebook.

Now, I run my analysis or create a data visualization.

Example: Visualizing my story data

I will cover the code for creating my story data visualization in another post, but here's what it looks like.

I have created visualizations like this one for my most successful stories, allowing me to detect patterns and differences quickly.

Conclusion

With a few minutes of manual labor, you can extract data from your stories quickly (until there's a better alternative).

With access to raw data, you can better understand why some of your stories perform better than others.

And you can create beautiful data visualizations that tell you everything you need to know in one look.

That's it, see you next time.

Matplotlib Tutorial: Let’s Take Your Country Maps to Another Level

Oscar Leo — Fri, 29 Sep 2023 07:18:32 +0000

In this tutorial, I will create a reusable and flexible function that allows me to instantly draw a beautiful map of any area.

With such a function, I can save a ton of time when I want to create charts or infographics with a geographical component.

I also want to show the power of reusable code since many programmers working with data visualization forget about such best practices.

If you're a Medium, you can find the tutorial there as well. I would also love to see you at my free newsletter, Data Wonder, and on Twitter.

With that said, let’s get started.

Step 1: Download geo data

The only thing you need to do before you can start the tutorial is to download geo data from here:

https://datacatalog.worldbank.org/search/dataset/0038272/World-Bank-Official-Boundaries

I use the one called World Boundaries GeoJSON — Very High Resolution.

It’s the official boundaries from the World Bank and a public dataset you can use however you want.

Step 2: Import libraries

As usual, we start by importing the necessary libraries, and we don’t need many. Since we have geographical data, we want geopandas to make plotting as easy as possible.

import numpy as np
import pandas as pd
import seaborn as sns
import geopandas as gpd
import matplotlib.pyplot as plt

import matplotlib.patheffects as PathEffects
from matplotlib.patches import Polygon

One import that you might have yet to see is PathEffects. I will use that to create a border around the country labels later.

Step 3: Create a seaborn style

Before plotting, I always create a seaborn style to get a coherent look. Here, I only define a background_color, font_family, and text_color. I’m setting the background to light blue to represent the ocean.

font_family = "sans"
background_color = "#D4F1F4"
text_color = "#040303"

sns.set_style({
    "axes.facecolor": background_color,
    "figure.facecolor": background_color,
    "font.family": font_family,
    "text.color": text_color,
})

You can style other chart aspects, such as the grid, but I prefer to hide most standard chart components when drawing maps with plt.axis(“off”).

Step 4: Load geo data

Now it’s time to load the geo-data from the World Bank using geopandas. I’m changing the CONTINENT of Seychelles and INCOME_GRP of Equatorial Guinea.

It’s not as strange as it looks because Seychelles is a part of Africa, and Equatorial Guinea is in the “Upper middle income” group according to the latest data from the World Bank.

world = gpd.read_file("WB_Boundaries_GeoJSON_highres/WB_countries_Admin0.geojson")
world.loc[world.NAME_EN == "Seychelles", "CONTINENT"] = "Africa"
world.loc[world.NAME_EN == "Equatorial Guinea", "INCOME_GRP"] = "3. Upper middle income"

africa = world[world.CONTINENT == "Africa"].reset_index(drop=True)

disputed_areas = gpd.read_file("WB_Boundaries_GeoJSON_highres/WB_Admin0_disputed_areas.geojson")
disputed_areas = disputed_areas[disputed_areas.CONTINENT == "Africa"]

Next, I separate the African countries because I want to plot them separately as the focus of my chart, and I do the same for disputed areas.

Step 5: Creating the draw map function

Now, I’m ready to create the first version of the function that will draw the map. I will extend it with some additional functions later on.

This first version loops over a list of geopandas data frames and plots them using the color and edgecolor columns.

The function takes a few parameters I don’t use yet, but they will be helpful later in the tutorial.

def draw_map(
    maps_to_draw, 
    boundry_map_index=0,
    use_hatch_for_indexes=[],
    padding={},
    labels=[],
    figsize=(40, 40)
):

    assert "color" in maps_to_draw[0].columns, "Missing color column in map dataframe"
    assert "edgecolor" in maps_to_draw[0].columns, "Missing edgecolor column in map dataframe"

    fig = plt.figure(figsize=figsize)
    ax = fig.add_subplot()

    for map_index, map_to_draw in enumerate(maps_to_draw):
        map_to_draw.plot(
            ax=ax, color=map_to_draw.color, edgecolor=map_to_draw.edgecolor,
            hatch="//" if map_index in use_hatch_for_indexes else "",
        )

    # Additional functions below this comment

    return ax

I want draw_map() to return the ax object because I often want to extend the chart with additional information based on the use case.

To use draw_map(), I must define the color and edgecolor columns for each geopandas data frame I want to plot.

I define the colors for world to draw every country as a faint background. For africa, I select bolder colors using the three income groups represented in the data.

One tip when selecting colors is to check your colors for color blindness using Coloring for Colorblindness.

world["color"] = "#f0f0f0"
world["edgecolor"] = "#c0c0c0"

africa["edgecolor"] = "#000000"
africa.loc[africa.INCOME_GRP == "5. Low income", "color"] = "#dadada"
africa.loc[africa.INCOME_GRP == "4. Lower middle income", "color"] = "#89bab2"
africa.loc[africa.INCOME_GRP == "3. Upper middle income", "color"] = "#1B998B"

disputed_areas["color"] = "#FFD6D6"
disputed_areas["edgecolor"] = "#000000"

Now that I have created the required columns, I can run draw_map().

ax = draw_map(maps_to_draw=[world, africa, disputed_areas])

plt.axis("off")
plt.show()

Here’s what I get.

It’s a great start, but we have a long way to go.

Step 6: Setting the boundaries

Since I want to create a map of Africa, it doesn’t make sense to show the entire world.

That’s why I created the following function that takes one of the geopandas data frames and sets the limits of the ax object.

def set_limits(ax, data, pad_left=0, pad_right=0, pad_top=0, pad_bottom=0):
    xmin_ = data.bounds.minx.min()
    ymin_ = data.bounds.miny.min()
    xmax_ = data.bounds.maxx.max()
    ymax_ = data.bounds.maxy.max()

    xmin = xmin_ - pad_left * (xmax_ - xmin_)
    xmax = xmax_ + pad_right * (xmax_ - xmin_)
    ymin = ymin_ - pad_bottom * (ymax_ - ymin_)
    ymax = ymax_ + pad_top * (ymax_ - ymin_)

    ax.set(xlim=(xmin, xmax), ylim=(ymin, ymax))

You can add padding to each side of the area to get some space for additional information, such as a legend.

Let’s add it to the draw_map() function using the boundary_map_index and padding parameters.

def draw_map(
    maps_to_draw, 
    boundry_map_index=0,
    use_hatch_for_indexes=[],
    padding={},
    labels=[],
    figsize=(40, 40)
):

    ...
    # Additional functions below this comment
    set_limits(ax, maps_to_draw[boundry_map_index], **padding)

    return ax

I set boundry_map_index to the index of the geopandas data frame in the maps_to_draw list that we want to show and added some padding.

Note: I also passed use_hatch_for_indexes=[2] to draw hatches over the disputed areas to show that they differ from the rest.

ax = draw_map(
    maps_to_draw=[world, africa, disputed_areas], boundry_map_index=1,
    padding={"pad_bottom": -0.08, "pad_top": 0.07, "pad_left": 0.07, "pad_right": 0.05},
    use_hatch_for_indexes=[2]
)

plt.axis("off")
plt.show()

Now we get a nice-looking map of Africa with the contours of nearby countries as well. If you don’t want the nearby countries to show, remove world from maps_to_draw.

Here’s the resulting map.

We’re getting somewhere.

Step 7: Adding country labels

Since most people don’t know all African countries, the next step is to add labels for each country.

I want to add a label close to the middle for large countries and a line between the country and the label for smaller ones.

def add_label(ax, label, fontsize=24, fontweight="bold", va="center", ha="center"):            
    annotation = plt.annotate(
        label["label"], 
        xy=label["xytext"] if "xypin" not in label.keys() else label["xypin"], 
        xytext=None if "xypin" not in label.keys() else label["xytext"], 
        xycoords="data", fontsize=fontsize, va=va, ha=ha,
        linespacing=1.3, color=label["color"], fontweight=fontweight, 
        arrowprops={
            "arrowstyle": "-",
            "linewidth": 2,
        })

    annotation.set_path_effects([PathEffects.withStroke(linewidth=6, foreground='w')])

Defining the position of the country labels is the only tedious process of this tutorial, but I’ve done that for you.

You can try to calculate the position for each label using the geometries in geopandas, but that’s easier said than done since the shape of the countries varies.

Sometimes, the fastest method is to roll up your sleeves and hard-code the values, which I have done here.

country_labels = [
    {"label": "Algeria", "color": "#040303", "xytext": (2.0, 27.5)},
    {"label": "Angola", "color": "#040303", "xytext": (17.7, -13.1)},
    {"label": "Benin", "color": "#040303", "xytext": (3.2, 5.4), "xypin": (2.3, 7.6)},
    {"label": "Botswana", "color": "#040303", "xytext": (24.4, -22.3)},
    {"label": "Burkina\nFaso", "color": "#040303", "xytext": (-1.4, 12.6)},
    {"label": "Burundi", "color": "#040303", "xytext": (43.3, -4.9), "xypin": (29.8, -3.6)},
    {"label": "Cameroon", "color": "#040303", "xytext": (12.5, 5.2)},
    {"label": "Cape Verde", "color": "#040303", "xytext": (-23.7, 19), "xypin": (-23.7, 16)},
    {"label": "Central African\nRepublic", "color": "#040303", "xytext": (21.1, 6.5)},
    {"label": "Chad", "color": "#040303", "xytext": (18.5, 16.0)},
    {"label": "Comoros", "color": "#040303", "xytext": (46.8, -9.6), "xypin": (43.3, -11.7)},
    {"label": "Cote\nd'Ivoire", "color": "#040303", "xytext": (-5.5, 8.5)},
    {"label": "Democratic\nRepublic of\nthe Congo", "color": "#040303", "xytext": (23.3, -2.7)},
    {"label": "Djibouti", "color": "#040303", "xytext": (47.0, 13.4), "xypin": (43.0, 12.2)},
    {"label": "Egypt", "color": "#040303", "xytext": (29.2, 26.6)},
    {"label": "Equatorial\nGuinea", "color": "#040303", "xytext": (5.9, -2.5), "xypin": (10.5, 1.6)},
    {"label": "Eritrea", "color": "#040303", "xytext": (43.0, 16.9), "xypin": (38.5, 16.2)},
    {"label": "Lesotho", "color": "#040303", "xytext": (35.0, -31.0), "xypin": (28.4, -29.5)},
    {"label": "Ethiopia", "color": "#040303", "xytext": (39.9, 8.5)},
    {"label": "Gabon", "color": "#040303", "xytext": (11.8, -0.7)},
    {"label": "Ghana", "color": "#040303", "xytext": (-1.3, 6.6)},
    {"label": "Guinea", "color": "#040303", "xytext": (-11.6, 11.0)},
    {"label": "Guinea-\nBissau", "color": "#040303", "xytext": (-20.3, 10.3), "xypin": (-14.5, 12.2)},
    {"label": "Kenya", "color": "#040303", "xytext": (37.9, 0.5)},
    {"label": "Eswantini", "color": "#040303", "xytext": (35.5, -29.3), "xypin": (31.5, -26.8)},
    {"label": "Liberia", "color": "#040303", "xytext": (-10.6, 3.6), "xypin": (-9.6, 6.7)},
    {"label": "Libya", "color": "#040303", "xytext": (17.5, 27.5)},
    {"label": "Madagascar", "color": "#040303", "xytext": (46.7, -19.6)},
    {"label": "Malawi", "color": "#040303", "xytext": (38.9, -21.3), "xypin": (35.0, -15.6)},
    {"label": "Mali", "color": "#040303", "xytext": (-1.9, 17.8)},
    {"label": "Mauritania", "color": "#040303", "xytext": (-11.1, 19.6)},
    {"label": "Morocco", "color": "#040303", "xytext": (-6.9, 31.3)},
    {"label": "Mozambique", "color": "#040303", "xytext": (40.8, -15.2)},
    {"label": "Namibia", "color": "#040303", "xytext": (17.3, -20.7)},
    {"label": "Niger", "color": "#040303", "xytext": (9.8, 17.5)},
    {"label": "Nigera", "color": "#040303", "xytext": (7.8, 9.8)},
    {"label": "Republic of\nthe Congo", "color": "#040303", "xytext": (7.8, -7.2), "xypin": (12.0, -4.1)},
    {"label": "Rwanda", "color": "#040303", "xytext": (43.8, -3.6), "xypin": (30.1, -2.0)},
    {"label": "São Tomé and\nPríncipe", "color": "#040303", "xytext": (-0.9, 0.2), "xypin": (6.8, 0.2)},
    {"label": "Senegal", "color": "#040303", "xytext": (-15.0, 14.7)},
    {"label": "Seychelles", "color": "#040303", "xytext": (55.6, -2), "xypin": (55.6, -4.5)},
    {"label": "Sierra Leone", "color": "#040303", "xytext": (-16.4, 6.3), "xypin": (-12.0, 8.5)},
    {"label": "Somalia", "color": "#040303", "xytext": (45.7, 2.7)},
    {"label": "South\nAfrica", "color": "#040303", "xytext": (22.4, -31.0)},
    {"label": "South\nSudan", "color": "#040303", "xytext": (30.2, 7.0)},
    {"label": "Sudan", "color": "#040303", "xytext": (29.7, 16.0)},
    {"label": "Tanzania", "color": "#040303", "xytext": (35.0, -6.7)},
    {"label": "The\nGambia", "color": "#040303", "xytext": (-20.3, 13.6), "xypin": (-15.4, 13.6)},
    {"label": "Togo", "color": "#040303", "xytext": (1.0, 4.1), "xypin": (1.0, 7.5)},
    {"label": "Tunisia", "color": "#040303", "xytext": (9.3, 38.9), "xypin": (9.3, 35.7)},
    {"label": "Uganda", "color": "#040303", "xytext": (32.6, 0.9)},
    {"label": "Zambia", "color": "#040303", "xytext": (26.1, -14.9)},
    {"label": "Zimbawe", "color": "#040303", "xytext": (29.7, -19.1)},
]

We add the function directly below set_limits.

def draw_map(
    maps_to_draw, 
    boundry_map_index=0,
    use_hatch_for_indexes=[],
    padding={},
    labels=[],
    figsize=(40, 40)
):

    ...
    # Additional functions below this comment
    set_limits(ax, maps_to_draw[boundry_map_index], **padding)

    for label in labels:
        add_label(ax, label)

    return ax

And pass it to draw_map().

ax = draw_map(
    maps_to_draw=[world, africa, disputed_areas], boundry_map_index=1,
    padding={"pad_bottom": -0.08, "pad_top": 0.07, "pad_left": 0.07, "pad_right": 0.05},
    use_hatch_for_indexes=[2]
    labels=country_labels
)

plt.axis("off")
plt.show()

Here’s what we get.

Fantastic! Now, I can see what countries I’m looking at, and the last thing to do is to explain the colors.

Step 8: Adding a legend and title

Adding legends in matplotlib is simple, so to spice things up, I want to use the shape of the countries instead of a standard circle or square.

This step is an example where I use the ax that draw_map() returns instead of adding it directly to the function.

To take the geometry of a country and draw it in a different place, I’ve created the following function that takes a row and puts the shape in (x_loc, y_loc).

def draw_legend_geometry(ax, row, x_loc, y_loc, height):
    x = np.array(row.geometry.boundary.coords.xy[0])
    y = np.array(row.geometry.boundary.coords.xy[1])

    x = x - (row.geometry.centroid.x - x_loc)
    y = y - (row.geometry.centroid.y - y_loc)

    ratio = height / (y.max() - y.min())
    x = x * ratio + (x_loc - x_loc * ratio)
    y = y * ratio + (y_loc - y_loc * ratio)

    ax.add_artist(Polygon(np.stack([x, y], axis=1), facecolor=row.color, edgecolor=row.edgecolor, hatch=row.hatch))

In addition to changing the location, it also changes the scale of the geometry to have a specific height.

To use draw_legend_geometry(), I create a data frame called legend, where I select appropriate countries to represent each color. Currently, it only works for countries with one Polygon, not MultiPolygon.

It’s a bit “hacky” how I add INCOME_GRP to Western Sahara, use that to sort values, and add row.INCOME_GRP[3:] as text, but it’s ok for now.

I’ve also added a title and data source after drawing the legend.

ax = draw_map(
    maps_to_draw=[world, africa, disputed_areas], boundry_map_index=1,
    padding={"pad_bottom": -0.08, "pad_top": 0.07, "pad_left": 0.07, "pad_right": 0.05},
    use_hatch_for_indexes=[2]
    labels=country_labels,
)

legend = pd.concat([
    disputed_areas[disputed_areas.NAME_EN == "Western Sahara"],
    africa[africa.NAME_EN.isin(["Niger", "Senegal", "Botswana"])]
])

legend.loc[legend.NAME_EN == "Western Sahara", "INCOME_GRP"] = "6. Disputed area"
legend = legend.sort_values("INCOME_GRP")
legend["hatch"] = ["", "", "", "//"]

for i, row in legend.reset_index().iterrows():
    draw_legend_geometry(ax, row, -25, -20 - 3.5*i, 2.5)
    ax.annotate(row.INCOME_GRP[3:], (-22, -20 - 3.5*i), fontsize=28, fontweight="bold", va="center")

fontstyles = {"fontweight": "bold", "ha": "left"}
plt.annotate("Data source:", xy=(0.05, 0.32), fontsize=24, xycoords="axes fraction", **fontstyles)
plt.annotate("The World Bank", xy=(0.133, 0.32), fontsize=24, xycoords="axes fraction", color="#1B998B", **fontstyles)
plt.title("Income Groups in Africa", x=0.05, y=0.29, fontsize=42, **fontstyles)

plt.axis("off")
plt.show()

If I run the code above, I get the following map, and it’s identical to the one you saw at the start of the tutorial.

That was the last step of the tutorial. Congratulations on making it to the end!

Conclusion

You can make the draw_map() function even more flexible by adding more parameters for styling (I have several hard-coded values), but for me, this covers 95% of my use cases.

I hope you enjoyed the tutorial and learned something you can use in your projects.

If you did, like, subscribe, and share so that more people can learn how to draw beautiful maps with Python and Matplotlib.

Thank you for reading, and see you next time.

How to Create Data Maps of the United States With Matplotlib

Oscar Leo — Tue, 05 Sep 2023 18:23:00 +0000

Hello, and welcome to this tutorial.

Today, I will teach you to create the map that you see above using geo-data and the Social Connectivity Index.

If you want to know more about the visualization and dataset, you can take a look at this article in my new free newsletter, Data Wonder.

Let’s get started with the tutorial.

Step 1: Download data

Before we begin, we need to download a dataset exciting enough for this tutorial and geo-data to draw accurate maps of the United States.

For the maps, I’m using shape files from Cencus.gov. You can use the following links to download both states and counties.

To have a complementary dataset, I’ve selected the Facebook Connectivity Index, which measures the likelihood that two people in different counties are connected on Facebook.

You can download the connectivity data using this link.

Once the downloads have finished, unzip them and put them in a good location. I’m using ./data in the tutorial, but you can do whatever you like.

It should look something like this.

Let’s write some code.

Step 2: Import libraries and prepare Seaborn

The only new library (if you’ve done any of my other Matplotlib Tutorials) is geopandas, which we will use to draw maps.

# Import libraries

import numpy as np
import pandas as pd
import seaborn as sns
import geopandas as gpd
import matplotlib.pyplot as plt

from PIL import Image
from matplotlib.patches import Patch, Circle

Next, let’s define a few features about the style using seaborn.

edge_color = "#30011E"
background_color = "#fafafa"

sns.set_style({
    "font.family": "serif",
    "figure.facecolor": background_color,
    "axes.facecolor": background_color,
})

Now it’s time to learn how to draw a map.

Step 3: Load and prepare geo-data

I use geopandas to load the data and remove “unincorporated territories” such as Guam, Puerto Rico, and American Samoa.

# Load and prepare geo-data
counties = gpd.read_file("./data/cb_2018_us_county_500k/")
counties = counties[~counties.STATEFP.isin(["72", "69", "60", "66", "78"])]
counties = counties.set_index("GEOID")

states = gpd.read_file("./data/cb_2018_us_state_500k/")
states = states[~states.STATEFP.isin(["72", "69", "60", "66", "78"])]

A geopandas data frame has a geometry column that defines the shape of each row. It allows us to draw a map by calling counties.plot() or states.plot() like this.

ax = counties.plot(edgecolor=edge_color + "55", color="None", figsize=(20, 20))
states.plot(ax=ax, edgecolor=edge_color, color="None", linewidth=1)

plt.axis("off")
plt.show()

Here, I start by drawing the counties with transparent borders, and then I reuse ax when I call states.plot() so that I don’t draw separate maps.

This is what I get.

The results don’t look great, but I will make a few quick adjustments to get us on the right track.

The first adjustment is to change the map projection to one centered on the United States. You can do that with geopandas by calling to_crs().

# Load and prepare geo-data
...

counties = counties.to_crs("ESRI:102003")
states = states.to_crs("ESRI:102003")

Here’s the difference.

It’s common to draw Alaska and Hawaii underneath the mainland when drawing data maps of the United States, and that’s what we will do as well.

With geopandas, you can translate, scale, and rotate geometries with built-in functions. Here’s a helpful function to do that.

def translate_geometries(df, x, y, scale, rotate):
    df.loc[:, "geometry"] = df.geometry.translate(yoff=y, xoff=x)
    center = df.dissolve().centroid.iloc[0]
    df.loc[:, "geometry"] = df.geometry.scale(xfact=scale, yfact=scale, origin=center)
    df.loc[:, "geometry"] = df.geometry.rotate(rotate, origin=center)
    return df

I calculate a center point for the entire data frame that defines the origin of rotation and scaling. If I don’t, geopandas does that automatically for each row, which makes the map look completely messed up.

This next function takes our current data frames, separates Hawaii and Alaska, calls translate_geometries() to adjust their geometries, and put them back into new data frames.

def adjust_maps(df):
    df_main_land = df[~df.STATEFP.isin(["02", "15"])]
    df_alaska = df[df.STATEFP == "02"]
    df_hawaii = df[df.STATEFP == "15"]

    df_alaska = translate_geometries(df_alaska, 1300000, -4900000, 0.5, 32)
    df_hawaii = translate_geometries(df_hawaii, 5400000, -1500000, 1, 24)

    return pd.concat([df_main_land, df_alaska, df_hawaii])

We add adjust_maps() to our code.

# Load and prepare geo-data
...

counties = adjust_maps(counties)
states = adjust_maps(states)

And our map now looks like this.

Time for the next step.

Step 4: Adding data

To add data, we start by loading the Facebook connectivity data. I’m turning the user_loc and fr_loc columns into strings and adding leading zeros to make them consistent with the geo data.

# Load facebook data
facebook_df = pd.read_csv("./data/county_county.tsv", sep="\t")
facebook_df.user_loc = ("0" + facebook_df.user_loc.astype(str)).str[-5:]
facebook_df.fr_loc = ("0" + facebook_df.fr_loc.astype(str)).str[-5:]

The user_loc and fr_loc columns define a county pair, and the third column, scaled_sci, is the value we want to display.

There are 3,227 counties in the dataset, which means there are a total of 10,413,529 pairs, but we will show the connectivity indexes for one county at a time.

# Create data map
county_id = "06075" # San Fransisco
county_name = counties.loc[county_id].NAME
county_facebook_df = facebook_df[facebook_df.user_loc == county_id]

Next, I define a selected_color and data_breaks which contains percentiles, colors, and legend texts for later.

# Create data map
...

selected_color = "#FA26A0"
data_breaks = [
    (90, "#00ffff", "Top 10%"),
    (70, "#00b5ff", "90-70%"),
    (50, "#6784ff", "70-50%"),
    (30, "#aeb3fe", "50-30%"),
    (0, "#e6e5fc", "Bottom 30%"),
]

The following function defines the color for each row using a county_df and the data_breaks we just defined.

def create_color(county_df, data_breaks):
    colors = []

    for i, row in county_df.iterrows():
        for p, c, _ in data_breaks:
            if row.value >= np.percentile(county_df.value, p):
                colors.append(c)
                break

    return colors

We calculate the correct values and add create_color() like this.

# Create data map
...

counties.loc[:, "value"] = (county_facebook_df.set_index("fr_loc").scaled_sci)
counties.loc[:, "value"] = counties["value"].fillna(0)
counties.loc[:, "color"] = create_color(counties, data_breaks)
counties.loc[county_id, "color"] = selected_color

ax = counties.plot(edgecolor=edge_color + "55", color=counties.color, figsize=(20, 20))
states.plot(ax=ax, edgecolor=edge_color, color="None", linewidth=1)
ax.set(xlim=(-2600000, None)) # Removing some of the padding to the left

plt.axis("off")
plt.show()

Here’s what we get.

It looks fantastic, but we need to add some information.

Step 5: Adding information

The first piece of information we need is a title to explain what the data visualization is about.

Here’s a function that does that.

def add_title(county_id, county_name):
    plt.annotate(
        text="Social Connectedness Ranking Between US Counties and",
        xy=(0.5, 1.1), xycoords="axes fraction", fontsize=16, ha="center"
    )

    plt.annotate(
        text="{} (FIPS Code {})".format(county_name, county_id), 
        xy=(0.5, 1.03), xycoords="axes fraction", fontsize=32, ha="center",
        fontweight="bold"
    )

Next, we need a legend and supporting information that explains the data since it’s a bit complex.

The function for adding a legend uses the data_breaks and the selected_color to create Patch(es) that we add using plt.legend().

def add_legend(data_breaks, selected_color, county_name):
    patches = [Patch(facecolor=c, edgecolor=edge_color, label=t) for _, c, t in data_breaks]
    patches = [Patch(facecolor=selected_color, edgecolor=edge_color, label=county_name)] + patches

    leg = plt.legend(
        handles=patches,
        bbox_to_anchor=(0.5, -0.03), loc='center',
        ncol=10, fontsize=20, columnspacing=1,
        handlelength=1, handleheight=1,
        edgecolor=background_color,
        handletextpad=0.4
    )

I also have a simple function to add some additional information below the legend.

def add_information():
    plt.annotate(
        "The Facebook Connectivity Index measure the likelyhood that users in different\nlocations are connected on Facebook. The formula divides the number of Facebook\nconnections with the number of possible connections for the two locations.",
        xy=(0.5, -0.08), xycoords="axes fraction", ha="center", va="top", fontsize=18, linespacing=1.8
    )

    plt.annotate(
        "Source: https://dataforgood.facebook.com/", 
        xy=(0.5, -0.22), xycoords="axes fraction", fontsize=16, ha="center",
        fontweight="bold"
    )

Lastly, I have the add_circle() function to indicate which county we’re looking at by drawing a circle around it.

def add_circle(ax, counties_df, county_id):
    center = counties_df[counties_df.index == county_id].geometry.centroid.iloc[0]
    ax.add_artist(
        Circle(
            radius=100000, xy=(center.x, center.y), facecolor="None", edgecolor=selected_color, linewidth=4
        )
    )

We add all of them below the rest of the code under the # Create data map comment.

# Create data map
...

add_circle(ax, counties, county_id)
add_title(county_id, county_name)
add_legend(data_breaks, selected_color, county_name)
add_information()

plt.axis("off")
plt.show()

Here’s the finished data visualization.

Congratulations, you now know how to create fantastic data maps of the United States in Matplotlib! :)

Conclusion

Data maps are fantastic when you want to visualize geographic information in a way that captures the user’s eye.

This time, we worked with the Social Connectedness Index from Facebook, but you can change that to anything else with geographic information.

I hope you enjoyed this tutorial and learned something new.

If you did, make sure to join my newsletter, Data Wonder.

See you next time.

How to Create a Lipstick Chart With Matplotlib

Oscar Leo — Thu, 31 Aug 2023 19:09:48 +0000

Today, I will show you how to create a lipstick chart for visualizing progress on metrics where the lower the value, the better.

I've prepared a simple dataset about mortality and diseases, so that you can focus on creating the visualization.

The data comes from the World Bank, and if you want to learn more, I've written about the visualization in my new free newsletter Data Wonder.

Let's get started.

Step 1 - Importing libraries

The first and simplest part is to import the required libraries like pandas and matplotlib.

import numpy as np
import pandas as pd

import seaborn as sns
import matplotlib.pyplot as plt

from PIL import Image
from matplotlib.lines import Line2D

Congratulation, you just completed step 1! 🥳

Step 2 - Create a Seaborn style

Next, I want to create a color-scheme and select a font. Sites like Coolors and Colorhunt are great resources when searching for beautiful colors.

Here's my code and settings to create the seaborn style for this tutorial.

FONT_FAMILY = "serif"
BACKGROUND_COLOR = "#FAE8E0"
TEXT_COLOR = "#33261D"
BAR_COLOR = "#EF7C8E"

sns.set_style({
    "axes.facecolor": BACKGROUND_COLOR,
    "figure.facecolor": BACKGROUND_COLOR,

    "text.color": TEXT_COLOR,
    "font.family": FONT_FAMILY,

    "xtick.bottom": False,
    "xtick.top": False,
    "ytick.left": False,
    "ytick.right": False,

    "axes.spines.left": False,
    "axes.spines.bottom": False,
    "axes.spines.right": False,
    "axes.spines.top": False,
})

I'm removing all the ticks and lines to create a clean visualization, and the grids don't add any valuable information to our lipstick chart.

Step 3 - Reading the data

You can read the CSV directly from the url as I do in the code below.

df = pd.read_csv(
    "https://raw.githubusercontent.com/oscarleoo/matplotlib-tutorial-data/main/mortality-and-decease.csv"
)

Here's what the dataframe looks like.

Most values are self-explanatory except for per, which shows each row's scale. For example, the latest "Maternal mortality" value was 223 out of 100,000 births.

Step 4 - Adding bars

Now it's time to add some data.

I'm adding bars for both 2000 and the latest values. Since my goal is to show the relative decrease for each value, I'm dividing each row by its 2000 value.

That means each bar for 2000 will reach 1, so it's only a visual helper and doesn't add any additional information.

Here's my function to add bars.

def add_bars(ax, x, width, alpha, label):
    sns.barplot(
        ax=ax, x=x, y=[i for i in range(len(x))], label=label,
        width=width, alpha=alpha,
        color=BAR_COLOR,
        edgecolor=TEXT_COLOR,
        orient="h"
    )

I create a figure and run the add_bars() function like this.

fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(18, 2.7 * len(df)))

add_bars(
    ax=ax, x=df["2000"] / df["2000"],
    width=0.55, alpha=0.2, label="2000"
)

add_bars(
    ax=ax, x=df["latest_value"] / df["2000"],
    width=0.7, alpha=1, label="Latest"
)

The result for the code we have so far looks like this.

Let's continue.

Step 5 - Formatting the axes

The names for each row are to long to use without line-breaks. That's why I created the following function to add \n to the strings in a few places.

def split_name(name, limit=20):
    split = name.split()
    s = ""

    for s_ in split:
        if len(s.split("\n")[-1] + s_) > limit:
            s += "\n" + s_
        else:
            s += " " + s_

    return s.strip()

I also want increase the fontsize and remove unnecessary information to make the chart readable. The code to create the visualization now looks like this.

fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(18, 2.7 * len(df)))
...

ax.set(xlabel=None, ylabel=None, xticks=[])
ax.tick_params("y", labelsize=28, pad=32)
ax.tick_params("x", labelsize=20, pad=16)

ax.set_yticks(
    ticks=[i for i in range(len(df))],
    labels=[split_name(n, limit=19) for n in df["indicator_name"]],
    linespacing=1.7, va="center"
)

Here's the updated results.

Let's add some additional information.

Step 5 - Adding useful information

You always want to make sure that the users understand what they are looking at. Right now, we have no such information.

For starters, I want to add the current values, which I do with the following function.

def add_info_text(ax, row, index):
    value = round(row["latest_value"], 1)
    per = row["per"]
    year = row["latest_year"]
    text = "{:,} out of\n{:,} ({})".format(value, per, year)

    ax.annotate(
        text=text, 
        xy=(0.02, index), 
        color="#fff", 
        fontsize=24,
        va="center", 
        linespacing=1.7
    )

And since the purpose is to show the relative decrease of each metric compared to its value from 2000, I have another function showing the change for each row.

def add_change_text(ax, row, index):
    change = round(100 * row["change"], 1)
    text = "{:,}%".format(change)
    x = row["latest_value"] / row["2000"] + 0.02

    ax.annotate(
        text="{:,}%".format(change), xy=(x, index), fontsize=22,
        va="center",  linespacing=1.7
    )

I add both functions under a for-loop.

fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(18, 2.7 * len(df)))
...

for index, row in df.reset_index().iterrows():
    add_info_text(ax, row, index)
    add_change_text(ax, row, index)

Here's the output.

It's starting to look good.

Step 6 - Adding a title and legend

In this step, I'm simply using some built in Matplotlib functions to add a title and legend. Since we defined label in add_bars() much of the styling is automatic.

Apart from defining a title and legend, I'm also adding a border using Line2D for visual effect.

fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(18, 2.7 * len(df)))
...

line = Line2D([-0.33, 1.0], [-0.9, -0.9], color=TEXT_COLOR)
line.set_clip_on(False)
ax.add_artist(line)

title = "Lipstick Chart - Relative\nDecreases Compared\nto 2000"
plt.title(title, x=-0.32, y=1.11, fontsize=58, ha="left", linespacing=1.6)
plt.legend(bbox_to_anchor=(0.75, 1.14), loc='lower center', borderaxespad=0, ncol=1, fontsize=44, edgecolor="#FAE8E0")

Here's what the chart looks like now.

Step 7 - Creating an image and adding padding

The chart looks a bit cramped, so the last step is to add some padding. I'm doing that by turning the figure into a PIL Image with the following function.

def create_image_from_figure(fig):
    plt.tight_layout()

    fig.canvas.draw()
    data = np.frombuffer(fig.canvas.tostring_rgb(), dtype=np.uint8)
    data = data.reshape((fig.canvas.get_width_height()[::-1]) + (3,))
    plt.close() 

    return Image.fromarray(data)

And here's the function to add padding.

def add_padding_to_chart(chart, left, top, right, bottom, background):
    size = chart.size
    image = Image.new("RGB", (size[0] + left + right, size[1] + top + bottom), background)
    image.paste(chart, (left, top))
    return image

We have now written all code required to create the data visualization we aimed for.

Here's the full code snippet that uses all functions to create the final lipstick chart.

fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(18, 2.7 * len(df)))

add_bars(
    ax=ax, x=df["2000"] / df["2000"],
    width=0.55, alpha=0.2, label="2000"
)

add_bars(
    ax=ax, x=df["latest_value"] / df["2000"],
    width=0.7, alpha=1, label="Latest"
)

ax.set(xlabel=None, ylabel=None, xticks=[])
ax.tick_params("y", labelsize=28, pad=32)
ax.tick_params("x", labelsize=20, pad=16)

ax.set_yticks(
    ticks=[i for i in range(len(df))],
    labels=[split_name(n, limit=20) for n in df["indicator_name"]],
    linespacing=1.7, va="center"
)

for index, row in df.reset_index().iterrows():
    add_info_text(ax, row, index)
    add_change_text(ax, row, index)

line = Line2D([-0.35, 1.0], [-0.9, -0.9], color=TEXT_COLOR)
line.set_clip_on(False)
ax.add_artist(line)

title = "Lipstick Chart - Relative\nDecreases Compared\nto 2000"
plt.title(title, x=-0.32, y=1.11, fontsize=58, ha="left", linespacing=1.6)
plt.legend(bbox_to_anchor=(0.75, 1.14), loc='lower center', borderaxespad=0, ncol=1, fontsize=44, edgecolor="#FAE8E0")

image = create_image_from_figure(fig)
image = add_padding_to_chart(image, 20, 50, 10, 50, BACKGROUND_COLOR)

And here's the finished product.

We're done!

Conclusion

Thank you for reading this tutorial; I hope you learned some tricks you can reuse for your data visualization projects.

If you want to see more tutorials and beautiful data visualizations, follow me here, subscribe to Data Wonder, and to oscarl3o on Twitter.

See you next time.

How to Create Eye-Catching Country Rankings Using Python and Matplotlib

Oscar Leo — Tue, 29 Aug 2023 09:29:03 +0000

Hi, and welcome to this tutorial, where I’ll teach you to create a country ranking chart using Python and Matplotlib.

What I like about this visualization is its clean and beautiful way of showing how countries rank compared to each other on a particular metric.

The alternative to using a standard line chart showing the actual values get messy if some countries are close to each other or if some countries outperform others by a lot.

If you want access to the code for this tutorial, you can find it in this GitHub repository.

If you enjoy this tutorial, make sure to check out my other accounts.

Data Wonder on Substack
oscarl3o on Twitter
oscarleo on Medium

Let’s get started.

About the data

I’ve created a simple CSV containing GDP values for today’s ten largest economies for this tutorial.

The data comes from the World Bank, and the full name of the indicator is "GDP (constant 2015 us$)".

If you want to know more about different ways of measuring GDP, you can look at this Medium story, where I use the same type of data visualization.

Let’s get on with the tutorial.

Step 1: Creating rankings

Step one is to rank the countries for each year in the dataset, which is easy to do with pandas.



def create_rankings(df, columns):
    rank_columns = ["rank_{}".format(i) for i in range(len(columns))]
    for i, column in enumerate(columns):
        df[rank_columns[i]] = df[column].rank(ascending=False)

    return df, rank_columns

The resulting columns look like this.

That’s all the preprocessing we need to continue with the data visualization.

Step 2: Creating and styling a grid

Now that we have prepared our data, it’s time to create a grid where we can draw our lines and flags.

Here’s a function using Seaborn that creates the overall style. It defines things like the background color and font family. I’m also removing spines and ticks.



def set_style(font_family, background_color, grid_color, text_color):
    sns.set_style({
        "axes.facecolor": background_color,
        "figure.facecolor": background_color,

        "axes.grid": True,
        "axes.axisbelow": True,

        "grid.color": grid_color,

        "text.color": text_color,
        "font.family": font_family,

        "xtick.bottom": False,
        "xtick.top": False,
        "ytick.left": False,
        "ytick.right": False,

        "axes.spines.left": False,
        "axes.spines.bottom": False,
        "axes.spines.right": False,
        "axes.spines.top": False,
    }
)

I run the function with the following values.



font_family = "PT Mono"
background_color = "#FAF0F1"
text_color = "#080520"
grid_color = "#E4C9C9"

set_style(font_family, background_color, grid_color, text_color)

To create the actual grid, I have a function that formats the y- and x-axis. It takes a few parameters that allow me to try different setups, such as the size of the labels.



def format_ticks(ax, years, padx=0.25, pady=0.5, y_label_size=20, x_label_size=24):
    ax.set(xlim=(-padx, len(years) -1 + padx), ylim=(-len(df) - pady, - pady))

    xticks = [i for i in range(len(years))]
    ax.set_xticks(ticks=xticks, labels=years)

    yticks = [-i for i in range(1, len(df) + 1)]
    ylabels = ["{}".format(i) for i in range(1, len(df) + 1)]
    ax.set_yticks(ticks=yticks, labels=ylabels)

    ax.tick_params("y",labelsize=y_label_size, pad=16)
    ax.tick_params("x", labeltop=True, labelsize=x_label_size, pad=8)

Here’s what it looks like when I run everything we have so far.



# Load data
years = ["2000", "2005", "2010", "2015", "2020", "2022"]
df = pd.read_csv("rankings.csv", index_col=None)
df, rank_columns = create_rankings(df, years)

# Create chart
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(15, 1.6*len(df)))
format_ticks(ax, years)

And here’s the resulting grid.

Now we can start to add some data.

Step 3: Adding lines

I want a line showing each country's rank for each year in the dataset—an easy task in Matplotlib.



def add_line(ax, row, columns, linewidth=3):
    x = [i for i in range(len(columns))]
    y = [-row[rc] for rc in columns]

    ax.add_artist(
        Line2D(x, y, linewidth=linewidth, color=text_color)
    )

Then I run the function for each row in the dataset like this.



# Load data
years = ["2000", "2005", "2010", "2015", "2020", "2022"]
df = pd.read_csv("rankings.csv", index_col=None)
df, rank_columns = create_rankings(df, years)

# Create chart
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(15, 1.6*len(df)))
format_ticks(ax, years)

# Draw lines
for i, row in df.iterrows():
    add_line(ax, row, rank_columns)

I’m using the same color for each line because I want to use country flags to guide the eye. Using a unique color for each line makes sense, but it looks messy.

Step 4: Drawing pie charts

I want to indicate how a country’s economy grows over time without adding text. Instead, I aim to inform in a visual format.

My idea is to draw a pie chart on each point showing the size of a country’s economy compared to its best year.

I’m using PIL to create a pie chart image, but you can use Matplotlib directly. I don’t because I had some issues with aspect ratios.



def add_pie(ax, x, y, ratio, size=572, zoom=0.1):
    image = Image.new('RGBA', (size, size))
    draw = ImageDraw.Draw(image)

    draw.pieslice((0, 0, size, size), start=-90, end=360*ratio-90, fill=text_color, outline=text_color)
    im = OffsetImage(image, zoom=zoom, interpolation="lanczos", resample=True, visible=True)

    ax.add_artist(AnnotationBbox(
        im, (x, y), frameon=False,
        xycoords="data",
    ))

The value for the size parameter is slightly larger than the size of my flag images which are 512x512. Later, I want to paste the flags on the pie charts.

Here’s the updated code.



# Load data
years = ["2000", "2005", "2010", "2015", "2020", "2022"]
df = pd.read_csv("rankings.csv", index_col=None)
df, rank_columns = create_rankings(df, years)

# Create chart
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(15, 1.6*len(df)))
format_ticks(ax, years)

# Draw lines
for i, row in df.iterrows():
    add_line(ax, row, rank_columns)

    for j, rc in enumerate(rank_columns):
        add_pie(ax, j, -row[rc], ratio=row[years[j]] / row[years].max())

And here’s the result.

It’s starting to look informative, so it’s time to make it beautiful.

Step 5: Adding flags

I love using flags in my charts because they are simply beautiful.

Here, the purpose of the flags is to make the chart visually appealing, explain which countries we’re looking at, and guide the eye along the lines.

I’m using these rounded flags. They require a license, so, unfortunately, I can’t share them, but you can find similar flags in other places.

I’ve had some issues getting the pie and flag to align perfectly, so instead of creating a separate function to add a flag, I’m rewriting the add_pie() function.



def add_pie_and_flag(ax, x, y, name, ratio, size=572, zoom=0.1):
    flag = Image.open("<location>/{}.png".format(name.lower()))
    image = Image.new('RGBA', (size, size))
    draw = ImageDraw.Draw(image)
    pad = int((size - 512) / 2)

    draw.pieslice((0, 0, size, size), start=-90, end=360*ratio-90, fill=text_color, outline=text_color)
    image.paste(flag, (pad, pad), flag.split()[-1])

    im = OffsetImage(image, zoom=zoom, interpolation="lanczos", resample=True, visible=True)

    ax.add_artist(AnnotationBbox(
        im, (x, y), frameon=False,
        xycoords="data",
    ))

I add it right after the pie chart function.



# Load data
years = ["2000", "2005", "2010", "2015", "2020", "2022"]
df = pd.read_csv("rankings.csv", index_col=None)
df, rank_columns = create_rankings(df, years)

# Create chart
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(15, 1.6*len(df)))
format_ticks(ax, years)

# Draw lines
for i, row in df.iterrows():
    add_line(ax, row, rank_columns)

    for j, rc in enumerate(rank_columns):
        add_pie_and_flag(
            ax, j, -row[rc], 
            name=row.country_name,
            ratio=row[years[j]] / row[years].max()
        )

And now you can behold the visual magic of using flags. It’s a huge difference compared to the previous output.

We suddenly have something that looks nice and is easy to understand. The last thing to do is to add some helpful information.

Step 5: Adding additional information

Since not everyone knows all the flags by heart, I want to add the country’s name to the right.

I also want to show the size of the economy and how each country compares to the highest ranking.

Here’s my code for doing that.



def add_text(ax, value, max_value, y):
    trillions = round(value / 1e12, 1)
    ratio_to_max = round(100 * value / max_value, 1)

    text = "{}\n${:,}T ({}%)".format(
        row.country_name, 
        trillions,
        ratio_to_max
    )

    ax.annotate(
        text, (1.03, y), 
        fontsize=20,
        linespacing=1.7,
        va="center",
        xycoords=("axes fraction", "data")
    )

As before, I add the function to the main code block. Note that I’m also adding a title.



years = ["2000", "2005", "2010", "2015", "2020", "2022"]
df = pd.read_csv("rankings.csv", index_col=None)
df, rank_columns = create_rankings(df, years)

fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(15, 1.6*len(df)))
format_ticks(ax, years)

for i, row in df.iterrows():
    add_line(ax, row, rank_columns)

    for j, rc in enumerate(rank_columns):
        add_pie_and_flag(
            ax, j, -row[rc], 
            name=row.country_name,
            ratio=row[years[j]] / row[years].max()
        )

    add_text(ax, value=row[years[-1]], max_value=df.iloc[0][years[-1]], y=-(i + 1))
    plt.title("Comparing Today's Largest Economies\nGDP (constant 2015 us$)", linespacing=1.8, fontsize=32, x=0.58, y=1.12)

Voila.

That’s it; we’re done.

Conclusion

Today, you’ve learned an alternative way to visualize.

I like this type of data visualization because it’s easy on the eye and conveys a ton of information with very little text.

If you enjoyed it as much as I did, make sure to subscribe to my channel for more of the same! :)

Thank you for reading.

How to Create a Beautiful Polar Histogram With Python and Matplotlib

Oscar Leo — Mon, 28 Aug 2023 07:59:55 +0000

Hi, and welcome to this Python + Matplotlib tutorial, where I will show you how to create the beautiful polar histogram you see above.

Polar histograms are great when you have too many values for a standard bar chart. The circular shape where each bar gets thinner towards the middle allows us to cram more information into the same area.

I’m using data from the World Happiness Report and information about income levels from the World Bank.

You can find the code and data I’m using in this GitHub repository.

Let’s get started.

Step 1: Preparations

Let's start with a few preperations.

Importing libraries

We only need standard Python libraries familiar to everyone. PIL is not mandatory, but it’s my preferred choice for handling images which we do when adding flags.



import math
import numpy as np
import pandas as pd

import seaborn as sns
import matplotlib.pyplot as plt

from PIL import Image
from matplotlib.lines import Line2D
from matplotlib.patches import Wedge
from matplotlib.offsetbox import OffsetImage, AnnotationBbox

The only thing that stands out is a few specific Matplotlib imports at the end. I’ll cover those components later in the tutorial.

I use pandas to load the data.



df = pd.read_csv("./hapiness_report_2022.csv", index_col=None)
df = df.sort_values("score").reset_index(drop=True)

Seaborn style settings

Next, I use Seaborn to create a base style by defining the background, text color, and font.



font_family = "PT Mono"
background_color = "#F8F1F1"
text_color = "#040303"

sns.set_style({
    "axes.facecolor": background_color,
    "figure.facecolor": background_color,
    "font.family": font_family,
    "text.color": text_color,
})

There are several more parameters for set_style, but these four are the only ones I need in this tutorial.

I use websites such as Colorhunt and Coolors to create beautiful color palettes.

Global settings

I’m also adding a few global settings to control the general look. The first four define the range, size, and width of the wedges in the histogram.



START_ANGLE = 100 # At what angle to start drawing the first wedge
END_ANGLE = 450 # At what angle to finish drawing the last wedge
SIZE = (END_ANGLE - START_ANGLE) / len(df) # The size of each wedge
PAD = 0.2 * SIZE # The padding between wedges

INNER_PADDING = 2 * df.score.min()
LIMIT = (INNER_PADDING + df.score.max()) * 1.3 # Limit of the axes

Inner padding creates distance between the origo and the start of each wedge. It opens a space in the middle of the graph where I can add a title.

Boilerplate code

As a software engineer, I strive to write reusable code, and it’s the same when I’m working on data visualizations.

That’s why I always start by creating a few lines of boilerplate code that I can extend with reusable functions.



fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(30, 30))
ax.set(xlim=(-LIMIT, LIMIT), ylim=(-LIMIT, LIMIT))

for i, row in df.iterrows():
    bar_length = row.score
    name = row.country
    length = bar_length + INNER_PADDING
    start = 100 + i*SIZE + PAD
    end = 100 + (i+1)*SIZE
    angle = (end + start) / 2

    # Create variables here

    # Add wedge functions here

# Add general functions here

plt.axis("off")
plt.tight_layout()
plt.show()

For the rest of the tutorial, I will create and add functions and variables under one of the three comments.

Step 2: Drawing wedges

To get more power over the visuals in Matplotlib, it helps to use the underlying components rather than the built-in graph functions.

Drawing a wedge

For example, instead of using plt.pie() to create a pie chart, you can use plt.patches.Wedge() to draw the individual pieces.

That’s why I created the following function, which draws a wedge based on angles, length, bar length, and color.



def draw_wedge(ax, start_angle, end_angle, length, bar_length, color):
    ax.add_artist(
        Wedge((0, 0),
            length, start_angle, end_angle,
            color=color, width=bar_length
        )
    )

In the boilerplate code, I add draw_wedge() under the “Add functions here” comment as below.



bar_length = row.score
length = bar_length # + INNER_PADDING
start = 100 + i*SIZE + PAD
end = 100 + (i+1)*SIZE
.
.
.

# Add functions here    
draw_wedge(ax, start, end, length, bar_length, "#000")

I use row.score to define bar_length so that the visible part of the bars has an accurate size relation to each other.
For now, I’ve removed the INNER_PADDING to show you what it does.

When I run the code, I get the following figure.

As you can see, we have a long way to go until we get something similar to the polar histogram that you saw at the beginning, but at least we’ve managed to draw the wedges.

We get a lot of visual artefacts close to the middle, so let’s uncomment INNER_PADDING.

Here’s what we get.

Much better.

Adding color

Next, I have a simple color function that decides the color for each wedge based on the income level of that country.



def color(income_group):
    if income_group == "High income":
        return "#468FA8"
    elif income_group == "Lower middle income":
        return "#E5625E"
    elif income_group == "Upper middle income":
        return "#62466B"
    elif income_group == "Low income":
        return "#6B0F1A"
    else:
        return "#909090"

I use that function as input to the draw_wedge function.



# Add functions here    
draw_wedge(ax, start, end, length, bar_length, color(row.income))

Here’s the result.

With INNER_PADDING and color() there are no strange artifacts left. It’s time to add information that explains what we’re looking at.

Step 3: Adding labels

Let’s add labels for each bar in the polar histogram. I want each bar to display the country’s flag, name, and happiness score.

Defining the positions

When you add flags and text to a chart in Matplotlib, you need to calculate the correct positions.

That’s often tricky, especially when you have an unusual shape like we have in the polar histogram.

The function below takes the length of a wedge and its angle to calculate a position. Padding pushes the position away from the bar to add some visual space.



def get_xy_with_padding(length, angle, padding):
    x = math.cos(math.radians(angle)) * (length + padding)
    y = math.sin(math.radians(angle)) * (length + padding)
    return x, y

We can use this function for both flags and text.

Adding flags

For flags, I’m using these rounded ones from FlatIcon.

They require a license, so, unfortunately, I can’t share them, but you can find similar flags in other places.

Here’s my function to add a flag to the graph. It takes the position, the country’s name (which corresponds to the name of the correct file), zoom, and rotation.



def add_flag(ax, x, y, name, zoom, rotation):
    flag = Image.open("<location>/{}.png".format(name.lower()))
    flag = flag.rotate(rotation if rotation > 270 else rotation - 180)
    im = OffsetImage(flag, zoom=zoom, interpolation="lanczos", resample=True, visible=True)

    ax.add_artist(AnnotationBbox(
        im, (x, y), frameon=False,
        xycoords="data",
    ))

I change how the flag rotates if the angle exceeds 270 degrees. That happens when we start adding bars on the right part of the chart. At that point, the flag is to the left of the text, and changing the rotation makes reading more natural.

Now, we can calculate the angle, use get_xy_with_padding() and put flags on the chart.



bar_length = row.score
length = bar_length + INNER_PADDING
start = START_ANGLE + i*SIZE + PAD
end = START_ANGLE + (i+1)*SIZE

# Add variables here
angle = (end + start) / 2
flag_zoom = 0.004 * length
flag_x, flag_y = get_xy_with_padding(length, angle, 0.1 * length)

# Add functions here
...
add_flag(ax, flag_x, flag_y, row.country, flag_zoom, angle)

The flag_zoom parameters decide the size of the flag and depend on the score. If a country has a low score, there’s less room for a flag, and we need to make it a bit smaller.

Fantastic.

Adding country names and scores

To add the name and score of the country, I’ve written the following function.

As with the flags, I change the rotation if the angle exceeds 270 degrees. Otherwise, the text would be upside down.



def add_text(ax, x, y, country, score, angle):
    if angle < 270:
        text = "{} ({})".format(country, score)
        ax.text(x, y, text, fontsize=13, rotation=angle-180, ha="right", va="center", rotation_mode="anchor")
    else:
        text = "({}) {}".format(score, country)
        ax.text(x, y, text, fontsize=13, rotation=angle, ha="left", va="center", rotation_mode="anchor")

We calculate the position of the text in the same way as we did with the flags.

The only difference is that we add more padding since we want it further from the wedges.



bar_length = row.score
length = bar_length + INNER_PADDING
start = START_ANGLE + i*SIZE + PAD
end = START_ANGLE + (i+1)*SIZE

# Add variables here
angle = (end + start) / 2
flag_zoom = 0.004 * length
flag_x, flag_y = get_xy_with_padding(length, angle, 0.1 * length)
text_x, text_y = get_xy_with_padding(length, angle, 16*flag_zoom)

# Add functions here
...
add_flag(ax, flag_x, flag_y, row.country, flag_zoom, angle)
add_text(ax, text_x, text_y, row.country, bar_length, angle)

Now we have the following graph, and it’s starting to look much better.

Now it’s time to tell the users what they are looking at.

Step 4: Adding information

We have added all the data. It’s time to make the chart readable by adding helpful information and guidance.

Drawing reference lines

An excellent type of visual helper is reference lines; they work just as well here as with standard bar charts.

The idea is to draw a line at a specific score, which indirectly helps us compare different countries.

Here’s my function to draw reference lines. I’m reusing the draw_wedge() function to draw a wedge from 0 to 360 degrees.



def draw_reference_line(ax, point, size, padding, fontsize=18):
    draw_wedge(ax, 0, 360, point+padding+size/2, size, background_color)
    ax.text(-0.6, padding + point, point, va="center", rotation=1, fontsize=fontsize)

I run the function once for each score to draw multiple reference lines.



# Add general functions here
draw_reference_line(ax, 2.0, 0.05, INNER_PADDING)
draw_reference_line(ax, 4.0, 0.05, INNER_PADDING)
draw_reference_line(ax, 6.0, 0.05, INNER_PADDING)

Here’s the result.

It makes a significant difference.

Adding a title

The purpose of the gap in the center of the graph is to create a natural place for a title. Having the title in the center is unusual and can immediately capture a viewer’s interest.

The code for adding the title is standard Matplotlib functionality.



# Add general functions here
...
plt.title(
  "World Happiness Report 2022".replace(" ", "\n"), 
  x=0.5, y=0.5, va="center", ha="center", 
  fontsize=64, linespacing=1.5
)

Here’s what it looks like.

It’s getting close, but we still have one more thing to do.

Adding a legend

There’s no way for the viewer to understand what the colors mean, but we can fix that by adding a legend.

To add a legend, I’ve created the following function that takes the labels to add, their colors, and a title.



def add_legend(labels, colors, title):
    lines = [
        Line2D([], [], marker='o', markersize=24, linewidth=0, color=c) 
        for c in colors
    ]

    plt.legend(
        lines, labels,
        fontsize=18, loc="upper left", alignment="left",
        borderpad=1.3, edgecolor="#E4C9C9", labelspacing=1,
        facecolor="#F1E4E4", framealpha=1, borderaxespad=1,
        title=title, title_fontsize=20,
    )

I add the function under “Add general functions here” and run it together with everything else.



# Add general functions here
...

add_legend(
    labels=["High income", "Upper middle income", "Lower middle income", "Low income", "Unknown"],
    colors=["#468FA8", "#62466B", "#E5625E", "#6B0F1A", "#909090"],
    title="Income level according to the World Bank\n"
)

The final result looks like this.

That’s it. We have recreated the beautiful polar histogram you saw at the top.

Your entire main block of code should now look like this.



fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(30, 30))
ax.set(xlim=(-LIMIT, LIMIT), ylim=(-LIMIT, LIMIT))

for i, row in df.iterrows():
    bar_length = row.score
    length = bar_length + INNER_PADDING
    start = START_ANGLE + i*SIZE + PAD
    end = START_ANGLE + (i+1)*SIZE
    angle = (end + start) / 2

    # Add variables here
    flag_zoom = 0.004 * length
    flag_x, flag_y = get_xy_with_padding(length, angle, 8*flag_zoom)
    text_x, text_y = get_xy_with_padding(length, angle, 16*flag_zoom)

    # Add functions here
    draw_wedge(ax, start, end, length, bar_length, color(row.income))
    add_flag(ax, flag_x, flag_y, row.country, flag_zoom, angle)
    add_text(ax, text_x, text_y, row.country, bar_length, angle)

ax.text(1-LIMIT, LIMIT-2, "+ main title", fontsize=58)

# Add general functions here
draw_reference_line(ax, 2.0, 0.06, INNER_PADDING)
draw_reference_line(ax, 4.0, 0.06, INNER_PADDING)
draw_reference_line(ax, 6.0, 0.06, INNER_PADDING)
plt.title("World Happiness Report 2022".replace(" ", "\n"), x=0.5, y=0.5, va="center", ha="center", fontsize=64, linespacing=1.5)

add_legend(
    labels=["High income", "Upper middle income", "Lower middle income", "Low income", "Unknown"],
    colors=["#468FA8", "#62466B", "#E5625E", "#6B0F1A", "#909090"],
    title="Income level according to the World Bank\n"
)

plt.axis("off")
plt.tight_layout()
plt.show()

That’s it for this tutorial; congratulations on reaching the end.

Conclusion

Today, we learned to create a beautiful polar histogram using Matplotlib and Python.

Polar histograms are surprisingly easy to create, allowing us to cram more information into a single chart.

I used the World Happiness Report in this tutorial, but you can change it to another inspiring dataset.

I hope you learned a few techniques to help you bring your chart ideas to life.