DEV Community: James Wallis

SkylarkTV: Streaming Platform Built with Next.js and Skylark CMS

James Wallis — Sat, 12 Jul 2025 16:09:32 +0000

After I left Skylark and the project was discontinued after 2-3 years of development, I was able to fork SkylarkTV, our demo application, to use a portfolio piece. There was one snag - the Skylark server no longer exists... Rather than let this substantial engineering effort go to waste, I partnered with Claude AI to analyze the existing GraphQL queries and create a comprehensive mock system. By examining the query patterns, response structures, and business logic embedded in the frontend, we successfully recreated the entire API surface using Mock Service Worker (MSW). This transformation turned what could have been a dead project into a living portfolio piece that demonstrates the same complex functionality without requiring expensive backend infrastructure.

Project Overview

SkylarkTV is a mock streaming television platform that demonstrates the full capabilities of Skylark, a headless CMS designed for world-class streaming products. This project showcases advanced content management, complex availability systems, and production-quality frontend architecture.

Live Demo: skylarktv.wallis.dev
Source Code: GitHub Repository
Tech Stack: Next.js, TypeScript, Tailwind CSS, React Query, MSW, Skylark

The Challenge

Building a streaming platform requires handling complex business logic that goes far beyond simple content display. The application needed to support and demonstrate Skylark features such as:

Multi-dimensional content availability (time, region, device, customer type)
Time-travel functionality for viewing content at specific points in time
Internationalization with dynamic language switching
Complex content relationships (brands → seasons → episodes)
Real-time availability filtering based on sophisticated business rules
Enterprise-scale content management with thousands of media objects

Key Achievements

Mock System Architecture

When the original Skylark backend was discontinued, I architected a comprehensive MSW-based mocking system that:

Preserves all original functionality without requiring expensive backend infrastructure
Handles complex GraphQL queries with proper relationship resolution
Maintains data consistency across related content types
Enables portfolio demonstration without ongoing operational costs

Content Management Features

Intelligent search across movies, episodes, brands, articles, and people
Dynamic content highlighting with search term emphasis
Sophisticated filtering by availability dimensions and time constraints
Responsive design optimized for desktop, tablet, and mobile devices

Technical Highlights

Availability Rules Engine

The platform includes a sophisticated rules engine that processes complex availability logic:

Time-based access windows
Geographic restrictions
Device-specific content
Customer tier limitations

In SkylarkTV, Customer Types demonstrate the most dramatic impact on content availability. In the screenshot below, you can see three different views of the same homepage: Premium (left), Standard (middle), and Kids (right). Notice the subtle difference between Premium and Standard users - a single rail is removed - while the Kids experience is completely transformed with age-appropriate content and a simplified interface.

Complex Relationship Management

// Example: Brand → Season → Episode hierarchy
const brand = await getBrand(brandId);
const seasons = await getSeasonsForBrand(brandId);
const episodes = await getEpisodesForSeason(seasonId);

The platform handles complex content relationships while maintaining performance through intelligent caching and lazy loading.

Multi-language Support

Content can be dynamically localized with proper fallback handling:

const localizedContent = mergeTranslatedContent(
  baseContent,
  translations[languageCode]
);

Future Enhancements

In the future I want to use SkylarkTV to validate whether with thorough testing, developers can merge dependency upgrades without worrying about breaking changes. The current plan is to create Playwright tests across the whole application, configure Renovate in the repository and allow automatic pull request merging when the build passes.

Conclusion

SkylarkTV represents a sophisticated example of modern web development, combining advanced frontend techniques with complex business logic. The project demonstrates the ability to work with enterprise-scale content management systems while maintaining excellent user experience and developer productivity.

The transition from a live backend to a comprehensive mock system showcases problem-solving skills and architectural thinking - turning a cancelled project into a valuable portfolio piece that continues to demonstrate technical capabilities without ongoing infrastructure costs.

Using Composite GitHub Actions to make your Workflows smaller and more reusable

James Wallis — Wed, 08 Dec 2021 14:29:41 +0000

At work there has been a drive to move existing CI/CD pipelines from Jenkins to GitHub Workflows in order to empower developers with more control over test, feature and production deployments.

In addition, we've recently moved to a monorepo structure and are building new workflows that deploy multiple applications at once. A result of using a monorepo structure is that the GitHub Workflows vary in size and purpose. For example, some carry out simple tasks such as unit testing an individual component whereas other, larger workflows deploy full production environments.

This post aims to demonstrate how composite GitHub Actions can be used to to split workflows into smaller, reusable components.

Local GitHub Actions

Before discussing Composite actions, we should first talk about local GitHub Actions. Generally, when you're creating GitHub Workflows you will be using actions found on the Marketplace but you don't have to publish to the Marketplace to use your own. Did you know you can use local GitHub Actions stored in your repository?.

Local actions are well suited to componentized GitHub Workflows as they can be maintained alongside other infrastructure code stored in the same repository. It's a good practice to store them next to the GitHub Workflow's .github/workflows directory in .github/actions but in reality they can be located anywhere in the repository.

Each custom action requires its own directory and action.yml to define it. Once the action.yml is created you can add the standard metadata to define how the action will operate.

When you're ready to use your local action in a workflow, use the following syntax (assuming you've stored your custom actions in the .github/actions directory).

jobs:
  run-local-action:
    steps:
      # Checkout the repository first
      # Otherwise the workflow won't be able to find the action
    - use: actions/checkout@v2
    - name: Run custom action
      # Use the location in the repository (without action.yml)
      uses: ./.github/actions/my-custom-action
      with:
        custom-input: 10

Local actions are a great way to create GitHub actions that are tailored to how you want them to operate without having to manage yet another product release.

Composite Actions

A Composite action is one of three different types custom GitHub Actions that can be created (composite, JavaScript and Docker). The main difference is that a composite action's action.yml -> runs property contains a list of steps to run as opposed to a program to execute.

runs:
  using: "composite"
  steps:

The steps section acts almost exactly the same as the steps section in a workflow, with a few differences that are outlined later.

Creating a local, composite GitHub Action looks like the perfect way to split up GitHub Workflows!

Should you split your workflows up into composite actions?

Having utilised composite actions for a few weeks I believe you absolutely should use them as they have led to smaller, more readable workflows as each action has a specific purpose.

Below I've added some advantages and limitations of using composite actions in your workflows.

Advantages

Separate large workflows into multiple files
Create componentized actions to be used in multiple workflows - reducing duplication
Many steps are condensed into a single one within the Actions view on GitHub, improving the ability to track a workflows progress
The descriptive nature of an action.yml file improves the readability of a GitHub workflow when understanding necessary inputs and outputs.

Limitations

They can't read GitHub Secrets - you have to pass them in
Have to define the shell on each step - although this is more of a minor annoyance.

Example

Imagine you have a GitHub Workflow that deploys a Node.js/Express API somewhere and a React application to AWS S3. It will be stored at .github/workflows/deploy-app.yml in your repository and would look something like the below:

name: Deploy app
on:
  pull_request:
jobs:
  deploy-api:
    outputs:
      # Output the deployed API URL for consumption later
      url: ${{ steps.deploy.output.url }}
    steps:
      - uses: actions/checkout@v2
      # ...more steps to deploy an API and output a URL
  deploy-frontend:
    name: Deploy frontend to AWS S3
    needs: deploy-api
    steps:
      - uses: actions/checkout@v2
      - uses: aws-actions/configure-aws-credentials@v1
        with:
          # Always store tokens in a secret manager. Here I am using GitHub Secrets
          aws-access-key-id: ${{ secrets.AWS_ACCESS_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_KEY }}
          aws-region: eu-west-1
      - uses: actions/setup-node@v2
        with:
          node-version: 14
      - name: Install dependencies
        working-directory: ./frontend
        run: npm ci
      - name: Build app
        working-directory: ./frontend
        run: npm run build
        env:
          APP_NAME: Demo
          API_URL: ${{ needs.deploy-api.outputs.url }}
      - name: Upload app to AWS S3
        working-directory: ./frontend
        run: aws s3 sync build s3://my-bucket --delete

While this workflow itself is relatively small and readable, imagine if you added additional apps to deploy or simply a more complicated frontend deployment - it would likely become bloated and much more difficult to read.

To future-proof this workflow, we can consolidate much of the frontend-deploy job into a composite action:

Creating the custom action file

First we create the directory and action.yml. As I said previously, it's consistent to store custom actions in a .github/actions directory. So my deploy-frontend-to-s3 action will be stored as .github/actions/deploy-frontend-to-s3/action.yml giving us the repository structure:

.github
  - workflows
    - deploy-app.yml
  - actions
    - deploy-api/action.yml
    - deploy-frontend-to-s3/action.yml

Implementing the `deploy-frontend-to-s3` composite action

Now that the action.yml file is created we can migrate many of the deploy-frontend job's steps out of the workflow and into it.

First we can add the name, description and inputs to the action.yml:

name: "Deploy Frontend to S3"
description: "Builds and deploys the React frontend to AWS S3"
inputs:
   aws-access-key-id:
     required: true
     description: "The aws-access-key-id used to authenticate with AWS"
   aws-secret-access-key:
     required: true
     description: "The aws-secret-access-key used to authenticate with AWS"
   app-name:
     required: false
     description: "The app name used by the React app"
     default: Demo
   api-url:
     required: true
     description: "The URL of the Express app"

Next we can create the part of the action which will run the commands previously located in the workflow. Note how we can refer to the inputs above with ${{ inputs.input-name }}.

# ...name, description and inputs as above
runs:
  using: "composite"
  steps:
    - uses: aws-actions/configure-aws-credentials@v1
      with:
        # Actions cannot access secrets so pass them in as inputs
        aws-access-key-id: ${{ inputs.aws-access-key-id }}
        aws-secret-access-key: ${{ inputs.aws-secret-access-key }}
        aws-region: eu-west-1
    - uses: actions/setup-node@v2
      with:
        node-version: 14
    - name: Install dependencies
      working-directory: ./frontend
      run: npm ci
    - name: Build app
      working-directory: ./frontend
      run: npm run build
      env:
        APP_NAME: ${{ inputs.app-name }}
        API_URL: ${{ inputs.api-url }}
    - name: Upload app to AWS S3
      working-directory: ./frontend
      run: aws s3 sync build s3://my-bucket --delete

Notice that the steps section is almost a copy and paste from the steps section of the workflow. The only part missing is the actions/checkout as it is required to be in the workflow so that GitHub can access the local action.

Using the custom actions in the `deploy-app` workflow

Now that we've built the custom action, we can use it in our workflow:

name: Deploy app
on:
  pull_request:
jobs:
  deploy-api:
    outputs:
      # Output the deployed API URL for consumption later
      url: ${{ steps.deploy.output.url }}
    steps:
      - uses: actions/checkout@v2
      -  id: deploy
         uses: ./.github/actions/deploy-api
  deploy-frontend:
    name: Deploy frontend to AWS S3
    needs: deploy-api
    steps:
      - uses: actions/checkout@v2
      - name: Deploy frontend
        # Directory name only
        uses: ./.github/actions/deploy-frontend-to-s3
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_KEY }}
          # won't pass in app-name as it's defaulted to "Demo"
          api-url: ${{ needs.deploy-api.outputs.url }}

And we're done! We've reduced the size of the deploy-app workflow and made a reusable composite action to deploy the frontend to S3.

Using files in the action directory

While building a composite action you may want to create utility files which are consumed by the action.yml. When doing this you should refer to them using the ${{ github.action_path }} variable so that if you relocate the action, you will not need to update any paths.

So for a file named .github/actions/deploy-frontend-to-s3/fetch-bucket-name.sh, you can refer to it as ${{ github.action_path }}/fetch-bucket-name.sh in the action.

Round up

In this post I have demonstrated how composite actions can help to break a GitHub Workflow into smaller, consumable components and explained why its advantageous to do so.

Do you deal with huge workflows at work or are you already using composite actions? Let me know in the comments below.

If this article has helped you, drop a reaction!

Thanks for reading!

Animating Next.js page transitions with Framer Motion

James Wallis — Mon, 05 Jul 2021 21:18:44 +0000

A few months ago I rebuilt my Dev.to powered Next.js website from scratch. While building it, I decided adding animations would bring its simple design to life. Previously, I'd used CSS transitions and JavaScript to achieve animations on a webpage. This time I wanted to use an animation library built for React.js that I could use in future projects.

Enter Framer Motion.

Framer Motion

A production-ready motion library for React.

- https://www.framer.com/motion

It's a library that enables the animations of React components on a page and while the component is entering and also leaving.

Framer Motion can do all of the following:

Spring animations
Simple keyframes syntax
Gestures (drag/tap/hover)
Layout and shared layout animations
SVG paths
Exit animations
Server-side rendering
Variants for orchestrating animations across components
CSS variables

And can bring a static page to life:

Various animations powered by Framer Motion

Read more about Framer Motion and view examples on their website.

Animating Next.js page transitions

As well as making user triggered animations, Framer Motion can animate a component when it is mounting (entering) and unmounting (leaving). I use this capability to animate the components that come and go when the page changes. In Next.js terms, this is everything apart from _app.js - so all pages and other components. Where possible, using _app.js to persist layouts between page changes will reduce the amount of rendering that React has to do each time the page changes - potentially improving your app performance.

Preparing the codebase

Before I added any animations to my website I did two pieces of refactoring:

Moved common components that shouldn't animate on every page change into _app.js. In my case this meant moving the Header and Footer which you can see on GitHub.
Added a wrapper component to control the animation states within pages. On my website it is the Layout component. Note the <motion.main> component which is specific to Framer Motion. In the rendered HTML output this will be a HTML main element, however, adding the motion. supplied by Framer Motion provides the ability to pass certain animation props such as transition, initial and animate.

Entry animations

Looking at the Layout component you will see an object named variants (see below). Variants promote cleaner code by removing the requirement to add the animation object to the motion.main component. You can read more about them on the Framer Motion website.



const variants = {
    hidden: { opacity: 0, x: -200, y: 0 },
    enter: { opacity: 1, x: 0, y: 0 },
    exit: { opacity: 0, x: 0, y: -100 },
}

Now focussing on the motion.main component:



<motion.main
    variants={variants} // Pass the variant object into Framer Motion 
    initial="hidden" // Set the initial state to variants.hidden
    animate="enter" // Animated state to variants.enter
    exit="exit" // Exit state (used later) to variants.exit
    transition={{ type: 'linear' }} // Set the transition to linear
    className=""
>
    {children}
</motion.main>

The initial and animate states will control the entry animation for this component. When you change the page on my website, you should see the content change from having an opacity of 0 and x position of -200px to having an opacity of 1 and being in the center of the screen. This gives the effect of the content fading in from the left. By the way, "A Transition is an object that defines how values animate from one state to another" - from the Framer Motion website.

An entry animation is great but let's go a little further and animate components when they leave the page.

Adding `AnimatePresence` and exit animations

One feature of Framer Motion is that it can animate components after they've left the React DOM. To activate this feature you can use the AnimatePresence component. For my website, I use the optional exitBeforeEnter prop which tells the entrance animation to wait until the exit animation has ended before starting - without this the content would mount on top of the unmounting content, looking messy.

You'll need to add the AnimatePresence component to the _app.js file so that it never unmounts (unmounting would disable the exit animations). Note also the initial={false} prop which disables the entry animation when you first visit the website. Disabling it is just a personal preference, remove that line if you want to enable it.

Once AnimatePresence is added to _app.js, you can add an exit animation to your motion.main component. See this in the two code blocks above.

We're almost finished but we just need to fix an issue with Next.js scrolling to the top of the page when the route changes.

Solving the scroll on link change issue

When adding page navigation to a Next.js application you should be using the Link component. By default, when the Link component is clicked it scrolls to the top of the page before animating, making the page transitions look a bit clunky. See below:

Scrolling to the top before animating

Fortunately the fix for this is pretty easy. For each Link component that is used around your codebase, add the scroll={false} prop. This will disable the scrolling when its clicked. To make this easier and maintain clean code, I created a component that wraps Link but disables the scroll. I called it NoScrollLink and you can view it on GitHub.

After disabling the Link component's scroll, it's a good idea to scroll to the top of the page after the Framer Motion exit animation has completed. This gives the effect of content leaving at the current scroll height but the new content entering at the top of the page. Again this is easy, you can use the onExitComplete prop on the AnimatePresence component in _app.js. The following code snippet will scroll to the top once the exit animation has completed.



onExitComplete={() => window.scrollTo(0, 0)}

View on GitHub

Having added that, when you change page Framer Motion should unmount the old content, scroll to the top and mount the new content.

The finished product

If you've been following along or want to see it live on my website you'll see the following page transitions:

The finished page animation

Summary

In this article I wanted to help others add page transitions to their Next.js app with the help of Framer Motion. I overcame some obstacles when adding them to my website such as realising AnimatePresence needed to be in _app.js and how to stop the scroll to the top of the page after a Link is clicked.

If you've anything to add or just want to show some appreciation, leave a comment or react!

Thanks for reading!

I made a free Dev.to Writing Streak Calculator using Next.js, Day.js and the Dev.to API

James Wallis — Tue, 13 Apr 2021 22:26:44 +0000

Do you wonder what your current weekly posting streak is?

On Monday (12th April 2021), I was awarded the 16-week streak badge. It was a surprise, albeit a welcome one, as I figured that I was two weeks away from writing for sixteen continuous weeks with my first article being published on January 10th, 2021.

This made me think - how does Dev.to calculate the writing streak? Can I build a tool to help fellow Dev.to community members understand if they're on track for their next milestone.

Below you'll find the answers - tl;dr made a tool that's available for anyone to use.

The Dev.to Writing Streak Calculator

I'll demo the app first and further down explain how I calculate the user's current writing streak.

The Writing Streak Calculator

The app is simple by design.

It consists of a single input box for your username and a text box next to it. Once you've entered your username it will calculate your current writing streak and report it back to you (with some text that is borrowed from the Forem codebase).

To build it I used my usual tools with a couple of others:

Next.js
Tailwind CSS
Dev.to API
DayJS - Heavily for calculating the streaks
SWR - Fetches articles and user data from the Dev.to API
Preact - Reduced first load size from 83kB to 49kB (Used for the first time, seems good!)

Source code on GitHub.

That is pretty much it. At this point, there are no overly complicated features. Some things I'd like to add, however, are:

Date since last posted
Longest writing streak
A warning that the user needs to post if they're close to the end-of-week deadline
Support other Forem sites (CodeNewbie etc)

Something more you want to see? Add it in the comments or open an issue.

In case you missed it above, here's the link for the tool. Go check out your writing streak!
https://devto-writing-streak-calculator.wallis.dev

Bonus - display on your own site

There is also an API route that will fetch your latest and longest writing streak. You just pass your username as a query param.

https://devto-writing-streak-calculator.wallis.dev/api/calculate?username=jameswallis

I use it to display my current writing streak on my Advanced Dev.to Dashboard with historical analytics.

Writing Streak displayed on my analytics dashboard (final box on the right)

How the writing streak is calculated

Prewarning: There is the possibility that I'm calculating it incorrectly. Hopefully, a Forem contributor will call me out and I can correct it! Otherwise, if you think your streak isn't right let me know in the comments or raise an issue on GitHub. That being said, it's correct for me for two separate streaks so I'm confident.

I started by noting when I was awarded the 16 weeks and 4-week badges (I deleted the email for 8 weeks):

4 weeks: Monday 17th August 2020
16 weeks: Monday 12th April 2021

So badges are awarded on Mondays.

Next, I wanted to understand why my streak wasn't broken between the 30th December 2020 (2nd post in the streak) and the 10th January 2021 (3rd post in the streak) - that's an 11 day difference. After searching for missing articles and trying to find reason in the Dev.to Ruby on Rails code, I eventually just checked a calendar.

December 2020 - January 2021. Red circles denote published dates.

It turned out that the 10th January 2021 was the week after the 30th December 2020 when the week starts on a Monday.

So I came to the conclusion that as long as a post is published between Monday and Sunday then the streak would continue!

Here's the link again:
https://devto-writing-streak-calculator.wallis.dev

Summary

I've introduced the writing streak calculator tool made for the Dev.to community.

If you've enjoyed this article or the tool, react!

Let me know your latest streak or any comments down below.

Thanks for reading!

I built an advanced Dev.to dashboard with historic data using Next.js and Azure Functions 📈

James Wallis — Tue, 06 Apr 2021 22:55:59 +0000

Do you ever watch your view count rise and wonder which posts are being read the most? I know I did.

That is one reason why I chose to develop my own Dev.to analytics dashboard to display historical data such as view, reaction and follower increase over the last 24 hours, 7 days and 30 days.

You can view the live dashboard here: https://devto-analytics.wallis.dev. You'll notice that it's styled to look like Dev.to.

Incredibly, Dev.to has recently added their own analytics page as well as having an "analytics" option on each post (next to edit, manage). There are some differences between the built-in dashboard and mine.

Why build an analytics dashboard?

I really enjoy posting on Dev.to. I recently rewrote my whole website to use Dev.to as a CMS to enable myself to continue posting here while using my own site as the canonical URL.

I wanted, however, to see more in-depth information about each post such as view, reaction and follower increases over a given period of time. By default, I didn't have access to any data that would allow me to calculate any increases.

As a result, I ended up building my own analytics dashboard using Next.js, Tailwind CSS, Recharts, the Dev.to API and Azure Functions & Cosmos DB.

You can view the code on GitHub.

How I source the historical data

If you've used the Dev.to API before, you'll have noticed that, for the moment, you're unable to access any historical data. This makes it difficult to know how popular a post is over a given period of time (before the built-in analytics).

To record my historical data I created an Azure Function that saves my latest article and follower data, gathered using the Dev.to API, to a Azure Cosmos Database. The functions that save my article and follower data run every hour.

In addition, I created a couple of HTTP Azure Functions so that I can access the data.

If you're wondering, I'm entirely within the free tier on Azure - so it isn't costing me anything to record my historic data.

You can view the Azure Functions implementation on GitHub.

The dashboard

The dashboard consists of three pages:

Home/overview page - basically an overview of my current stats and periodical view/reaction/comment/follower increases
Breakdown graphs page - Charts that break down stats from the overview page so that I can easily see information such as what articles have been most read this week.
Summary graphs page - Charts that display the increase of views/reactions/followers from hour-to-hour and day-to-day

You can view the dashboard implementation on GitHub.

Home/overview page

The dashboard's UI is based heavily on Dev.to's styling. I wanted it to appear as an extension to Dev.to's current implementation. The home page is styled to look like an advanced version of the Dev.to dashboard page that contains your general stat overview (total page views, reactions, comments) and a list of any draft and published articles.

Moreover, it was a lot of fun essentially cloning Dev.to to try to make the dashboard look as alike as possible. Try switching between the dashboard home page and your Dev.to dashboard in different tabs!

On my dashboard, I've added follower and last posted date to the overview stats - as I care about them more than "Listing's created" and "Credits available". Most overview stats also contain two pieces of historical data. For page views, for example, I'm displaying the total page view increase for the past 24 hours and the past 7 days.

Overview stats

Looking further down the page, you'll see a sortable list of my published posts containing the usual stats for each (views, reactions and comments). I've also added the 24 hours, 7 days and 30 day increase for each stat, for every post.

Individual post stats

Breakdown graphs page

This page displays graphs that break down increases for views, reactions and comments. Its purpose is to show me what posts have been most popular over a given period of time.

Hovering over a breakdown graph

While its styling is based on Dev.to, unlike the home page I haven't copied a specific Dev.to page.

Summary graphs page

The final page contains graphs that details how views, reactions and comments have increased over a period of time. Using this page I can interpret such information as if my follower increases have stagnated or are increasing at a good level.

Page view trends over the past 24 hours and 7 days

Difference between my dashboard and the built-in analytics page

If you've read this far you're likely wondering what the difference between my dashboard and the built-in Dev.to analytics dashboard. The following is my opinion, let me know in the comments if you disagree.

What Dev.to built-in analytics does better:

Dev.to has access to a lot more historical data so they can report your page views way back

I'm bias, let me know if you think there are more than this.

What my dashboard does better:

Integrates increases/historical data into the main dashboard screen better. I'd love it if my Dev.to dashboard displayed my daily, weekly and monthly increases for each post.
Displays follower and last posted date in the overview stats - I don't care about listings or credits. Moreover, I'd love to display my current posting streak instead of the time since I last posted. This would help me ensure I'm on track for my 16-week badge.
The breakdown page makes it easy to see why my view count has risen.

Technical details

Next.js - powers the dashboard.
- The built-in data fetching method getStaticProps is used to preload the article and follower data at build time. Incremental Static Regeneration rebuilds the page to minimise how out of date the initially served page is.
- useSWR fetches the current article and follower data once the page has loaded.
- Combining getStaticProps and useSWR means that the dashboard loads fast, but will always display the most up-to-date data.
Azure Functions and Azure Cosmos DB
- Saves historical data gathered using the Dev.to API.
- Serves the data via a HTTP route for the UI to consume.
Tailwind CSS - styling
React-icons - various icons such as the question mark and GitHub icon on the navigation bar.
Recharts - the chart library used on the graph pages
TypeScript project references
- Enable the sharing of TypeScript interfaces, and other code, between the dashboard and the Azure Functions codebases.
- View in the common directory of the GitHub repository.
- For more information on TypeScript project references, read my post on them.

Future improvements

There are few features that I want to add to the dashboard:

Calculate and display my posting streak (by week) in the stat overview - should help with the 16-week streak badge.
Average stats - such as average page views or followers gained per day.
Predicted stats - days until I hit 100,000 page views for example.
Display whether I've added a canonical URL to the post (for it to display on my website) and reposted it on other sites such as Hashnode and Medium.

Summary

In this post, I've introduced my Dev.to analytics dashboard that helps me understand how my stats are changing over time and identify popular posts.

Would you like to make your own dashboard like this? Have any thoughts on the dashboard? Let me know in the comments!

If you are looking to fork/clone my GitHub repository then I'll need to make the documentation a little better 😅 - it's pretty straightforward to do once your Azure Functions/Cosmos DB is set up.

P.S. Why isn't it password protected?

I'm aware that by posting this blog I'm allowing access to my private post data to anyone who stumbles across it. I'm OK with this because:

I want to show the dashboard off as a portfolio piece
I want others to be able to copy/clone it so that they can see their own historical Dev.to data
I couldn't come up with a reason to hide it. I understand why it is private by default but in my case, I'm happy with others seeing it.

Let me know what you think of my Dev.to analytics dashboard!

Thanks for reading!

Using TypeScript Project References to share common code

James Wallis — Tue, 30 Mar 2021 22:50:43 +0000

Ever wondered if you can share interfaces, types and functions between TypeScript projects?

I'm currently developing a project consisting of two separate TypeScript applications, one being a React.js dashboard and the other an Azure Function app written in Node.js. As part of the project, the dashboard calls an API in the Azure Function app. This got me thinking, as I'm in control of both the data source and the application that uses the data, is there a way that I can share certain interfaces between the two projects?

The answer is yes, since version 3 of TypeScript you can use Project References to share code between TypeScript projects. When using Project References in my project, however, I couldn't find any official examples on how to use them - hence this post!

While the implementation below is what has worked for me, if you have any improvements, let me know in the comments.

What are Project References?

Project references allow you to structure your TypeScript programs into smaller pieces. By doing this, you can greatly improve build times, enforce logical separation between components, and organize your code in new and better ways.

How to use

Take a project that consists of a frontend and a backend written in TypeScript. Both contain an interface called IData which is exactly the same. Currently, each time I make a change, I have to duplicate it in the other file (which is extremely annoying).

The directory of the project is:

myproject
- frontend
  - app.ts
  - interfaces
    - IData.ts
  - tsconfig.json
- backend
  - server.ts
  - interfaces
    - IData.ts
  - tsconfig.json

In order to use a single IData.ts file between both projects, we can use Project References.

Adding the common TypeScript project

We will start by creating a third TypeScript project called common, adding an empty tsconfig.json file and copying the IData.ts interface over. We can also remove it from the frontend and backend apps. So the directory structure will be:

myproject
- frontend
  - app.ts
  - tsconfig.json
- backend
  - server.ts
  - tsconfig.json
- common
  - interfaces
    - IData.ts
  - tsconfig.json

This isn't enough though. In the common app's tsconfig.json we need to add the following:

{
    "compilerOptions": {
        "target": "es5", // Or whatever you want
        "module": "es2015", // Or whatever you want
        "declaration": true,
        "declarationMap": true,
        "outDir": "./dist",
        "composite": true
    }
}

The key parts are:

declaration: Generates a declaration file that the frontend and backend apps can use to reference items in the common app.
composite: Ensures TypeScript can quickly determine where to find the outputs of the referenced project
declarationMap: Enables editor features like “Go to Definition” and Rename to transparently navigate and edit code across project boundaries in supported editors

Referencing the common project in `frontend`/`backend`

To reference the common IData interface in the frontend and backend apps we need to make a simple change to both of their tsconfig.json files. Add the references property to your existing tsconfig.json.

{
    "compilerOptions": {
        // The usual
    },
    "references": [
        { "path": "../common" }
    ]
}

Building the `frontend`/`backend` apps

Now that we've added the reference to the common app in order to access its interfaces we need to compile both the frontend and backend apps.

When doing so, ensure you use the --build option so that TypeScript automatically builds all referenced projects.

tsc --build .

Note: If you're using Next.js with TypeScript, I didn't need to do this. Both next dev and next build kept working just the same.

Importing the common interface into `frontend`/`backend`

This is easier than you might first think, just import IData using its relative path. TypeScript will do the magic when you compile it.

import IData from '../common/interfaces/IData'

Note: if your compiler has problems with the IData file, you can add type after the import. See: https://dev.to/uppajung/comment/1f6bc

Summary

In this post, I've demonstrated how to use TypeScript Project References to use a common project for shared interfaces, functions, types and more!

Feedback on my approach is appreciated! As I said above, I couldn't find an official example to guide me on how to use Project References so any feedback in the comments will help me improve this tutorial and my own TypeScript projects!

Thanks for reading!

Using getStaticProps and getStaticPaths with TypeScript - Next.js

James Wallis — Wed, 24 Mar 2021 21:24:10 +0000

My personal website is built on Next.js and uses both the getStaticProps and getStaticPaths functions to dynamically generate the /blog/ and /portfolio/ pages at build time. While updating both methods to use their proper TypeScript types, following the documentation, I ran into an error when reading the parameter that I was passing from getStaticPaths into getStaticProps.

The error that appeared was:

Property 'slug' does not exist on type 'ParsedUrlQuery | undefined'

After doing some research and finding a discussion on the Next.js GitHub regarding this issue, I recognised it was a gap in their documentation. It explains how to add the type to getStaticProps when used on its own but it doesn't demonstrate how to access the property you've declared in getStaticPaths.

Background

getStaticProps and getStaticPaths are two methods that can be used for data fetching in Next.js. Briefly speaking getStaticProps lets you fetch data at build time and getStaticPaths enables you to specify dynamic routes to pre-render pages based on data.

For more information on these functions, read my post on different ways to fetch data in Next.js.

The error

Using the example code below I will demonstrate the TypeScript error and advise you on how to fix it. I'm using the variable name slug to create the dynamic routes, but you could use anything - another common name is id.

import { GetStaticPaths, GetStaticProps } from 'next'
import { ParsedUrlQuery } from 'querystring'

export const getStaticPaths: GetStaticPaths = async () => {
    const arr: string[] = ['slug1', 'slug2']
    const paths = arr.map((slug) => {
        return {
            params: { slug },
        }
    })
    return { paths }
}

export const getStaticProps: GetStaticProps = async (context) => {
    // This is where the error occurs
    const { slug } = context.params // Property 'slug' does not exist on type 'ParsedUrlQuery | undefined'
    const props = fetch(`/api/${slug}`)
    return { props }
}

Note the first line of the getStaticProps. Here we are attempting to access the slug variable that was created in getStaticPaths and returned inside the params object. This is the line that causes the error as context.params has the type ParsedUrlQuery | undefined and slug does not exist in ParsedUrlQuery.

const { slug } = context.params

The fix

Fortunately, fixing the issue is as simple as creating an interface that extends ParsedUrlQuery and contains a slug property.

interface IParams extends ParsedUrlQuery {
    slug: string
}

Once we've added that to the file, we need to assert that context.params is of type IParams. This is done like so:

const { slug } = context.params as IParams

It's as simple as that! Just adding the IParams interface makes the TypeScript error disappear.

Updated example code:

import { GetStaticPaths, GetStaticProps } from 'next'
import { ParsedUrlQuery } from 'querystring'

interface IParams extends ParsedUrlQuery {
    slug: string
}

export const getStaticPaths: GetStaticPaths = async () => {
    const arr: string[] = ['slug1', 'slug2']
    const paths = arr.map((slug) => {
        return {
            params: { slug },
        }
    })
    return { paths }
}

export const getStaticProps: GetStaticProps = async (context) => {
    const { slug } = context.params as IParams // no longer causes error
    const props = fetch(`/api/${slug}`)
    return { props }
}

Round up

If this post has helped you use Next.js together with TypeScript, react or let me know in the comments!

Thanks for reading!

Sources:

The easiest way to download SVGs from a website

James Wallis — Tue, 23 Mar 2021 01:37:09 +0000

Usually, when adding icons to a website, I'll use React-icons. There are times, however, when I prefer to copy an exact SVG from a website. Downloading an SVG isn't as easy as downloading an image on a website. Fortunately, SVG Export extracted them for me.

The problem

Ever use inspect element to download images that you're unable to right-click and either save or open? You'll likely have noticed that with SVGs it's not easy to download them from a website. - or at least I did when I was attempting to "clone" a webpage for my latest project.

Take the Dev.to logo that is visible on every page on Dev.to, for example. Right-clicking on that doesn't give you the option to open it in a new tab or save it. Moreover, using inspect element only gives you the potentially huge SVG tags, leaving you to work out where the SVG begins and ends in the page's HTML.

Unable to open or download the Dev.to icon SVG!

The solution

Enter SVG Export, by Stephen Delaney is a browser extension for Google Chrome and Mozilla Firefox.

After installing the extension, clicking the extension icon will trigger SVG Export to extract all available SVGs on your current webpage, ensuring their inline styles are kept. A new tab will be opened displaying all SVGs. From there, locate the SVGs you're after and either download the SVG as a file, copy it as code or paste it into your favourite design tools like Sketch, Figma or Framer.

To download the Dev.to icon mentioned above, I simply opened any page on Dev.to, clicked the SVG Export icon to start the extract, and finally located it in the newly opened tab.

SVG Export in action

Round up

In this post, I demonstrated how I was able to easily extract an SVG from a webpage using SVG Export.

Do you have another, perhaps better, method of downloading SVGs from websites? If you do, let me know in the comments 👇

If this article has helped, drop a reaction!

Thanks for reading!

50,000 views milestone, continuing my blogging journey

James Wallis — Tue, 16 Mar 2021 23:59:20 +0000

🎉 Last week I hit 50,000 total views! 🎉

🔥 And also 1000 total reactions! 🔥

🚀 And was awarded the Next.js badge! 🚀

I'm buzzing. I wanted to reach 50,000 total views by June this year but it's the start of March and I've already achieved it - I thought for sure that it would be a close call given the time it took to reach 20,000 views. The icing on the cake? My latest post earned me the Next.js author of the week badge, which I've been eyeing for ages!

Back in January, when I reached 20,000 I wrote a milestone post that discussed what I'd learnt about blogging so far. In this post, I'll reflect on it and will attempt to understand how I've more than doubled my total views in a third of the time.

Reviewing my 20,000 milestone post

What I'd learnt a few months into posting

In the last post, I created a list of things I'd learnt over the course of my first few months of blogging. Here are some of the points that I still believe are important:

Post titles matter. I still believe this and carefully word titles trying to include "How to" or some type of call to action to entice the reader.
Keep posting. I've acquired the 8-week posting streak badge and (think) I'm on track for the 16-week badge.
Don't be discouraged by a low amount of reactions. While I'm now making posts that gain a considerable amount of attention (for me), I think about half of those that I've posted in the past few months have had ~10 reactions.
Post what you find interesting. This is a good one! My most popular posts have been about rebuilding my website to use Dev.to as a CMS.

What I planned to do next

At the end of the post, I came up with a plan to reach 50,000 views.

This is what I said...

In the next six months, I'm aiming to reach 50,000 views. And, by the end of the year, I want to achieve 100,000 views.
To achieve this I will:

Create more posts!

Update old posts with new content and correcting tutorials where the underlying software has changed.

Publicise my posts across the web and share them on Linkedin and other social sites. So far I haven't advertised them at all and rely on followers on dev.to, it's algorithm and Google rankings to gain views on my articles. This would be beneficial to gain more views, but means advertising my writing to people that know me - so we'll see whether I actually do it or not.

...and after some reflection, I can tell you that I haven't really done any of them. I've created more posts and updated a couple of tutorials with corrections. Yet, here I am hitting 50,000 views?

So what have I done instead?

Been persistent. Currently, I'm creating a post about once a week. This has helped me grow to ~700 followers, that's a lot of people to receive a notification each time I make a post. Additionally, I believe that it's helped me naturally improve the quality of my posts as I gain more experience.
Categorised my posts. Over the past couple of months I've written three types of posts. Sticking to three formats has led to a smooth experience when I write a post as I have a format that I can follow for each type. If I want to, I will deviate from the formats, but they help to get me started.
1. Portfolio items where I write about past projects (these show up on my website under the /portfolio path, more info here).
2. "5 things to check out". I've written about 5 Next.js features, 5 places to get pre-built Tailwind CSS components. These posts are fun to write as I have to find 5 things to write about so I'm learning while I'm writing.
3. Projects created with a post topic in mind. My two most popular posts have both been about rewriting my website to use Dev.to as a CMS. While this project has made it ten times easier to add posts to my website (with the canonical URL pointing to my site), I knew before starting development that it would be interesting to write about - and that bet has paid off.
Varied my posts. Posting every week can be exhausting. Some weeks I'll write a shorter post, like this one, and other weeks I'll set time aside to focus on a longer, tutorial style post.

What's next?

🎯 100,000 views 🎯

Reaching 50,000 views isn't the end of my blogging journey. My next goals are to reach:

100,000 total views
2,000 total reactions
10,000 views on a single post
The 16 week streak badge

To achieve this I will:

Create more posts!
Cross-post to other blogging sites like Medium and Hashnode. Both seem a lot slower than Dev.to to gain views on posts but I want to persist and add most of my articles there - we'll see how it works out!
Publicise my posts. I'll start with LinkedIn and then see how I get on. I know I said this last time but I really should start to do this. I've been lucky enough to have been shared by the Dev Community Twitter account so I really should do the same!
- I'm curious, how much success do you have on other blogging platforms like Medium and Hashnode? Do you use any others?

Summary

I'm continuing to enjoy writing, posting and interacting with members of the Dev.to community. Though, I have to admit I'm excited to achieve the 16-week streak badge so I can have a little break - I wish there was a way to see if am on track.

Let me know what you think about my blogging experience and if you have any advice for me!

Where do you cross-post to?

Thanks for reading! Hope you're having a great 2021!

How to add a blog using Dev.to as a CMS to a Next.js website

James Wallis — Wed, 10 Mar 2021 00:00:24 +0000

For a shorter introduction (about half the length) check out "I completely rewrote my personal website using Dev.to as a CMS".

Preface

I've been posting on Dev.to for a few months now. I love the platform, the editor, the ability to draft, edit and publish an article making it available to the millions of Dev.to users.

Recently, I decided that I wanted to present them on my own website. After researching different ways to achieve this, I concluded using the Dev.to API to create the blog section of my website would be the perfect solution. I decided that articles would only show up on my website if I'd added a canonical URL to the article on Dev.to - meaning my website is seen as the source of the article (even though it was written on Dev.to).

Continuing to use Dev.to also means that I don't need to configure storage for saving the articles or any images used. Additionally, I can take advantage of the built-in RSS feed which other blogging sites can read to automatically import my articles.

I came up with the following list of requirements:

Use the Dev.to API to fetch all my articles and display them on my website.
Fetch and render each article at build time to ensure the website would be fast and to ensure good SEO for the individual blog pages. Using dynamic pages would make the website load slower as it would query the Dev.to API on the client-side and also mean that I would have the same SEO data, such as page title, for each blog page.
Set the canonical URL of an article on Dev.to and have that be the article's URL on my website. I wanted to continue to use the Dev.to editor to write and manage my articles, so they should only show on my website once I've added a canonical URL.
Have a nice URL for the blog posts on my website that I would be in complete control of. Neither the post ID nor the Dev.to path to the article.
Rebuild each time an article is created or updated. This was crucial as the blog would be static - I didn't want to press the rebuild each time I changed something.

I was able to achieve all of this using a combination of Next.js dynamic pages, Vercel deploy hooks and the public Dev.to API.

Setting up the project

Key technologies used

TypeScript - if you prefer plain JavaScript for code examples, this GitHub repository has the same functionality as described below but is purely JavaScript.
Next.js, React.js etc (required to create a Next.js app).
Tailwind CSS, Tailwind CSS Typography plugin (for styling).
Remark Markdown parser and plugins such as remark-html to convert the Markdown returned by the Dev.to API to HTML. Other plugins I use enable features such as code highlighting, GitHub flavour Markdown compatibility (for ~~strikethrough~~ etc) and stripping out Front Matter from the displayed HTML.
The Dev.to API and it's https://dev.to/api/articles/me endpoint.
Vercel deploy hooks. I use Vercel to host my Next.js site and their deploy hooks allow me to rebuild my website automatically when an article is added or edited on Dev.to.

To see all the packages I'm currently using on my website, check out the package.json on GitHub.

The two Next.js functions that run my website

My personal website is built using Next.js. To ensure that all content continued to be generated at build time, I used two built-in Next.js functions that can be used to fetch data for pre-rendering. These are:

getStaticProps - fetch data from a source (think API or file) and pass it into the component via props.
getStaticPaths- provides the ability to use dynamic routes with a static site.

I'll be using both functions to make the dynamic article page called [slug].ts - the square brackets denote that it is a Next.js dynamic page and the name slug is the name of the parameter that will be passed into getStaticProps from getStaticPaths.

How do I determine which articles appear on my website?

For articles to appear on my website they have to have a canonical URL pointing at https://wallis.dev/blog.

Whenever I refer to the page slug I'm referring to the last section of the canonical URL (after /blog). When reading the canonical URL from the Dev.to API I use the following function to convert the URL to the slug.

const websiteURL = 'https://wallis.dev/blog/';

// Takes a URL and returns the relative slug to your website
export const convertCanonicalURLToRelative = (canonicalURL) => {
    return canonicalURL.replace(websiteURL, '');
}

When I pass https://wallis.dev/blog/a-new-article to convertCanonicalURLToRelative it will return the slug a-new-article.

How to add a blog with using Dev.to as a backend

The individual article pages (`/blog/${slug}`)

Overview

Each individual article page is generated at build time using the getStaticPaths Next.js function that fetches all my Dev.to published articles, and saves them to a cache file. getStaticProps then fetches an individual article from the cache and passes it into the page component via its props.

A cache file must be used because Next.js doesn't allow passing data from getStaticPaths to getStaticProps - aside from the page slug. For this reason, the page slug is used to fetch an article from the cache file.

Flow Diagram

The diagram below should explain the process that is followed when creating dynamic pages through Next.js using the getStaticPaths and getStaticProps functions. It outlines the most important function calls, briefly explains what they do, and what is returned.

Implementation

View on GitHub

Below you will find the code that dynamically creates each article page.

import fs from 'fs';
import path from 'path';

import Layout from '../../components/Layout';
import PageTitle from '../../components/PageTitle';
import IArticle from '../../interfaces/IArticle';
import { getAllBlogArticles, getArticleFromCache } from '../../lib/devto';

const cacheFile = '.dev-to-cache.json';

interface IProps {
    article: IArticle
}

const ArticlePage = ({ article }: IProps) => (
    <Layout title={article.title} description={article.description}>
        <img
            src={article.coverImage}
            alt={`Cover image for ${article.title}`}
            className="md:mt-6 lg:mt-10 xl:mt-14 h-40 sm:h-48 md:h-52 lg:h-64 xl:h-68 2xl:h-80 mx-auto"
        />
        <PageTitle title={article.title} center icons={false} />
        <section className="mt-10 font-light leading-relaxed w-full flex flex-col items-center">
            <article className="prose dark:prose-dark lg:prose-lg w-full md:w-5/6 xl:w-9/12" dangerouslySetInnerHTML={{ __html: article.html }} />
        </section>
    </Layout>

)

export async function getStaticProps({ params }: { params: { slug: string }}) {
    // Read cache and parse to object
    const cacheContents = fs.readFileSync(path.join(process.cwd(), cacheFile), 'utf-8');
    const cache = JSON.parse(cacheContents);

    // Fetch the article from the cache
    const article: IArticle = await getArticleFromCache(cache, params.slug);

    return { props: { article } }
}

export async function getStaticPaths() {
    // Get the published articles and cache them for use in getStaticProps
    const articles: IArticle[] = await getAllBlogArticles();

    // Save article data to cache file
    fs.writeFileSync(path.join(process.cwd(), cacheFile), JSON.stringify(articles));

    // Get the paths we want to pre-render based on posts
    const paths = articles.map(({ slug }) => {
        return {
            params: { slug },
        }
    })

    // We'll pre-render only these paths at build time.
    // { fallback: false } means other routes should 404.
    return { paths, fallback: false }
}

export default ArticlePage

The flow diagram above combined with the comments throughout the code should enable a full understanding of the code. If you have any questions, comment below.

You'll notice that two functions are called from the lib/dev.ts file. getArticleFromCache does what it suggests, it finds an article in the cache and returns it. getAllBlogArticles, on the other hand, is the function that fetches all my articles from Dev.to and converts the supplied markdown into HTML - using functions from lib/markdown.ts.

Devto.ts

import axios, { AxiosResponse } from 'axios';
import IArticle from '../interfaces/IArticle';
import ICachedArticle from '../interfaces/ICachedArticle';
import { convertMarkdownToHtml, sanitizeDevToMarkdown } from './markdown';

const username = 'jameswallis'; // My Dev.to username
const blogURL = 'https://wallis.dev/blog/'; // Prefix for article pages

// Takes a URL and returns the relative slug to your website
export const convertCanonicalURLToRelative = (canonical: string) => {
    return canonical.replace(blogURL, '');
}

// Takes the data for an article returned by the Dev.to API and:
// * Parses it into the IArticle interface
// * Converts the full canonical URL into a relative slug to be used in getStaticPaths
// * Converts the supplied markdown into HTML (it does a little sanitising as Dev.to allows markdown headers (##) with out a trailing space
const convertDevtoResponseToArticle = (data: any): IArticle => {
    const slug = convertCanonicalURLToRelative(data.canonical_url);
    const markdown = sanitizeDevToMarkdown(data.body_markdown);
    const html = convertMarkdownToHtml(markdown);

    const article: IArticle = {
        // parse into article object
    }
    return article;
}

// Filters out any articles that are not meant for the blog page
const blogFilter = (article: IArticle) => article.canonical.startsWith(blogURL);

// Get all users articles from Dev.to
// Use the authenticated Dev.to article route to get the article markdown included
export const getAllArticles = async () => {
    const params = { username, per_page: 1000 };
    const headers = { 'api-key': process.env.DEVTO_APIKEY };
    const { data }: AxiosResponse = await axios.get(`https://dev.to/api/articles/me`, { params, headers });
    const articles: IArticle[] = data.map(convertDevtoResponseToArticle);
    return articles;
}

// Get all articles from Dev.to meant for the blog page
export const getAllBlogArticles = async () => {
    const articles = await getAllArticles();
    return articles.filter(blogFilter);
}

// Get my latest published article meant for the blog (and portfolio) pages
export const getLatestBlogAndPortfolioArticle = async () => {
    const articles = await getAllArticles();
    const [latestBlog] = articles.filter(blogFilter);
    const [latestPortfolio] = articles.filter(portfolioFilter); // ignore this! It's meant for another page (see the wallis.dev GitHub repository for more information)
    return [latestBlog, latestPortfolio];
}

// Gets an article from Dev.to using the ID that was saved to the cache earlier
export const getArticleFromCache = async (cache: ICachedArticle[], slug: string) => {
    // Get minified post from cache
    const article = cache.find(cachedArticle => cachedArticle.slug === slug) as IArticle;
    return article;
}

The key points to note about the devto.ts file is:

I've used the authenticated https://dev.to/api/articles/me endpoint to fetch all my articles from Dev.to. This endpoint is the only one that returns all my articles (ok, 1000 max...) and includes the article markdown. Authenticating also gives a slightly higher API limit.
- Previously I used the built-in HTML returned in the https://dev.to/api/articles/{id} but I kept hitting the API limit as each build made as many API calls as I had articles.
- Get a Dev.to API Token following the instructions on the API docs.
The convertDevtoResponseToArticle function converts the markdown into HTML using a function from the lib/markdown.ts.

Markdown.ts

import unified from 'unified';
import parse from 'remark-parse';
import remarkHtml from 'remark-html';
import * as highlight from 'remark-highlight.js';
import gfm from 'remark-gfm';
import matter from 'gray-matter';
import stripHtmlComments from 'strip-html-comments';

// Corrects some Markdown specific to Dev.to
export const sanitizeDevToMarkdown = (markdown: string) => {
    let correctedMarkdown = '';

    // Dev.to sometimes turns "# header" into "#&nbsp;header"
    const replaceSpaceCharRegex = new RegExp(String.fromCharCode(160), "g");
    correctedMarkdown = markdown.replace(replaceSpaceCharRegex, " ");

    // Dev.to allows headers with no space after the hashtag (I don't use # on Dev.to due to the title)
    const addSpaceAfterHeaderHashtagRegex = /##(?=[a-z|A-Z])/g;
    return correctedMarkdown.replace(addSpaceAfterHeaderHashtagRegex, '$& ');
}

// Converts given markdown into HTML
// Splits the gray-matter from markdown and returns that as well
export const convertMarkdownToHtml = (markdown: string) => {
    const { content } = matter(markdown);

    const html = unified()
        .use(parse)
        .use(gfm) // Allow GitHub flavoured markdown
        .use(highlight) // Add code highlighting
        .use(remarkHtml) // Convert to HTML
        .processSync(stripHtmlComments(content)).contents;

    return String(html);
}

This file is pretty simple; the comments should explain everything, so I won't add anything more. If you'd like to learn more about using Remark converts with Next.js, you can read my blog titled "How to use the Remark Markdown converters with Next.js projects".

Summary

Phew, that was a lot. Hopefully, I didn't lose you in the code examples and explanations!

Everything above explains how I've built the dynamic article pages on my website. I've included all the code that you'll need to create the dynamic blog pages on your own website.

By the way, when the code above is compiled it produces an article page such as https://wallis.dev/blog/nextjs-serverside-data-fetching.

Let's move onto the blog overview page (wallis.dev/blog).

The article overview page (`/blog`)

Building a page for each of your Dev.to articles at build time is great but how will a user find them without an overview page?! They probably won't!

Overview

The overview page is much simpler than the dynamic article pages and only uses functions from the lib/devto.ts file introduced above. So this section will be shorter than the last.

Flow Diagram

As before, I've made a diagram to display the process followed when displaying all the article summaries on the overview page. You'll notice that this time I'm only using getStaticProps rather than getStaticProps and getStaticPaths. This is because I'm only loading data for one page rather than creating dynamic pages (which is what getStaticPaths allows you to do).

Implementation

View on GitHub

import Layout from '../components/Layout'
import PageTitle from '../components/PageTitle'
import Section from '../components/Section'
import ArticleCard from '../components/ArticleCard'
import IArticle from '../interfaces/IArticle'
import { getAllBlogArticles } from '../lib/devto'

interface IProps {
    articles: IArticle[]
}

const title = "Blog ✍️"
const subtitle = "I share anything that may help others, technologies I\'m using and cool things I\'ve made."

const BlogPage = ({ articles }: IProps) => (
    <Layout title={title} description={subtitle}>
        <PageTitle
            title={title}
            subtitle={subtitle}
        />

        <Section linebreak>
            {articles.map(({ title, description, publishedAt, tags, canonical }) => (
                <ArticleCard
                    key={title}
                    title={title}
                    description={description}
                    date={publishedAt}
                    tags={tags}
                    canonical={canonical}
                />
            ))}
        </Section>
    </Layout>
)

export async function getStaticProps() {
    // Get all the articles that have a canonical URL pointed to your blog
    const articles = await getAllBlogArticles();

    // Pass articles to the page via props
    return { props: { articles } };
}

export default BlogPage

Essentially the above code:

Loads the articles from the Dev.to API
Passes them into the component
Maps over each article and creates a summary card for each which links to the dynamic article page created in the previous step.

The overview page looks like this:

Summary

Amazing, that's the overview page complete! If you're following along you should now have:

Blog pages being created dynamically
An overview page that links to the dynamic blog pages

Rebuild each time an article is created or updated

The final step that I took to create my Dev.to powered website is to set up a Vercel deploy hook. My website is hosted on Vercel so I am able to use a deploy hook to programmatically trigger a rebuild, refreshing the article content in the process.

Deploy Hooks allow you to create URLs that accept HTTP POST requests in order to trigger deployments and re-run the Build Step.

https://vercel.com/docs/more/deploy-hooks

To trigger the deploy hook, I have created a Dev.to API webhook that calls it each time an article is created or updated.

Configuring the automatic rebuild

A prereq for this section is that you're website needs to be deployed onto Vercel. I've created instructions on how to do this.

To create a deploy hook, follow the Vercel documentation - it's a lot more simple than you'd think.

Once you have the deploy URL we can use the Dev.to API to create a webhook to trigger it.

You can do this using curl (make sure you add your API_KEY and change the target_url to be your Vercel deploy hook URL):

curl -X POST -H "Content-Type: application/json" \
  -H "api-key: API_KEY" \
  -d '{"webhook_endpoint":{"target_url":"https://example.org/webhooks/webhook1","source":"DEV","events":["article_created", "article_updated"]}}' \
  https://dev.to/api/webhooks

For more information, see the Dev.to API docs.

Summary

Nice one, now your website will automatically redeploy each time you create or update an article on Dev.to!

Next steps

I love my website right now and using Dev.to to manage most of its content has made adding content much more efficient than previously. However, there are a couple of things I want to improve in the future:

If a user is viewing a blog on Dev.to and it links to another of my articles, the user should stay on Dev.to. But if they're on wallis.dev, they should stay on it rather than being taken to Dev.to.
Another Dev.to user made a comment in another of my articles and made the point that if Dev.to suddenly turned off, I'd lose my articles. However unlikely, I want to set up a system to take daily backups of my articles to mitigate the risk of losing them.

Round up

In this article, I've taken you through the code that allows Dev.to to power my website. If you venture onto my GitHub you'll see that in addition to having a blog section (https://wallis.dev/blog), I also use Dev.to to display my portfolio entries (https://wallis.dev/portfolio).

If you want more background on why and how I've used the Dev.to API to power my website, read my initial post discussing it.

If you found this article interesting or it has helped you to use Next.js and the Dev.to API to build your own website using Dev.to as a CMS, drop me a reaction or let me know in the comments!

Anything I can improve? Let me know in the comments.

Thanks for reading!

PS, I'm currently deciding whether I should create a tutorial series that will take you through building a Dev.to powered blog from scratch - is this something you would read/follow?

Different ways to fetch data in Next.js (server-side) and when to use them

James Wallis — Tue, 02 Mar 2021 23:48:09 +0000

When building an application powered by Next.js it's probable that you'll need to fetch data from either a file, an internal API route or an external API such as the Dev.to API. Moreover, determining what data fetching method to use in a Next.js application can easily become confusing - especially as it isn't as simple as making an API request inside your components render function, as you might in a stock React app.

The following guide will help you carefully select the server-side data fetching method that suits your app (FYI you can use multiple methods in a single app). For each method, I have outlined when it runs, it's benefits and an example of when you could use the method in your Next.js application.

The following methods fetch data either at build time or on each request before the data is sent to the client.

getStaticProps (Static Generation)

Fetch data at build time.

The getStaticProps method can be used inside a page to fetch data at build time, e.g. when you run next build. Once the app is built, it won't refresh the data until another build has been run.

Note: Added in Next 9.3

Usage:

export async function getStaticProps(context) {
  const res = await fetch(`https://.../data`)
  const data = await res.json()

  if (!data) {
    return {
      notFound: true,
    }
  }

  return {
    props: {}, // will be passed to the page component as props
  }
}

Benefits:

It enables the page to be statically generated and will produce fast load times of all the data fetching methods.
If each page in your app uses getStaticProps (or no server-side data fetching methods) then Next.js will be able to export it into static HTML using next export. This is advantageous if you want to create a static site that can be hosted on places such as GitHub Pages.
The data is rendered before it reaches the client - great for SEO.

Example usage:

Imagine you have a personal blog site that renders pages from markdown files at build time. getStaticProps can read the files and pass the data into the page component at build time. When you make a change to a blog entry, you rebuild the site to see the changes. ameira.me, a site I built, uses this method - each time Ameira makes a change to her portfolio, Vercel automatically rebuilds and republishes the site.

getServerSideProps (Server-side Rendering)

Fetch data on each request.

The getServerSideProps method fetches data each time a user requests the page. It will fetch the data before sending the page to the client (as opposed to loading the page and fetching the data on the client-side). If the client makes a subsequent request, the data will be fetched again.

Note: Added in Next 9.3

Usage:

export async function getServerSideProps(context) {
  const res = await fetch(`https://...`)
  const data = await res.json()

  if (!data) {
    return {
      notFound: true,
    }
  }

  return {
    props: {}, // will be passed to the page component as props
  }
}

Benefits:

The data is refreshed each time a client loads the page meaning that it is up to date as of when they visit the page.
The data is rendered before it reaches the client - great for SEO.

Example usage:

getServerSideProps is perfect for building an application that requires the client to see the most up to date information, but isn't refreshed while the client is on the page (see client-side for constantly updating information). For example, if I had a page on my personal site that displayed information about my last GitHub commit or my current Dev.to stats, I'd want these fetched each time a page is viewed.

getInitialProps (Server-side Rendering)

Fetch data on each request.

getInitialProps was the original way to fetch data in a Next.js app on the server-side. As of Next.js 9.3 you should use the previously discussed methods over getInitialProps but I'm including it in this article because:

It can still be used - although the Next.js maintainers advise you not to as getStaticProps and getServerSideProps enable you to choose from static or server-side data fetching.
Knowing about getInitialProps helps when you come across an old Stack Overflow query that mentions it, and also that you shouldn't just copy and paste that solution!.

Note: If you're on Next.js 9.3 or above, use the two methods above.

Usage:

function Page({ stars }) {
  return <div>Next stars: {stars}</div>
}

Page.getInitialProps = async (ctx) => {
  const res = await fetch('https://api.github.com/repos/vercel/next.js')
  const json = await res.json()
  return { stars: json.stargazers_count }
}

export default Page

Benefits:

Same as getServerSideProps - use getServerSideProps!

Example usage:

Same as getServerSideProps - use getServerSideProps!

How to decide which one to use?

When using Next.js, I always aim to make each page static. This means that I try to avoid using getServerSideProps and favour getStaticProps. However, if the data that I am fetching changes often then of course I will use getServerSideProps. I never use getInitialProps anymore.

So normally I try getStaticProps and if that is causing data to become outdated then I move to getServerSideProps.

A word on client-side data fetching

This article hasn't covered any client-side data fetching methods but you can use the useEffect hook to make the request or the useSwr custom hook made by Vercel engineers which implements stale-while-revalidate.

SWR is a strategy to first return the data from cache (stale), then send the fetch request (revalidate), and finally come with the up-to-date data.

- swr.vercel.app

SWR Usage:

import useSWR from 'swr'

function Profile() {
  const { data, error } = useSWR('/api/user', fetcher)

  if (error) return <div>failed to load</div>
  if (!data) return <div>loading...</div>
  return <div>hello {data.name}!</div>
}

Final words

In this article, I've introduced three Next.js methods that can be used to fetch data either at build time or before each client request.

Liked this article? Hit the like button!

Thanks for reading!

5 APIs to use in your next project

James Wallis — Tue, 23 Feb 2021 23:28:30 +0000

Being a developer is great - you can build almost anything you want! Sometimes, however, it's difficult to decide what to build next as there are so many different project ideas out there. To help you find your next project, I've come up with a list of APIs that you can play with and provided potential project ideas.

Dev.to

Using Dev.to's own API you could create your own personal editor that can create and publish new posts as well as edit old ones. While creating this app you'd have to build a markdown editor and a system to publish your articles to Dev.to.

Additionally, you could use the Dev.to API to dynamically display your articles on your own personal website. In fact, I recently rewrote my website to use Dev.to as a CMS to make the blog and portfolio sections easier to manage.

Unsplash

Unsplash is a website that hosts "beautiful, free images and photos that you can download and use for any project". Its API can be used to fetch photos from Unsplash.

The Unsplash API could be used to make an application with a background that changes to a random image at different times during the day. For example, you could build your own Chrome new tab page that has a beautiful changing background and contains links that you find useful. Bonus points if the background reflects the time of day e.g. light during the day and darker at night.

Spotify API

The Spotify API is one I've used in the past to create a home automation dashboard that shows what song I'm currently playing and has controls to play, pause, skip tracks and set the volume (in addition to other functionality). In addition to these features, Spotify also has APIs that:

Provide audio analysis to learn about a tracks danceability, energy, valence, and more
Control playback and can use the Web SDK to play full tracks
Customize and display a user's recommendations
Search for tracks in a user's region or anywhere
For others and further app ideas check the discover page.

Hue

If you have any products from the Phillips Hue family, it's likely that they can be controlled using the Hue API. The official app can become a bit busy when you have multiple devices, so you could build an app that can control Hue bulbs around the house, changing their colour and brightness. You could also go back to basics and build a command-line app to control your lights!

Slack

Using the Slack API you can build tools that integrate directly with Slack. The Slack API can be used to build a chatbot that can carry out various functions such as setting reminders, checking the weather or communicating with other APIs I've introduced above - for example, a chatbot that can control your Hue lights.

Bonus - A Dashboard to rule them all

If you're looking for a larger project to take on, you could combine all the APIs I've listed above to create a dashboard that can:

Display your total views/reactions using the Dev.to API
Have a beautiful background using the Unsplash API
Control your currently playing track and volume level through the Spotify API.
Display the state of lights around your house and control them using the Hue API.
Integrate the Slack API to make a chatbot that can directly control the whole dashboard!

Final words

These are just a few services that have APIs that you can use to build your next project. Hopefully, this article has given you that little bit of inspiration you needed to develop your next app!

Used any of these APIs before or have any to suggest? Pop them in the comments.

Liked this article? Hit the like button!

Thanks for reading!

DEV Community: James Wallis

SkylarkTV: Streaming Platform Built with Next.js and Skylark CMS

Project Overview

The Challenge

Key Achievements

Mock System Architecture

Content Management Features

Technical Highlights

Availability Rules Engine

Complex Relationship Management

Multi-language Support

Future Enhancements

Conclusion

Using Composite GitHub Actions to make your Workflows smaller and more reusable

Local GitHub Actions

Composite Actions

Should you split your workflows up into composite actions?

Advantages

Limitations

Example

Creating the custom action file

Implementing the deploy-frontend-to-s3 composite action

Using the custom actions in the deploy-app workflow

Using files in the action directory

Round up

Animating Next.js page transitions with Framer Motion

Framer Motion

Animating Next.js page transitions

Preparing the codebase

Entry animations

Adding AnimatePresence and exit animations

Solving the scroll on link change issue

The finished product

Summary

I made a free Dev.to Writing Streak Calculator using Next.js, Day.js and the Dev.to API

Bonus - display on your own site

How the writing streak is calculated

Summary

I built an advanced Dev.to dashboard with historic data using Next.js and Azure Functions 📈

Why build an analytics dashboard?

How I source the historical data

The dashboard

Difference between my dashboard and the built-in analytics page

Technical details

Future improvements

Summary

Using TypeScript Project References to share common code

What are Project References?

How to use

Adding the common TypeScript project

Referencing the common project in frontend/backend

Building the frontend/backend apps

Importing the common interface into frontend/backend

Summary

Using getStaticProps and getStaticPaths with TypeScript - Next.js

Background

The error

The fix

Round up

The easiest way to download SVGs from a website

The problem

The solution

Round up

50,000 views milestone, continuing my blogging journey

Reviewing my 20,000 milestone post

What I'd learnt a few months into posting

What I planned to do next

So what have I done instead?

What's next?

Summary

How to add a blog using Dev.to as a CMS to a Next.js website

Preface

I came up with the following list of requirements:

Setting up the project

Key technologies used

The two Next.js functions that run my website

How do I determine which articles appear on my website?

How to add a blog with using Dev.to as a backend

The individual article pages (/blog/${slug})

Overview

Implementing the `deploy-frontend-to-s3` composite action

Using the custom actions in the `deploy-app` workflow

Adding `AnimatePresence` and exit animations

Referencing the common project in `frontend`/`backend`

Building the `frontend`/`backend` apps

Importing the common interface into `frontend`/`backend`

The individual article pages (`/blog/${slug}`)

The article overview page (`/blog`)