Using DEV.to as a CMS

2021 update: I no longer endorse this. Enough people use the Dev.to API now that Netlify consistently gets rate limited (despite being authenticated) so that your CI breaks.

This is a post on Dev.to that should also appear on my personal site. Canonical URL is manually set so that my site is the authoritative source, but the content lives in Dev.to.

See the comparison:

• Dev.To URL: https://dev.to/swyx/using-dev-to-as-a-cms-3472/
• Swyx.io URL: https://www.swyx.io/writing/devto-cms/

In September last year, DEV started publicizing their API (I don't know exactly when they launched it). I've had the idea to use DEV as a headless CMS for a while, but today I actually did it. It is nice because it gets syndication and comments for people who use Devto, as well as a nice image upload solution that doesn't involve checking into Git.

Steps

You should be pretty comfortable wrangling APIs and piping content into your own blog. I wrote my own static site generator so I am pretty comfortable with this, but if you are new to this game you may need some extra help based on your setup.

• Get your API key at https://dev.to/settings/account
• See the docs at https://docs.dev.to/api/#operation/getUserPublishedArticles

Our game plan is to only pull published articles from our account. The API is paginated, and although we can cheat by pulling a 1000-article page, I'm going to do it the "right" way by looping though each page until we reach an end. You could also store this and tweak this logic so you only fetch pages up to a certain preset date.

Here's some code that I wrote for proof of concept

require('dotenv-safe').config() // have DEV_TO_API_KEY in process.env
const fetch = require('node-fetch')
;(async function() {
  let allArticles = []
  let page = 0
  let per_page = 30 // can go up to 1000
  let latestResult = []
  do {
    page += 1 // bump page up by 1 every loop
    latestResult = await fetch(
      `https://dev.to/api/articles/me/published?page=${page}&per_page=${per_page}`,
      {
        headers: {
          'api-key': process.env.DEV_TO_API_KEY
        }
      }
    )
      .then(res => res.json())
      .then(x => (allArticles = allArticles.concat(x)))
      .catch(err => {
        console.error(err) // very basic error handling, customize as needed
        throw new Error(`error fetching page ${page}, {err}`)
      })
  } while (latestResult.length === per_page)
  console.log(allArticles.length)
})()

Now, this just gets you the basic data. You have more work to do to get stuff to show up on page nicely.

Notes:

• Dev.to's API only exports the raw markdown. You will have to do the postprocessing yourself, including stripping and processing frontmatter, and adding syntax highlighting and whatever else you would like. See my devto source plugin for how I did it. (Update: rhymes corrected me that the single post api endpoint gives you html with highlightjs syntax highlighting, but I continue to do my own postprocessing for the other benefits I get with remark.)
• In Dev.to, if you try to specify any datelike format in frontmatter, e.g. 2020-02-20 - you get a base: Tried to load unspecified class: Date. I opted out of it by adding string quotes around it like "2020-02-20"
• Dev.to lets you add any frontmatter you like, including stuff it already has, like slug and subtitle. it just wont recognize it. This is great for porting over frontmatter that isn't recognized by Dev.to.
• I have 95 posts on my site, and 55 posts on Dev.to. Some were duplicate posts. I solved this by removing duplicate content from my site (leaving Dev.to the authortitative store, since it also had social metadata), and then merging the content in my site generator data layer. This took a while 😅.
• There isn't a nice way to auto-set canonical url for Dev.to. Ideally I would like to just specify a slug, and that would populate the slug field and the canonical_url field. I may have to write my own Dev.to client for this - not hard given it doesnt require WYSIWYG. However I would need to replicate the Image Upload functionality.

Comments

[rhymes]

Hi shawn, neat tricks! A note about this:

Dev.to's API only exports the raw markdown

the API exports the processed HTML if you retrieve a single article by its id, for example:

> curl https://dev.to/api/articles/266934 | jq '.body_html'
"<p>This is a post on Dev.to that should also appear on my personal site. Canonical URL is manually set so that my site is the authoritative source, but the content lives in Dev.to.</p>\n\n<p>See the comparison:</p>\n\n<ul>\n<li>Dev.To URL: <a href=\"https://dev.to/swyx/using-dev-to-as-a-cms-3472/\">https://dev.to/swyx/using-dev-to-as-a-cms-3472/</a>\n</li>\n<li>Swyx.io URL: <a href=\"https://www.swyx.io/writing/devto-cms/\">https://www.swyx.io/writing/devto-cms/</a>\n</li>\n</ul>\n\n<p><a href=\"https://twitter.com/bendhalpern/status/1176663688742395904?s=20\">In Se

[swyx]

It looks like they use highlight.js for syntax highlighting, whereas I use github.com/octref/shiki/, and also use github.com/remarkjs/remark-toc for table of contents so i probably need to continue processing the raw markdown but thank you!

[swyx]

ah TIL! thank you!

[Chandu J S]

Awesome! I did the same with Gatsby for my website. 😃
I used gatsby-source-dev plugin for populating my blog section.

🌏 I Made My Website With Gatsby!

Chandu J S ・ Feb 21 ・ 3 min read

#gatsby #portfolio #javascript #webdev

Article from my website, here.

The following query was used to get the data.

query {
  allDevArticles {
    edges {
      node {
        article {
          id
          title
          readable_publish_date
          tags
          cover_image
          positive_reactions_count
          slug
          body_html
          url
        }
      }
    }
  }
}

Check the file here

[Rene Padillo 🇵🇭]

interesting, thanks for sharing 👍🏼