How to download all your DEV articles in markdown format

Update: Looks like the dev API has changed. I added a new params option to get max 1000 articles.

We can agree that DEV is the best site for devs in the world. What if you want to cross-post your dev articles into your own personal blog site, but too lazy to copy-paste each post? 😴

I used to copy paste every single article I wrote into my personal site, but quickly found it tedious. I created a script to easily generate DEV articles.

There are two ways to do this: work from the script below or clone the Github repo.

The script

Here is main script. It is pretty short. It uses axios to fetch the data and dotenv to store secrets.

require('dotenv').config()
const axios = require('axios')
const fs = require('fs')
const path = require('path')
const URL = 'https://dev.to/api/articles/me/published'

const options = {
  headers: { 'api-key': process.env.DEV_KEY },
  params: { 'per_page' : 1000 }
}

const contentDir = '/content/'
const contentPath = __dirname + contentDir

axios
  .get(URL, options)
  .then(response => {
    response.data.forEach(article => {
      const normalizedTitle = article.title
        .replace(/\s+/g, '-')
        .replace(/[^a-zA-Z0-9-]/g, '')
        .toLowerCase()
      const normalizedPublishedAt = article.published_at.split('T')[0]
      const fileName = `${normalizedPublishedAt}_${normalizedTitle}.md`

      console.log(`${contentDir}${fileName} created!`)
      fs.writeFileSync(
        `${contentPath}${fileName}`,
        article.body_markdown,
        'utf-8'
      )
    })
    console.log('*** dev.to data successfully fetched ***')
  })
  .catch(error => {
    console.log('ERROR: ', error)
  })

DEV API makes it very easy to do it. The response of the API request returns an array of all articles you've written.

I extracted the markdown content, renamed the file with YYYY-MM-DD_your-article-title.md (I also removed the unique ID at the end, ex: .../your-article-title-1a2b).

The repo

I also created a github repo:

https://github.com/iggredible/dev-fetch-articles

Just clone the repo, install all dependencies, and create content/ directory in project root. Don't forget to give it your .env information!

git clone https://github.com/iggredible/dev-fetch-articles.git
cd dev-fetch-articles
npm install
mkdir content
mkdir .env
// create a key called DEV_KEY and give it your dev key values from dev.to site

You can find your DEV_KEY secret inside Settings -> Account on dev.to site.

Once done, just run npm run fetch-dev. You will find all your articles inside content/ folder you just created. You can use this script into your app or use this script to generate content that you can copy to your app.

Making it better

If you see how the code can be improved, feel free to submit PR on Github or drop in the comment below.

Thanks for reading and happy blogging! 📖🔥

Comments

[Corey McCarty]

I'm currently using the 11ty static site generator blog and publishing to a personal domain (from markdown) and dev.to listens to the rss to pick up articles automatically. All I have to do is change published value in the front matter to true and it publishes.

[Josef Biehler]

Thanks for mentioning this. Maybe I try this out.

[Igor Irianto]

Oh, I've heard of Eleventy last year, I think. That's a cool strategy. I will have to take a look at the rss thing!
Thanks for the input 😁. Really appreciate it!

[rhymes]

Hi Igor! Nice post and usage of the API!

There's also a third option: you can request an export of your content, you can find it here at the bottom of the page: dev.to/settings/misc

[Igor Irianto]

I didn't know this feature exists! 😅

[Shijie Zhou]

Right @rhymes , I had done that before, but I think using the api can keep your article update without download the export json each time.

[rhymes]

Agree :) It's easier to script!

[Hugo van Kemenade]

This is now at dev.to/settings/account (sorry for the bump, but this page shows in searches :)

[Josef Biehler]

I am not sure if this will also solve this problem. But I use the approach mentioned here: dev.to/maxime1992/manage-your-dev-...

This works without any effort and using this you could reference your repository from your personal blog or publish blog posts to dev.to and to your personal blog during build.

Nice idea!

[Igor Irianto]

Really good link. Thanks Josef! I wasn't aware of this method. This is actually really cool. I have had GH account for several years but I never used template option.

Really good link! 👍 Thank you very much!