DEV Community

Melvin Carvalho
Melvin Carvalho

Posted on • Updated on

How to Get a List of Posts From the API


I've decided to download my posts so that I can store them in git. That means that if the site ever goes down, I have a copy of my blog content. Step 1, is to get a list of my posts.


The API is documented here.

The following endpoint will give a list of the latest articles for a given user:

It supports pagination, each page will contain 30 articles by default. This can be increased with the per_page query parameter, which we will set to 1000. That should be enough for most users.

Replace melvincarvalho with your own username

The Code

So I wrote a JavaScript script that will pull out a list of my articles.

#!/usr/bin/env node

// requires
const argv = require('minimist')(process.argv.slice(2))
const fs = require('fs')
const path = require('path')
const $ = require('child_process').execSync

// data = {
  api: '',
  dataDir: path.join(__dirname, '..', 'posts'),
  filename: 'index.json',
  perPage: 1000,
  user: 'melvincarvalho'

// init
data.api = argv.api || data.api
data.dataDir = argv.dataDir || data.dataDir
data.filename = argv.filename || data.filename
data.perPage = argv.perPage || data.perPage
data.user = argv._[0] || data.user
console.log('data', data)

// main
let postsUri = `${data.api}`
postsUri += `?per_page=${data.perPage}`
postsUri += `&username=${data.user}`
const cmd = `curl '${postsUri}'`
console.log('cmd', cmd)
const json = JSON.parse($(cmd).toString())

// output
if (!fs.existsSync(data.dataDir)) {
  fs.mkdirSync(data.dataDir, { recursive: true })
const output = JSON.stringify(json, null, 2)
const outFile = path.join(data.dataDir, data.filename)
console.log('output', output)
fs.writeFileSync(outFile, output)

Enter fullscreen mode Exit fullscreen mode

Usage: ./getposts.js [username]

The Code Explained

First we initialize the endpoint and username. Then we run some curl to get the result, and finally we format it and write it to a file.

The Output

Running this code will get a list of articles and save it to a file called posts/index.json

The JSON output can be seen here and the current script is here.

Now that I have a list of articles, it should be possible to download the markdown from individual article, too. I will hopefully cover that in a future post.

Useful Links

Top comments (0)