Migrating From Medium - A Developer's Guide

David Berlin — Mon, 17 Jun 2019 19:06:19 +0000

The large migration from Medium has led to the creation of many useful tools to help you accomplish that.

The tools mostly rely on the following methods of obtaining your content from Medium:

RSS Feeds

Medium provides officially supported RSS feeds that are available by prepending /feed/ to your feed's URL.

For example - https://medium.com/feed/@prashantramnyc.

<item>
    <title><![CDATA[Difference between var, let and const in Javascript.]]></title>
    <description><![CDATA[<div class="medium-feed-item">...</div>]]></description>
    <link>https://codeburst.io/difference-between-var-let-and-const-in-javascript-fbce2fba7b4?source=rss-eeafca132b1e------2</link>
    <guid isPermaLink="false">https://medium.com/p/fbce2fba7b4</guid>
    <category><![CDATA[codingbootcamp]]></category>
    <category><![CDATA[coding]]></category>
    <category><![CDATA[javascript-tips]]></category>
    <category><![CDATA[programming]]></category>
    <category><![CDATA[javascript]]></category>
    <dc:creator><![CDATA[Prashant Ram]]></dc:creator>
    <pubDate>Tue, 21 May 2019 18:59:39 GMT</pubDate>
    <atom:updated>2019-05-22T14:18:46.943Z</atom:updated>
</item>

Pros:

Officially supported
Works for both user and publication feeds

Cons:

Only latest articles are available - not a good solution for retrieving all your posts.
Feed won't necessarily have entire article content - as is the nature of RSS, some articles may only show an excerpt with a link to the full article.

Tools using this approach: the DEV feed import

JSON API

For the more adventurous, it's possible to retrieve a low-level JSON structure of feeds and posts.

This is achieved by using the format=json url parameter:

  {  
     "success":true,
     "payload":{  
        "value":{  
           "content":{  
              "subtitle":"Full code example: combining images, watermarking, fonts and text",
              "bodyModel":{  
                 "paragraphs":[  
                    {  
                       "name":"2e10",
                       "type":3,
                       "text":"Image Processing in NodeJS with Jimp",
                       "markups":[ ]
                    },
                    ...
                 ]
              }
           }
        }
     }
  }

Note that the JSON is returned with a while(1) that Medium put in place to prevent JSON hijacking.

Pros:

Contains all available posts and information
Can be automated

Cons:

Undocumented and subject to breaking changes
Complex JSON structure
Can be limited by Medium's paywall

Tools using this approach: gatsby-source-medium, mediumexporter

Export File

You can request to download all your information from Medium. After making the request you'll receive a link to a zip file with the following directory structure:

blocks
bookmarks
claps
highlights
interests
posts
profile
pubs-following
sessions
topics-following
users-following

Each directory contains HTML files with minimal styling and structure.

The posts directory contains all your posts including drafts and comments.

Pros:

Officially supported
Contains all of your posts in one accessible place
Has lots of other info that a user might want when migrating to a new platform

Cons:

Requires the manual process of requesting the Zip file (although an Email with the link is sent very quickly in our experience) - can't easily be automated.
Posts are missing some information - the post tags aren't available and it's tricky to detect if a post is a full post or a comment.

Tools using this approach: medium-2-md, medium_to_ghost, medium-to-own-blog, export-medium-to-gatsby, Stackbit!

As each of these methods has its own set of drawbacks, tools often combine them to get all the content they require. For example - starting with the export zip file and augmenting it with information from the JSON API.

At Stackbit

Stackbit makes it extremely easy to create modern websites powered by a variety of data sources including Medium.

At Stackbit we created a tool that works on the export file obtained from Medium. It converts the posts to Markdown files with a structure that is easy for us to transpile into any of our supported SSG's.

The importer follows this flow:

Extract information from each post's HTML - title, thumbnail, excerpt, images, etc. We use cheerio to parse out the information directly from the HTML:

  get title() {
      return this.$('title').text().trim();
  }

  get subtitle() {
      return this.$('h4[class*="graf--subtitle"]').text().trim();
  }

Download images - store them locally grouped by the post's slug
Simplify HTML - the exported HTML file is very noisy. We use sanitize-html to remove unneeded attributes and structural elements. This simplifies the task for the next step and helps us decouple ourselves from future changes to the format of the file. Some information is retained and manipulated to assist the next steps.
Convert post content to Markdown - combine with extracted info to export front matter with Markdown content. We replace external images with those that were downloaded. We use turndown and take advantage of custom rules to preserve IFrames such as Twitter embeds:

  turndownService.addRule('twitter-tweet', {
      filter: (node) => {
          return node.nodeName === 'BLOCKQUOTE' &&
                 node.getAttribute('class') === 'twitter-tweet';
      },
      replacement: (innerHTML, node) => node.outerHTML
  });

Extract profile information - the profile.html file contains the name of the user along with their Email address and social profiles that were connected to Medium. We create a JSON structure with this information to make it easily consumable.

At Stackbit the output can then be combined into our existing themes and transpiled to the user's SSG of choice.

The tool is available on GitHub and is live on the Stackbit site. We're always looking to improve things and welcome your input.

stackbithq / stackbit-medium-importer

Stackbit Medium Importer

stackbit-medium-importer

A small library to convert medium export zip files to static-site-generator (SSG) friendly Markdown files.

Demo

The Medium importer can be seen in action on the Stackbit website.

Install

npm install @stackbit/stackbit-medium-importer

How to use

The module comes with a utility to invoke the importer directly.

$ medium-importer
Usage: medium-importer --input-file=<input-file> --output-dir=<output-dir> --concurrency=<concurrency> --download-images=<download-images> --import-drafts=<import-drafts&gt
Options
  --help                 Show help  [boolean]
  --version              Show version number  [boolean]
  --input-file, -i       medium export zip file  [required]
  --output-dir, -o       target folder for the converted files  [required]
  --concurrency, -c      number of posts to process in parallel [default: 1]
  --download-images, -d  should images be downloaded [default: true]
  --import-drafts, -r    should drafts be imported [default: true]
Examples:
  medium-importer -i medium-export.zip -o output/

Importer output

The output directory is populated with the imported data, following a structure that makes it straightforward to later use with an SSG. For certain SSG's (like Hugo) it's possible to point the…

View on GitHub

Looking forward to hearing how you migrated off Medium.

DEV Community: David Berlin

Migrating From Medium - A Developer's Guide

RSS Feeds

JSON API

Export File

At Stackbit

stackbithq / stackbit-medium-importer

Stackbit Medium Importer

stackbit-medium-importer

Demo

Install

How to use

Importer output