DEV Community

Anand Rathnas
Anand Rathnas

Posted on • Originally published at jo4.io

The 5 Edge Cases That Broke Our Dev.to Auto-Crossposting (And How We Fixed Them)

This article was originally published on Jo4 Blog.

In our previous post, we covered the producer-consumer problem for blog scheduling. But we glossed over the crossposting part.

"Just POST to the dev.to API," we said. "How hard could it be?"

Narrator: It was hard.


The Setup

We have a Node.js script that runs daily via GitHub Actions:

// Simplified flow
1. Find all markdown posts with publishAfter <= today
2. Check if they exist on dev.to already
3. If not, create them
4. Send Slack notification
Enter fullscreen mode Exit fullscreen mode

Sounds straightforward. Here are the edge cases that broke it.


Edge Case 1: How Do You Know If a Post Already Exists?

The Problem

We can't just check our local database—we don't have one. The blog is a static site. So how do we avoid posting duplicates?

Naive approach: Keep a .crossposted.json file locally.

Why it fails: Someone manually posts to dev.to. Someone deletes the JSON file. Someone runs the script from a different machine. Duplicates everywhere.

The Fix: dev.to Is the Source of Truth

async fetchDevtoArticles() {
  const response = await fetch(`${DEVTO_API_URL}/me/published?per_page=100`, {
    headers: { 'api-key': this.apiKey },
  });

  const articles = await response.json();

  // Store by canonical_url for O(1) lookup
  for (const article of articles) {
    if (article.canonical_url) {
      this.devtoArticles.set(article.canonical_url, {
        id: article.id,
        url: article.url,
        title: article.title,
      });
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Before creating anything, we fetch ALL our existing dev.to articles. The canonical_url field is unique—it's the original source URL. If our canonical URL already exists, skip.

Bonus: dev.to returns a 422 error with "canonical" in the message if you try to create a duplicate. We catch that too:

if (response.status === 422 && errorText.toLowerCase().includes('canonical')) {
  return { duplicate: true, title: frontmatter.title };
}
Enter fullscreen mode Exit fullscreen mode

Belt and suspenders.


Edge Case 2: The 60-Day Time Bomb

The Problem

Our script only looks back 60 days on dev.to (performance optimization—we don't need articles from 2 years ago). But what happens to a post with publishAfter: "2025-01-01" that we never crossposted?

Scenario:

  1. January: Write a post, set publishAfter: "2025-01-15"
  2. January 15: Script runs, posts to dev.to ✅
  3. March 20: 65 days later, the script's dev.to lookback window no longer includes this article
  4. Some bug causes the post to be re-processed
  5. Duplicate post on dev.to ❌

The Fix: Auto-Update Stale Dates

If a post has publishAfter older than 60 days, we automatically update it to today:

if (this.isOlderThanDevtoMaxDays(publishAfter)) {
  console.log(`[publish-after] "${title}" has old publishAfter (${publishAfter}), updating to ${today}`);
  return { shouldProcess: true, needsDateUpdate: true, newDate: today };
}
Enter fullscreen mode Exit fullscreen mode

And here's the edge case's edge case—we need to commit this change to git:

// After processing all posts
if (this.filesToCommit.length > 0 && !this.dryRun) {
  commitChanges(this.filesToCommit, 'chore: auto-update old publishAfter dates to today');
}

function commitChanges(files, message) {
  for (const file of files) {
    execSync(`git add "${file}"`, { stdio: 'pipe' });
  }
  execSync(`git commit -m "${message}"`, { stdio: 'pipe' });
}
Enter fullscreen mode Exit fullscreen mode

Why commit? Because if the script runs again before you pull, it would try to update the same posts again. The commit ensures the updated dates persist.

Slack notification: "⚠️ publishAfter updated to today for "My Post" - please pull latest"


Edge Case 3: Accidental Content Overwrites

The Problem

You crosspost a blog post. A week later, you fix a typo locally. The script runs. Does it update dev.to?

If yes: What if you intentionally made dev.to-specific edits? Gone.
If no: How do you push updates when you actually want them?

The Fix: Explicit Update Intent

Updates only happen when you set updatedAt in frontmatter:

---
title: "My Post"
publishAfter: "2026-02-15"
updatedAt: "2026-02-20"  # <-- This triggers the update
---
Enter fullscreen mode Exit fullscreen mode
if (existingArticle) {
  if (frontmatter.updatedAt) {
    // Explicit intent to update - proceed
    result = await this.updateArticle(existingArticle.id, frontmatter, body);
  } else {
    // No updatedAt = skip (don't accidentally overwrite)
    console.log(`[exists] ${frontmatter.title}`);
    continue;
  }
}
Enter fullscreen mode Exit fullscreen mode

No updatedAt? No update. Simple opt-in.


Edge Case 4: dev.to Rate Limiting

The Problem

dev.to allows 10 requests per 30 seconds. Try to crosspost 15 articles at once and you'll hit 429s.

The Fix: Delay + Retry with Backoff

// After each successful post
await new Promise(resolve => setTimeout(resolve, 3500)); // 3.5s delay

// On rate limit (429)
if (response.status === 429 && retryCount < CONFIG.maxRetries) {
  const retryAfter = parseInt(response.headers.get('retry-after'), 10) || 60;
  console.log(`[rate-limited] Waiting ${retryAfter}s before retry...`);
  await new Promise(r => setTimeout(r, retryAfter * 1000));
  return this.createArticle(frontmatter, body, retryCount + 1);
}
Enter fullscreen mode Exit fullscreen mode

3.5 seconds between posts keeps us under the limit. If we do hit a 429, respect the retry-after header and try again.


Edge Case 5: Partial Failures

The Problem

You have 5 posts to crosspost. Posts 1 and 2 succeed. Post 3 fails (network error). What happens to posts 4 and 5?

The Fix: Continue on Failure + Report All

for (const file of files) {
  try {
    result = await this.createArticle(frontmatter, body);
    results.push({ action: 'created', title: frontmatter.title, url: result.url });
    await notifySlack(`Crossposted to dev.to: ${title}${result.url}`);
  } catch (error) {
    console.error(`[failed] ${frontmatter.title}: ${error.message}`);
    results.push({ action: 'failed', title: frontmatter.title, error: error.message });
    await notifySlack(`Failed to crosspost "${title}": ${error.message}`, true);
    // Continue to next post - don't abort
  }
}

// Exit with error code if any failures
if (failed > 0) {
  process.exit(1);
}
Enter fullscreen mode Exit fullscreen mode

Every post gets attempted. Every result gets recorded. Every failure gets Slacked. The exit code tells CI whether to retry.


The Complete Slack Notification System

Here's every scenario that triggers a notification:

Event Emoji Message
New crosspost Crossposted to dev.to: {title} → {url}
Updated post Updated on dev.to: {title} → {url}
Date auto-fixed ⚠️ publishAfter updated to today for "{title}" - please pull latest
Duplicate found ⚠️ Duplicate on dev.to for "{title}" - already exists, skipping
Failure ⚠️ Failed to crosspost "{title}": {error}

Slack Integration

One environment variable:

export SLACK_JO4_BLOGS_WH="https://hooks.slack.com/services/T00/B00/XXX"
Enter fullscreen mode Exit fullscreen mode

That's it. The script handles the rest:

async function notifySlack(message, isWarning = false) {
  if (!CONFIG.slackWebhook) {
    console.log(`[slack-skip] ${message}`);
    return;
  }

  const emoji = isWarning ? '⚠️' : '';
  const text = `${emoji} *Jo4 Blog*: ${message}`;

  await fetch(CONFIG.slackWebhook, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ text }),
  });
}
Enter fullscreen mode Exit fullscreen mode

No Slack webhook configured? It logs to console instead. Graceful degradation.


Configuration Knobs

const CONFIG = {
  localMaxDays: parseInt(process.env.LOCAL_MAX_DAYS, 10) || 30,
  devtoMaxDays: parseInt(process.env.DEVTO_MAX_DAYS, 10) || 60,
  maxRetries: 3,
  slackWebhook: process.env.SLACK_JO4_BLOGS_WH,
};
Enter fullscreen mode Exit fullscreen mode
Variable Default Purpose
LOCAL_MAX_DAYS 30 Only process posts from last X days (unless publishAfter is set)
DEVTO_MAX_DAYS 60 How far back to check dev.to for existing articles
SLACK_JO4_BLOGS_WH - Slack webhook URL
DEVTO_API_KEY - Your dev.to API key (required)

The Full Algorithm

1. Fetch all our articles from dev.to (last 60 days)
   → Store by canonical_url for O(1) lookup

2. For each local markdown file:
   a. Skip if draft or crosspost: false
   b. Skip if publishAfter > today (scheduled for future)
   c. If publishAfter is older than 60 days:
      → Update publishAfter to today
      → Queue file for git commit
      → Slack warning
   d. Check if canonical_url exists on dev.to:
      → If yes AND updatedAt is set: UPDATE
      → If yes AND no updatedAt: SKIP
      → If no: CREATE
   e. Wait 3.5 seconds (rate limiting)
   f. On 429: retry with backoff

3. Commit any auto-updated files to git

4. Report summary + exit code
Enter fullscreen mode Exit fullscreen mode

Lessons Learned

  1. Use the destination as source of truth - Don't maintain local state for external systems
  2. Explicit > implicit - Updates require updatedAt flag, no accidental overwrites
  3. Edge cases have edge cases - Old dates need auto-fixing, auto-fixes need git commits
  4. Fail gracefully - Continue on error, report everything, use exit codes for CI
  5. Integrate Slack early - One webhook URL, five notification scenarios, zero config UI

What edge cases have you hit with crossposting? Every automation has that one bug that only shows up at 3 AM.

Building jo4.io - URL shortener with analytics. Our blog auto-crossposts to dev.to using exactly this system.

Top comments (0)