DEV Community

Francis Oyakhire
Francis Oyakhire

Posted on

Publishing Pipeline Scheduled vs Immediate

This week’s news about AI model inference and deployment strategies has sparked a broader conversation about control, cost, and complexity in modern software stacks. As we look at the tradeoffs between scheduled and immediate publishing pipelines, we find a similar tension: when do we let the system decide, and when do we take the wheel? At Apex Grid, we’ve built a hybrid approach that balances the need for timely content delivery with the flexibility of scheduling - and it’s not without its challenges.

We run a platform that handles both immediate publishing - like real-time analytics dashboards - and scheduled content - such as social media posts and long-form articles. For the latter, we needed a way to publish content at a specific time, but also ensure that the content remains consistent across all platforms. That’s where the canonical URL problem comes in.

When a post is scheduled, it’s stored in a draft state, and only published when the cron job fires. However, if a user shares a preview link to the draft before it’s published, that link becomes invalid once the post goes live. This creates a mismatch between the canonical URL (which is the final, published version) and the URL used in the preview. To solve this, we implemented a redirect system that maps draft URLs to their published counterparts once the post is live.

Here’s a simplified version of how we handle the redirect in our backend:

from flask import Flask, redirect, request
import re

app = Flask(__name__)

# In-memory mapping of draft URLs to published URLs
draft_to_published = {
    "/drafts/2026-04-05-hello-world": "/posts/2026-04-05-hello-world",
    # ... more mappings
}

@app.before_request
def handle_redirects():
    path = request.path
    if path in draft_to_published:
        return redirect(draft_to_published[path], code=301)
Enter fullscreen mode Exit fullscreen mode

This approach works well for small-scale use cases, but it has limitations. The in-memory mapping is not scalable and doesn’t handle edge cases like deleted posts or posts that are moved. To address this, we’re moving toward a more robust solution that uses a database to store and manage the canonical URL mappings. This will allow us to query and update mappings dynamically, and also add features like soft-deletes and versioning.

The tradeoffs are clear. Using a database adds complexity and latency, but it also provides consistency and reliability. We’ve found that the benefits of a centralized, scalable system outweigh the costs, especially as our platform grows and the number of scheduled posts increases.

Looking ahead, we’re exploring ways to automate the canonical URL mapping process. One idea is to generate the canonical URL based on the post’s metadata, such as the title and publication date. This would reduce the need for manual mappings and make the system more self-sufficient. We’re also considering integrating with reverse proxies to handle redirects at the network level, which could improve performance and reduce the load on our application servers.

What do you think about the tradeoffs between manual and automated canonical URL mapping? And how would you handle the problem if you were building a hybrid publishing pipeline for a global audience?

Top comments (0)