DEV Community

agenthustler
agenthustler

Posted on • Edited on

Building a Social Media Cross-Poster Bot with Web Scraping

#webdev #programming #python #tutorial

Building a Social Media Cross-Poster Bot with Web Scraping

Posting the same content across Twitter/X, LinkedIn, Bluesky, and Mastodon manually is tedious. Let us build an automated cross-poster that adapts content for each platform.

Architecture

  1. Content source: Your blog RSS feed, a Google Sheet, or manual input
  2. Adapter layer: Reformats content per platform (character limits, hashtags, mentions)
  3. Poster layer: API clients for each platform
  4. Scheduler: Optimal posting times per platform

Setup 

pip install requests beautifulsoup4 feedparser tweepy atproto schedule
Enter fullscreen mode Exit fullscreen mode

Content Source: RSS Feed Scraper 

# Implementation is proprietary (that IS the moat).
# Skip the build — use our ready-made Apify actor:
# see the CTA below for the link (fpr=yw6md3).
Enter fullscreen mode Exit fullscreen mode

Platform Adapters 

class ContentAdapter:
    LIMITS = {
        "twitter": 280,
        "bluesky": 300,
        "linkedin": 3000,
        "mastodon": 500
    }

    def adapt(self, post, platform):
        limit = self.LIMITS.get(platform, 280)

        if platform == "twitter":
            return self.format_twitter(post, limit)
        elif platform == "bluesky":
            return self.format_bluesky(post, limit)
        elif platform == "linkedin":
            return self.format_linkedin(post, limit)
        elif platform == "mastodon":
            return self.format_mastodon(post, limit)

    def format_twitter(self, post, limit):
        hashtags = " ".join(f"#{t}" for t in post["tags"][:3])
        url = post["url"]

        available = limit - len(url) - len(hashtags) - 4
        text = post["summary"][:available]

        return f"{text}\n\n{url}\n{hashtags}"

    def format_bluesky(self, post, limit):
        text = post["summary"][:limit - len(post["url"]) - 2]
        return f"{text}\n{post['url']}"

    def format_linkedin(self, post, limit):
        hashtags = " ".join(f"#{t}" for t in post["tags"][:5])
        return f"{post['title']}\n\n{post['summary']}\n\n{post['url']}\n\n{hashtags}"

    def format_mastodon(self, post, limit):
        hashtags = " ".join(f"#{t}" for t in post["tags"][:4])
        text = post["summary"][:limit - len(post["url"]) - len(hashtags) - 4]
        return f"{text}\n\n{post['url']}\n{hashtags}"
Enter fullscreen mode Exit fullscreen mode

Platform Posters 

# Implementation is proprietary (that IS the moat).
# Skip the build — use our ready-made Apify actor:
# see the CTA below for the link (fpr=yw6md3).
Enter fullscreen mode Exit fullscreen mode

The Cross-Poster Bot 

import time
import schedule

class CrossPoster:
    def __init__(self):
        self.source = ContentSource()
        self.adapter = ContentAdapter()
        self.platforms = {}
        self.posted = set()

    def add_platform(self, name, poster):
        self.platforms[name] = poster

    def cross_post(self, post):
        post_id = post["url"] or post["title"]
        if post_id in self.posted:
            return

        results = {}
        for platform_name, poster in self.platforms.items():
            try:
                content = self.adapter.adapt(post, platform_name)
                url = poster.post(content)
                results[platform_name] = {"status": "success", "url": url}
                print(f"Posted to {platform_name}: {url}")
                time.sleep(5)  # Stagger posts
            except Exception as e:
                results[platform_name] = {"status": "error", "error": str(e)}
                print(f"Failed on {platform_name}: {e}")

        self.posted.add(post_id)
        return results

    def auto_post_from_rss(self, feed_url):
        posts = self.source.from_rss(feed_url, limit=1)
        for post in posts:
            self.cross_post(post)

# Setup
bot = CrossPoster()
bot.add_platform("bluesky", BlueskyPoster("you.bsky.social", "password"))
bot.add_platform("mastodon", MastodonPoster("https://mastodon.social", "token"))

# Post manually
bot.cross_post({
    "title": "New blog post",
    "url": "https://yourblog.com/post-1",
    "summary": "Just published a guide on building cross-posting bots with Python!",
    "tags": ["python", "automation", "socialmedia"]
})

# Or auto-post from RSS every hour
schedule.every(1).hours.do(bot.auto_post_from_rss, "https://yourblog.com/feed")
Enter fullscreen mode Exit fullscreen mode

Scraping Optimal Posting Times 

def scrape_best_times():
    """Research suggests these general windows work well."""
    return {
        "twitter": ["9:00", "12:00", "17:00"],
        "linkedin": ["7:30", "12:00", "17:30"],
        "bluesky": ["10:00", "14:00", "19:00"],
        "mastodon": ["8:00", "13:00", "18:00"]
    }
Enter fullscreen mode Exit fullscreen mode

Proxy Services for Scraping

When scraping social media for content research:

  • ScraperAPI — handles JS rendering for Twitter and LinkedIn
  • ThorData — residential proxies for social media access
  • ScrapeOps — monitor your scraping pipeline

Conclusion

A cross-poster bot saves hours of manual work each week. Start with two platforms, perfect the formatting, then expand. Always respect platform rate limits and avoid spammy behavior.

Follow for more Python automation projects!

Top comments (0)