My creator research workflow used to be a lot of copying and pasting between tabs. Scrape in one tab. Export a spreadsheet. Upload it to an AI chat. Prompt. Repeat for comments. Repeat for the website. Every hand-off lost some context, and the AI never saw the full picture.
I published a LinkedIn posts scraper on Apify Store, so the scraping part was already solved. What I wanted was the whole loop — posts, comments, site crawl, analysis — inside a single Claude Code session, with the AI deciding which tool to call next. The Apify MCP server made that possible. It exposes Actors as tools to AI clients over the Model Context Protocol (MCP), so an agent can discover and run them mid-conversation.
One prompt and just under 18 minutes later, I had three files on my disk: a content playbook for a creator I follow, an ideal customer profile (ICP) breakdown of his comment section, and a CSV of his most engaged commenters. Total usage cost: about $0.69.
Here's the whole thing end to end: the pipeline design, the one-time setup, the prompt I used, and what the data actually said. You can point the same prompt at any creator, a competitor, or your own profile.
The pipeline: three Actors, one Apify MCP server
The goal was a full teardown of one creator, and each Actor covers something the other two can't:
- My own LinkedIn Profile Posts Scraper produces the core dataset. Every post comes with 55 fields: full text, engagement broken down by reaction type, content category, ISO timestamps, word counts. I built it because the existing options didn't pre-compute the fields I needed for analysis, like total engagement and content category.
- LinkedIn Post Comments by HarvestAPI adds the audience side. Posts tell you what a creator says; comments tell you who's listening and what they push back on. The hand-off is convenient: my Actor outputs a post_url field, and this Actor takes a list of post URLs as input.
- Website Content Crawler pulls the creator's company site. That answers a question the posts alone can't: does the LinkedIn content match the positioning on his own website?
The MCP server is what makes these three callable in one session. You control which tools it exposes through the tools query parameter in the server URL. Discovery tools (search-actors, call-actor) let the agent find and run any Actor. Platform tools (runs, storage) let it inspect runs and datasets. And Actors pinned by name become first-class tools with inferred input schemas, so the agent knows how to call them before it tries.
Once I had the design, the setup took one command.
Prerequisites
- An Apify account and API token. You'll find the token under Settings → API & Integrations in Apify Console.
- An MCP client. I used Claude Code, but the same URL works in Cursor, VS Code, and Claude Desktop.
- That's it. The hosted server needs nothing installed locally.
Setting up the Apify MCP server
The fastest path is the configurator at mcp.apify.com. You tick the tools and Actors you want, and it generates the server URL for you. Here's the one I built for this pipeline:
Then add it to Claude Code with your token in the Authorization header:
claude mcp add --scope user --transport http apify "<your-url>" --header "Authorization: Bearer <your-token>"
Run /mcp in Claude Code to confirm the server shows up as connected.
One thing to watch, because it cost me a confused twenty minutes: MCP servers in Claude Code have scope. My first ever install registered at local scope, tied to the directory I happened to be standing in. When I later launched a session from a different folder, the server simply wasn't there. The agent, being resourceful, fell back to calling the Apify REST API directly instead of telling me. claude mcp list showed the problem in seconds.
The --scope user flag makes the server available in every project. Check /mcp before any serious run. (If you have a one-off scraping task, local scope is fine but if you don’t want to keep configuring this with each project --scope user is needed.)
The master prompt
The prompt is doing four deliberate jobs, and each one came from a mistake I made at least once:
- A CONFIG block at the top. Every knob — creator, post limits, comment depth, site — is a variable. Changing targets means editing one line.
- Schema confirmation before every call. The agent must call fetch-actor-details before running each Actor. Input schemas differ between Actors, and guessing field names is how runs fail.
- Stats computed in code. AI models are bad at mental arithmetic over 141 rows. The prompt forces the agent to write a Python script for the quantitative work, so every number in the report is exact.
- Deliverables as files. Chat output scrolls away. Files persist, diff, and import into other tools.
Here's the full prompt. Copy it, edit the CONFIG block, and run it:
\`
# CONFIG: edit these
CREATOR = "anthonypierri" # LinkedIn username or profile URL
POSTS_LIMIT = 150 # posts to scrape (Actor supports 1-2,000)
COMMENT_POSTS = 10 # most recent posts to pull comments from
COMMENTERS_PER_POST = 10 # comments per post
SITE = "https://fletchpmm.com" # creator's company site
SITE_PAGES = 20 # max pages to crawl
You have Apify MCP tools. Run this end to end, autonomously, no manual steps:
1. Before calling each pinned Actor, call fetch-actor-details to confirm its exact input schema. Do not guess input field names.
2. Run capable_cauldron/linkedin-profile-posts-scraper with usernames=[CREATOR], maxPostsPerProfile=POSTS_LIMIT. Wait for completion. Record run ID, dataset ID, item count, and cost.
3. Pull the dataset with get-actor-output and save it as creator_posts.json.
4. Write and run a Python script computing exact stats over ALL posts: engagement distribution, median baseline, outliers (state the multiple you used), breakdown by content_category, cadence from posted_date_iso, word-count patterns, hashtag usage. Save it as analyze.py, output stats.txt.
5. Take the COMMENT_POSTS most recent posts by posted_date_iso, feed their post_url values to harvestapi/linkedin-post-comments with maxItems=COMMENTERS_PER_POST. Save as comments.json.
6. Run apify/website-content-crawler on SITE capped at SITE_PAGES pages. Save as site.json.
7. Write three deliverables:
a. report.md - the playbook: bottom line first, winning formats/angles/hooks with real numbers and quoted posts, outlier teardowns, stop/continue/start, and an SOP for the next post.
b. audience.md - ICP breakdown from the comments: roles, seniority, company types, recurring questions and objections.
c. commenters.csv - one row per commenter: name, headline, profile URL, post URL, comment text, date.
8. Close with a run summary: every run ID, dataset ID, item counts, total cost.
`\
Running the teardown
I pasted the prompt into Claude Code and watched it work through the stack.
First, it called fetch-actor-details on all three Actors before touching any of them, the schema rule paying off on the very first call. Then it launched the posts scrape and confirmed the other two schemas in parallel, instead of waiting serially. Small thing, but it's the kind of parallelism a manual workflow never gets.
The posts run finished in 25 seconds with 150 activities (run ID kGpFgheB19B1X893i). Nine of them are other people's posts he amplified, which author_username separates out, leaving the 141 he wrote. Worth doing before you analyze anything: one amplified post pulled 1,218 engagement, enough to sit second on his outlier list and skew every baseline.
Then the agent did something I hadn't asked for. Instead of paging 150 post bodies through the MCP connection into its context, it downloaded all 509 KB to disk and analyzed it locally. Tool results inflate context; files don't. That instinct is why I trust this setup with bigger scrapes.
With the data local, it wrote analyze.py and ran it over all 141 posts, exactly what the compute-in-code rule was there for. The core of the script:
\`
import json, statistics as st
posts = json.load(open("creator_posts.json", encoding="utf-8"))
eng = [p["total_engagement"] for p in posts]
baseline = st.median(eng)
outliers = [p for p in posts if p["total_engagement"] >= 3.0 * baseline]
print(f"posts: {len(posts)}, median: {baseline}, outliers: {len(outliers)}")
`\
Every number in the findings below comes from code like this.
The comments run pulled 98 comments from the 10 most recent posts, and the crawler took 20 pages of the creator's site. Seventeen minutes and fifty-nine seconds after the prompt went in, the run summary landed: three Actors, three datasets, three deliverables, about $0.69 of usage.
Here's where that goes, at the list prices you'd pay on Apify's free tier. The posts scrape is $0.003 per post and $0.01 per profile, so 150 posts came to $0.46. The comments Actor charges $0.002 per comment, putting 98 comments at $0.20. The crawler bills compute rather than results, and 20 pages ran $0.03. Both Actors price lower on higher usage tiers, where the per-post rate drops to $0.001, so the same run gets cheaper as your volume grows.
One habit I'm taking from this run: while building the commenter CSV, the agent noticed the comments Actor's engagement.likes field is zero on every row. The real counts live in a nested engagement.reactions[] array. Had it trusted the obvious column, every comment would have looked equally unloved. Whatever Actor you use, diff a row or two against the live site before you analyze a thousand of them.
That's the whole run. Now for what the data said.
What 141 posts and 98 comments actually said
Quick framing so the numbers mean something: the corpus is 141 posts over 265 days, median engagement 215 (reactions + comments + reposts). I defined an outlier as any post at 3× the median or better, which works out to 645 engagement. Eight posts clear that bar.
He is a comment machine, and that is the moat
The median post gets 160 reactions and 57 comments, a 0.36 comment-to-reaction ratio. For a B2B account that's an unusually large share of engagement landing as comments instead of reactions. The 141 posts carry 10,095 comments between them, an average of 72 each, and the top posts pull 150–280. Comments are where prospects self-identify. An account like this is as much a lead-gen asset as a content play.
Naming companies beats abstract advice
Posts that name a real company (Attio, Okta, Gong, Figma, DocuSign, Salesforce) run at a 242 median (1.13× baseline) and produced 5 of the 8 outliers from just 30.5% of posts. What works is specificity with skin in the game: his "We are Attio customers" post, critiquing a tool he pays for, did 524 engagement with 208 comments. The abstract advice posts never break out. The concrete ones do.
The humor barbell
I measured humor by the audience's own 😂 reactions rather than my judgment, and the result is a genuine barbell. Two buckets win: pure comedy at 40%+ funny reactions (315 median, 1.47× baseline, 4 of the 8 outliers) and the dry end at 0–5% (302 median, 1.40×, 3 outliers).
Everything between them is dead ground, running 0.88–0.92× across 24 posts with not one outlier. Writing with no humor at all is no better: the 60 posts nobody laughed at sit at 0.87×, the weakest bucket in the corpus. Commit to the bit or keep it dry. Half-joking is the only approach that never works.
Documents raise the floor, text raises the ceiling
Carousels are the most reliable format: 21 posts, 252 median, a 14.3% outlier rate, the best of any format. Images are 42% of his output with a 1.7% outlier rate, the biggest block of safe, forgettable volume. Plain text is the opposite trade: the worst median (157, 0.73×) but 3 outliers, including the #1 post in the corpus: "Claude Design just KILLED Figma.", 1,282 engagement, 63% funny reactions, pure deadpan. If you want a lottery ticket, it's text.
Question hooks are dead
Fifteen posts opened with a question. Median: 147, a 0.68× index, zero outliers. "Which is it?" scored 75. "Guess the company" scored 75. If the idea is a question, answer it in the hook and let the comments argue.
The cadence trade
Set the first quarter of the corpus against the last and he tripled his posting cadence, from ~2 posts a week to ~7. Per-post median fell 30% over those same quarters (281 → 197), while mean weekly engagement roughly doubled (~735 → ~1,657). The volume trade works in aggregate. The cost shows up when you split the corpus into halves instead: outliers dried up from 6 in the older half to 2 in the newer. He's farming the median and starving the tail. The fix is ring-fencing two slots a week for the two formats that actually break out.
The audience is peers, not buyers
The comment section looks like a goldmine until you read the headlines. Of 86 unique commenters, roughly 28% are agencies, consultants, and freelancers selling adjacent services. Nearly half, 47.7%, don't state a job title at all; their headline is a pitch. Every single repeat commenter is a peer. The unambiguous buyers, founders and marketing leads at funded B2B software companies, are about 11.6% of the room.
And one finding that surprised me: he replied to zero of the 98 comments I sampled. That left 21 experience-backed objections publicly unanswered — including four independent, specific rebuttals of his "don't position against competitors" advice from practitioners with war stories. That's not a troll problem. It's the strongest post idea in his whole dataset, already written for him by his own audience.
The three deliverables
Everything above came out of report.md, the first deliverable. It’s the playbook, with the outlier teardowns and a step-by-step standard operating procedure (SOP) for the next post.
The second, audience.md, is the ICP breakdown: roles, seniority, company types, and the recurring questions and objections, with verbatims.
The third, commenters.csv, is the actionable one: 86 commenters with name, headline, profile URL, the post they commented on, and what they said:
| name | headline | comment_text | date |
|---|---|---|---|
| A. K. | Senior Product Manager, enterprise process-mining vendor | "how long does it usually take before the sales team starts begging for this? we went six months avoiding the competitor comparison…" | 2026-08-05 |
The name and employer are masked in that row but sit in full in the CSV, which is exactly why the note at the end of this article exists. That CSV drops straight into Google Sheets, and it's the piece a pure content analysis never gives you: a warm list of people who just proved, in public, that they care about the problem you solve. If you want the export automated instead, Apify has a native Google Sheets integration you can attach to the Actor run.
Pointing the whole thing at a different target is the CONFIG block: change CREATOR and SITE, rerun. Your own profile, a competitor, a prospect's champion — the prompt doesn't care.
Honest limitations
Three caveats, because the numbers deserve them.
First, the comments sample is 98 of the 561 comments on those 10 posts, and LinkedIn's "most relevant" ordering skews it to the top of each thread. The proportions are directional, the verbatims exact.
Second, 8 outliers is 8 data points; every pattern in the teardown is a strong association across a small tail, not causation.
Third, you're trusting the analysis code an agent wrote. That's exactly why the prompt makes it save analyze.py as a file. Read it before you quote the report to anyone.
What's next
My next step is running this weekly, scheduled, using the runs and storage tool categories to diff each week's dataset against the last — the same pipeline, pointed at my own niche, watching for what breaks out. The prompt, the analysis scripts, and the setup notes are in the GitHub repo accompanying this article.
The bigger shift for me: my Actor stopped being a thing I run and became a thing my agent reaches for. None of the three Actors changed. What changed is that something can now chain them together, and honestly, that stitching is the product.
FAQ
Do I need to build my own Actor to use the Apify MCP server? No. Any Actor on Apify Store works — pin it by name in the tools parameter, or let the agent find one with search-actors and run it with call-actor. Building my own posts scraper helped because I control the output schema, but it's not a requirement.
What does a run like this cost? About $0.69 on the free tier for 150 posts, 98 comments, and 20 crawled pages, and less on higher usage tiers where the per-post rate drops. Comment volume is the part that scales, at $0.002 per comment, so even a deep dive on a heavy commenter stays in pocket-change territory.
Does this work with Cursor or Claude Desktop instead of Claude Code? Yes. The hosted server URL is client-agnostic. The configurator at mcp.apify.com generates config snippets for the major MCP clients, and the setup is the same: URL plus your token in the Authorization header.
A note on ethics and personal data: everything scraped here is publicly available, meaning public posts, public comments and a public website, collected with no cookies and no logged-in session. Public does not mean unregulated, though. A commenter list is personal data about named individuals, so building one and using it for outreach brings obligations with it: a lawful basis under GDPR or CCPA, disclosure of where you got the data when you contact someone, and an opt-out and deletion request you actually honor. That's why the commenter in the sample table above is masked. Scrape responsibly and respect each platform's terms of service.
About the author: Naveen Choudhary builds automation pipelines that turn public web data into go-to-market assets. He published the LinkedIn Profile Posts Scraper on Apify Store, built after existing scrapers didn't pre-compute the engagement fields he needed for analysis. His work sits at the intersection of web scraping and AI agents: chaining Apify Actors through the Apify MCP server so an agent runs an entire research workflow from a single prompt. When he's not scraping LinkedIn, he's helping GTM teams automate their research and lead generation.
Top comments (0)