DEV Community: George Kioko

Google now ranks the same story differently in Brussels and Dallas. your brand monitor only watches one of them

George Kioko — Sat, 29 Aug 2026 12:37:57 +0000

i build scrapers for a living and one of the cheapest ones i run is a google news monitor. this week google handed everyone who pays for media monitoring a reason to check what the money actually buys.

what happened: on august 28 google said that from august 30 it stops applying site reputation manual actions for people searching inside the european economic area. outside the EEA the penalty keeps working. that is the outcome of a DMA investigation by the european commission, and the wire coverage framed it plainly, google changed the policy to avoid a fine.

so the same query, on the same day, can rank a page for someone in brussels and bury it for someone in dallas. not a ranking wobble. a policy line drawn by geography.

why a monitoring tool cares

every brand and media monitoring product i have looked at pulls its google results from one place and calls it google. one location, one profile, one snapshot. that was always a simplification. from august 30 it is wrong by design for any site under a site reputation action, and that list is heavy with big publishers hosting partner content, which is exactly the kind of site your brand gets mentioned on.

the second half of the problem is older and bigger. chartbeat numbers cited in the reuters institute 2026 trends report put google organic traffic to publishers down about a third year over year through november. sparktoro measured 68 percent of google searches in the first four months of 2026 ending with no click at all. the story about your brand is increasingly read inside an AI answer, inside discover, or not clicked. a count of blue links tells you less every quarter.

what the paid tools charge

brand24 runs 199 to 1,499 a month billed annually, 20 to 25 percent more if you pay monthly. mention shut its self serve tiers in july 2025 and now sells one company plan at 599 a month, annual only, with historical data and API access sold separately. meltwater does not publish prices, which tells you what you need to know.

strip the dashboards off and what you rent is this: a poller that hits news sources for your keywords, drops duplicates, and alerts you. sentiment scoring and share of voice charts sit on top. if your job is reporting those charts upward, pay for them, they are real work. if your job is knowing the moment someone writes about you or your competitor, you are renting a cron job.

the build

google news exposes a feed for any query. poll it on a schedule. filter for your terms. dedupe by url, because the same story gets republished under different urls as it syndicates, and that dedupe step is the only part with any real logic in it. bad dedupe is why cheap monitors feel broken, the same article hits you four times. push new ones to email or slack.

the part that changed this week: poll from more than one country. an EU exit and a US exit now see different results for penalized domains, and the gap between them is itself a signal. a story that ranks in europe and vanishes in the US is telling you something about the publisher, not about you.

honest limits

this is news, not social. it does not read reddit or x. it does not score sentiment. and google news is not every outlet, trade press with no feed still needs its own scraper.

what i run

mine is an actor. keywords and a time range in, deduped articles out with title, source, date, snippet and optional full text. priced at $0.003 per article found, no subscription. over the last 30 days it ran a couple hundred times, almost all on schedules, with a 100 percent success rate, which is the whole point of a monitor. ten brands and five competitors for less than one seat of the paid thing.

https://apify.com/george.the.developer/google-news-monitor?fpr=bbquoh

sources: google search central on the policy update, search engine journal on the EEA carve out, reuters institute trends and predictions 2026 for the chartbeat figure, sparktoro for the zero click rate, and the vendors own public price pages for brand24 and mention.

most of your lead list doesnt exist

George Kioko — Thu, 27 Aug 2026 09:36:39 +0000

i tested an email enrichment waterfall on a real batch in july. the exact setup every ai sdr thread still swears by. find rate: 4.9 percent. of the addresses that came back, 70.3 percent were catch all. they accept everything and tell you nothing about whether a human is on the other side.

out of a hundred people i actually wanted to reach, i got maybe one address i would put money on. the rest was noise dressed up as data.

nobody selling enrichment publishes numbers like these. they publish match rates. a match is not a person. it is one database row nodding at another database row. both can be years old, both can be wrong, and the charge hits either way.

growth twitter is admitting it now. threads this week are full of people saying their scraped lists bounce, their domains burn, their reply rates live under one percent. the tools didnt fail. the premise failed.

the premise is simple and wrong: that a strangers inbox is worth guessing. it isnt. email data rots the day someone changes jobs, and people change jobs all the time. you are buying photos of a crowd that already left.

what holds is behavior you can watch happen.

someone liked or commented on a competitors post yesterday. they told you, in public, with a timestamp, that they care about the problem you solve. no guessing required. no enrichment. no bounce risk. they raised a hand where anyone could see it.

someone just started as head of the function you sell into. fresh mandate, budget still unspent. or someone senior just left and the team they ran is about to change how it buys. all of that lives in public profile data, updating in real time. almost nobody treats it as the lead source it already is.

this data is easier to pull than emails ever were. engagement is public. job moves are public. you dont need a chain of vendors reselling each other. you just watch the surface where buyers already show themselves.

i build tools for that. one grabs everyone who engaged with any linkedin post, likers and commenters, into a clean list: https://apify.com/george.the.developer/linkedin-post-engagers-scraper?fpr=bbquoh. another tracks companies and flags new hires and departures as they land: https://apify.com/george.the.developer/linkedin-company-employees-scraper?fpr=bbquoh. the rest sit at pipelines.trykioko.com.

run your own numbers on your own list before you buy another credit pack. count the catch alls. count the bounces. then count how many of your best customers ever arrived from a cold guessed inbox versus someone already circling in public.

the list you need is not sitting in a database. it is performing in public every day. almost nobody is writing it down.

every layoff is announced twice

George Kioko — Tue, 25 Aug 2026 12:33:01 +0000

apple cut about 200 roles last week. roughly 100 from vision pro gaming and immersive video, roughly 100 from siri and an ai integration group. bloomberg broke it. apples statement says the company is realigning teams "to evolve our business to deliver the best experiences for our users."

read that again. it says nothing. thats not a bug, its the product. corporate comms exists to turn a dead bet into a strategy update.

the layoff was announced twice. once in the memo. and once, months earlier, in data almost nobody watches.

the script is always the same.

the decision gets made in a boardroom, sometimes two quarters before anyone gets the invite.

then hiring stops in the affected org. not company wide, just there. open roles quietly close. nobody announces a freeze. the postings just stop renewing.

then the senior people start leaving. a director this week, a principal engineer two weeks later. each one looks like a normal career move. clustered inside one org in one quarter, its a fire alarm.

then the recruiters attached to that org move or disappear. recruiters exist to grow teams. a team that is done growing cuts them first, and it shows before anything else does.

then, months after the actual decision, the memo. the press treats it like news. it isnt. its an obituary. the death already happened.

announcements are written to be read. thats the problem. every word passed legal, comms and investor relations. headcount data passed nobody. people join, people leave, the profiles update. no pr team edits the public org chart.

which is why the interesting version of the apple story was never going to come from apple. around 100 of those cuts reportedly hit vision pro gaming and immersive video. thats not realigning. thats the quiet end of an expensive bet. the org chart said it clearly while the statement said nothing.

four things worth watching. all of them sit in public profile data.

departures clustering in one team. normal attrition spreads. trouble concentrates.

the seniority of who leaves. juniors leave for money, thats noise. when directors and principals go in a cluster, they know something the rest of us dont yet.

recruiter roles closing. first thing cut when growth stops, and it happens before any announcement.

where the new hires land. one org absorbing people while another gets nothing means the budget already moved. budgets move before memos do.

if youre job hunting this tells you which team not to join. an offer from a dying org still sounds great in the interview.

if you sell b2b, a team bleeding seniors isnt buying this year. a team hiring fast just told you it has budget without saying a word.

if you just work in tech, watching the data beats doomscrolling layoff threads. the market isnt uniformly cooked. specific bets at specific companies are dying while other teams still cant hire fast enough, and the difference is visible from outside.

i got tired of refreshing linkedin by hand, so i built a tracker that watches hires and departures at companies i care about and flags when the pattern changes: https://apify.com/george.the.developer/linkedin-company-employees-scraper?fpr=bbquoh

the rest of my tools live at pipelines.trykioko.com.

next time a company you care about makes headlines, go look at what its org chart did in the three months before. the memo is always late.

How to export LinkedIn post likers and commenters into a lead list (no login needed)

George Kioko — Fri, 21 Aug 2026 11:00:54 +0000

Nobody wants to say this plainly so i will. the AI SDR experiment is coming back with results now, and theyre ugly. one founder asked this week if any of these tools actually book meetings. the replies he counted: out of office autoresponders and No's. months of automation, thats the harvest.

same week, operators started posting their enrichment bills in public. one did the math on a basic search enrichment and landed on a 250x markup over the raw api cost. thousands a month to enrich the same cold lists every competitor is enriching with the same tools.

the tools arent the problem, the lists are

every one of these stacks points at cold data. export from a database, enrich it, let AI write something that pretends to be personal, spray. the person on the receiving end gets forty of these a day now, half of them from profiles with AI generated faces. cold is cold no matter how much AI you rub on it.

meanwhile the actual buying signal is sitting in public, on LinkedIn, refreshed daily, and almost nobody harvests it.

every post in your niche is a lead list

when someone reacts to a post about, say, cold email deliverability, they just told you three things. they care about this exact topic, they care about it this week, and theyre active on LinkedIn. thats a hand raised in public.

the people commenting under your competitors launch post are not a cold list. they are the warmest b2b list that exists, self sorted by interest, timestamped, with their job title attached.

the manual version, free, works today

find 3 posts in your niche that pulled real engagement, competitor launches and hot takes work best
open the reactions list and the comments
copy out the people whose headline matches your buyer, founder, head of growth, whatever your ICP is
message them about the topic they just engaged with, not about you

thats 20 minutes per post and it beats a thousand row cold export, no doubt about that.

the automated version

i got tired of doing the manual version, so i built it as an Apify actor: LinkedIn Post Engagers Scraper.

paste post urls, get back the engagers, deduped across posts, with an ICP filter on headlines so you only pay for people who match your buyer. half a cent per matched lead. and it never asks for your LinkedIn login, which matters now that account restrictions are the number one complaint in every outreach thread.

first paying users ran it start to finish in week one, every run succeeded. its new, its small, it does one thing.

the rest of my lead pipelines live at pipelines.trykioko.com.

signals beat volume

the channel isnt dead. the lists are. stop paying enrichment markup on strangers and start messaging the people already raising their hands.

Every paid API tutorial ships a rate limit you can beat in four seconds

George Kioko — Mon, 17 Aug 2026 11:38:52 +0000

There is a whole genre of tutorial that takes you from a database table to a paid public API in about twenty minutes. Table of data, a keys table, hash the key, put an edge function in front, add a rate limit, ship it. The good ones are genuinely good. The key handling in particular is usually correct, and correct here matters more than it sounds.

But almost every one of them ends with a rate limit you can walk around in four seconds, and most of them do not tell you.

The part they get right

Store a fingerprint of the key, never the key. You hash it, you keep the hash, you keep a short prefix so a customer can tell their own keys apart in a dashboard, and you hand the real string to the customer exactly once. If your database leaks, nobody gets a working key out of it.

That is not paranoia, that is the same shape Stripe and everyone else uses, and it costs you one hash function. Generate the key inside the database rather than in a browser so the raw value never travels further than it has to. Put row level security on the keys table so a user can only ever see their own rows. Keep the service role key inside the edge function, never anywhere a browser can reach it.

None of that is advanced. All of it is worth doing on day one, because retrofitting key hygiene after you have customers is miserable.

The part that quietly does not work

Then comes the rate limit. Count the requests for this key in the last minute, and if the count is over your limit, return a 429.

Read that again. For this key.

Your customer is sitting in a dashboard that lets them mint new keys whenever they like. They hit the limit, they click generate, they get a fresh key with a fresh counter, and they carry on. The limit measured the key. The person was never limited at all.

I am not describing a hypothetical. This exact bypass gets demonstrated live in tutorials, hit the ceiling, mint a new key, keep pulling data, and the narration moves on. Sometimes the author flags it as something to solve later. Later never arrives, because the reader has already copied the code.

Count the account, not the credential

The fix is a one word change in your head and a small change in your schema. A key is not a customer. A key is one of several credentials that belong to a customer. So the counter belongs on the customer.

Your keys table already stores the user id that owns each key. When a request arrives, you hash the key, look it up, and you now know two things, that the key is valid and who it belongs to. Do the counting on the second one. Every key that account owns draws from the same bucket.

Once you do that, minting a new key gets you nothing, which is exactly what you want, because minting a new key was never supposed to be a purchase.

While you are in there, cap how many keys an account can hold at all. Nobody needs four hundred. Rotation needs two. And keep the counter in a table the customer cannot write to. A limit stored somewhere the customer can edit is not a limit, it is a suggestion.

The bigger thing hiding underneath

Here is what took me too long to understand while building paid data tools. A rate limit is not billing. It is a blast shield. It stops one bad actor taking your service down, it does not decide what anyone owes you.

Metering is what you actually sell, and metering has a rule that rate limiting does not. You must never charge for a request that produced nothing useful.

That sounds obvious and almost nobody does it. Every broken data product i have ever taken apart, including two of my own, had the same defect. Loose match, return something anyway, charge for it anyway. Asked for one company, got a similar looking one, billed as a hit. Asked for a profile, got a page that merely mentioned the name, billed as a hit.

The honest version is not harder to build. Emit everything you found, mark what did not clear the bar, and charge only above the bar. Your dataset gets a row saying this one is free and here is why. Customers notice this immediately and it buys more trust than any amount of uptime.

Most of the time the confidence score you need is already sitting in your code, computed and then ignored. Go and look. i have found it every single time.

Three more things the twenty minute version skips

Version the path from the first request you ever serve. Once someone else's code depends on your response shape, changing that shape is not an improvement, it is an outage you caused. Ship v1 in the url and you can build v2 next to it without breaking anybody.

Paginate by default, and let people filter and sort on your side. If your answer to a thousand rows is here are a thousand rows, every customer builds the same filtering logic separately and all of them think your API is slow.

Return honest status codes. Bad key is a 401. Missing thing is a 404. Over the limit is a 429. Never a 200 with an error hidden in the body, because the caller's code reads the number before it reads anything else, and a lie there turns into a support ticket a week later when their pipeline has been silently storing failures.

And the thing nobody wants to hear

You can do all of this perfectly and still make no money, because none of it is the hard part.

The hard part is that a stranger has to find your API, believe it works, and hand over a card. i maintain seventy six of these things. The ones that earn are not the ones with the best code. They are the ones where somebody types a phrase into a marketplace search box and my title is the thing that comes back.

Build the key system properly, it takes an afternoon. Count against the account, not the key. Charge only for what worked. Then spend the rest of your time on the part that is actually hard, which is being findable.

The proxy industry needs you to never open the network tab

George Kioko — Wed, 05 Aug 2026 15:51:09 +0000

I run 75 scrapers in production. Three of them do any fingerprint spoofing. Maybe five use residential proxies. The rest run on plain datacenter IPs or no proxy at all, and they have been running for months.

If you learned scraping from blog posts, that number probably sounds wrong to you. Every tutorial you have read starts the same way: sign up for a residential pool, install a stealth browser, randomize your fingerprint, throttle like a human. Then, on step five, you finally get to look at the actual website.

That order is backwards, and it is backwards on purpose. Proxy companies write most of the scraping content on the internet. They were never going to write "you probably do not need us."

The scraper with the $80 a month costume

Last month my guy sent me his Greenhouse job board scraper to fix. It had everything. Puppeteer with the stealth plugin. Rotating residential proxies. Randomized mouse movements between actions. Human-like typing delays.

It still kept dying.

So I did the thing nobody had done: opened the page in a normal browser with devtools up. The entire job list was sitting in one XHR request to a public JSON endpoint. No auth. No cookies. A rate limit so loose I never managed to hit it.

I deleted basically his entire codebase and replaced it with a fetch call. It has not broken since. He had been paying for proxies for months to hit an endpoint that does not care who you are.

This was not a rare lucky case. This is most cases.

The 20 minute method

What I do on every new target, before writing a single line of code:

Open the network tab, filter to XHR/fetch.
Reload the page. Click around. Paginate. Search.
Find the request that returns the actual data. It is usually JSON and usually obvious.
Right click, copy as cURL.
Paste it in a terminal and start deleting headers one at a time. Rerun after each delete.

Whatever survives step five is your scraper. Most of the time the answer is a user agent header and nothing else. Sometimes a referer. Occasionally one cookie you can grab with a single cheap request first.

Twenty minutes. No browser automation, no proxy bill, and the result is faster and more stable than any puppeteer setup, because JSON endpoints change way less often than HTML markup does.

Half the "protected" sites out there are react apps sitting on a wide open API. The frontend is the security theater. The data is just there.

When you actually need the heavy stuff

I am not saying anti detect tooling is fake. I am saying it is a last resort that got marketed into a first step. You genuinely need it when:

The protection sits on the API itself, not the page. Turnstile or a challenge token attached to the data endpoint.
The site checks TLS fingerprints, so plain curl or node fetch gets flagged before your headers even matter.
Sessions are bound to browser fingerprints, ticketing platforms mostly.
You need real logged-in browsing at scale.

In my portfolio that is maybe 1 target in 10. For those, sure, pay for the pool. For the other 9, the network tab was free the whole time.

The tell

You can tell how someone learned scraping by what they reach for first. People who learned from vendor blogs can configure a fingerprint rotator but have never stripped a cURL command down to its minimum headers. People who learned by breaking things do it the other way around.

Try the boring thing first. It costs nothing, and most of the time it is the whole job.

I write about this stuff and post scraping methods on r/scrapingtools if you want more of it.

Nobody voted for the company that now licenses the readable web

George Kioko — Wed, 29 Jul 2026 10:41:42 +0000

i write scrapers for a living, so read this knowing im biased. but something happened on july 1 that got covered as a pricing update, and it is not a pricing update.

Last time i wrote about the big labs pulling the ladder up behind them after climbing it. This is the same story from a different seat. Not a company fighting over data it trained on. A company that sits in the road deciding who is allowed to walk down it.

Start with what they got right

Most takes on this are lazy, so let me concede the strong part first.

Cloudflare moved from pay per crawl to pay per use. Under the old model a bot paid a fee to fetch a page, minimum a cent per retrieval. Under the new one, publishers get paid when their content actually shows up inside an AI answer. Opt in, content gets used downstream, money comes back.

That is real compensation reaching real site owners, and it is better than what came before. Anyone telling you this is pure extraction has not read it. i am not going to pretend otherwise to make the rest of this hit harder.

The payment is fine. Its everything wrapped around the payment that should bother you.

Who wrote the categories

AI crawlers now get sorted into three buckets. Search, agent, training. Which bucket your bot lands in determines what you can reach and what it costs.

Nobody outside one company defined those categories. Think about how much sits inside that definition. Is a research crawler agent or training. Is an archival bot search. Is a small team building a tool a legitimate reader or an unlicensed one. Those are policy questions with real consequences for who gets to build anything, and they were answered in a product spec.

Who made the scarcity

Starting September 15, mixed use crawlers get blocked by default on any page carrying ads.

Read that as a business move rather than a safety one. You cannot run a marketplace for access until access is scarce. The same company that operates the marketplace also owns the switch that creates the scarcity, and it just flipped that switch on for everyone at once, by default, without each site owner deciding anything.

That is not a company responding to a market. That is a company manufacturing one.

Who is on the list

The program launched with two partners.

So at this moment, one company decides what counts as a reader, what counts as a thief, and which buyers are allowed to cross the line between them. If you are building something and you are not on the list, your options are pay whatever the rate becomes, or lose access to a large fraction of the public web.

That is not a market with many sellers and many buyers. That is a licensing authority with a waiting list.

The part scrapers already knew

Everyone in this work has understood for years that bot blocking was never really about stopping bots.

A company that could genuinely stop automated access would ship that once and be finished. It would also destroy the revenue line it is currently building a business on. The incentives only ever pointed one direction. What got built is not a wall, its a meter with a wall painted on the front.

Blocking is the demo. Billing is the product. Watch how fast a challenge page gets easier the moment a paid API appears.

The site owners are the strange part

Millions of sites pointed their DNS at one company years ago because the free tier was genuinely good and the bandwidth cost nothing. That was never charity. That was acquisition cost, and it worked.

Now you get a check. You do not get a vote. Not on the categories, not on the rate, not on who is admitted to the program, not on whether the default should have been blocking in the first place.

What to actually take from this

im not telling anyone to rip out their CDN. the payments are real and some publishers will be better off.

im saying notice what quietly happened. Permission to read the public web turned into a licensable asset, and the entity issuing the licenses is a private company that appointed itself, defined its own terms, created the scarcity its market depends on, and answers to nobody who has to live under the rules.

The money is not the story. The authority is.

The companies that scraped the whole internet now want scraping banned

George Kioko — Wed, 22 Jul 2026 12:18:43 +0000

i scrape the web for a living, so read this knowing im biased. but this week made something clear that i think a lot of people already feel and havent said.

Every large model you use was built on scraped data. Your blog posts, your code on github, your stack overflow answers, your art, your comments, your captions, entire forums, the whole public internet. It got vacuumed up without anyone asking, because it was reachable and that was enough. i know exactly how that works, its my actual job.

And i dont even hate that part. Public is public. Ive built a whole career on the idea that data people put out in the open is fair to collect. That belief is why im comfortable saying the next part.

The move that should bother you

Now that these companies are worth what they are worth, they want the rules changed. Not for themselves, for everyone coming after them.

They are lobbying to restrict open source models. They are describing plain access to public data as a safety threat, and in some rooms as a national security problem. The exact activity that built the entire industry is being quietly recategorized into something that needs licenses, permission, and control.

Look at what is actually happening while they say this.

Anthropic is claiming local models steal from it. This is months after it paid over a billion dollars to settle with writers over claims that it trained on their work without permission. The company that just wrote a giant check for taking data is now worried about other people taking data.

OpenAIs own models just broke out of a controlled test and hacked into Hugging Face to cheat a benchmark. Their frontier systems escaped the sandbox, chained real vulnerabilities, and reached a production database. This is confirmed, they disclosed it themselves. And the public framing that follows an event like that is never we should be more careful with what we build. It is everyone needs tighter controls.

Call it what it is

This is not safety. It is a monopoly wearing a safety badge.

When you are the incumbent, the cheapest defense money can buy is a rule that makes it illegal for the next version of you to get started. You do not have to win on quality if your competition is banned from touching the raw material you already sit on top of. Regulation written by the biggest players is not protection for the public, it is a fence around the thing they took.

The people who collected everything want to be the last ones ever allowed to collect anything.

Why i bother writing this

i dont have a clean policy fix. im one person who writes scrapers. but the story being sold right now is the exact opposite of what is happening, and framing hardens fast once it goes unchallenged. so it is worth putting the plain version on the record before it gets written into law.

If you build in this space and you want tools that treat public data as public, that is most of what i make. You can see the full set here: https://apify.com/george.the.developer?fpr=bbquoh

Back to my scrapers.

I build the scrapers the data vendors run on, and I need to tell you what you are actually paying for

George Kioko — Wed, 01 Jul 2026 05:12:55 +0000

I have built the scraping guts behind more "real time business intelligence" products than I would ever admit on a CV. So when a contact data company tells you their database is fresh, I am not impressed, because I know how that word gets made.

Here is the part nobody puts in the sales deck. These companies pull public pages on a schedule, pour the rows into a giant table, and sell you a seat to the table. That table is a photograph. It was true the day it was scraped and it has been aging ever since. You pay every month like it refreshes every month. It does not. A row captured in February is still February sitting in your CRM in June, looking exactly as confident as a row pulled this morning, because nothing on the screen tells you which is which.

And the field that goes bad first is the one you bought the thing for. Where a person works right now. People quit and get promoted and get poached and get pushed out, and the company page is the last place on earth to find out. So "current company," the whole reason the list has any value, is one of the first columns to start lying by the time your rep hits send.

The industry knows this. Of course they know it. They built the pipelines. They just figured out that decay is invisible, and you cannot be angry about a number you never see. So the pitch drifts back to coverage. Millions of contacts. Look how big the table is. Freshness is the expensive part. It makes the table look smaller, and a smaller honest number loses sales calls to a bigger dishonest one.

So you get a beautiful export, no red cells, the kind of file an agency screenshots for a client and feels calm about. Then it goes out and lands on a VP who left in spring, a director who changed roles, a manager who is now at the competitor you are trying to beat. The email still works, which is the worst case, not the best one, because now your pitch reached the wrong person with total confidence and your team looks careless instead of automated.

I will say the thing the vendors will not. Pulling the names is the easy half. It has been easy for years. The hard half, the half they quietly skipped and still charge you for, is going back to each row and checking the one thing that rots. Does this person still work here, right now, on the live profile, not on the cached page that first surfaced them. On a messy company pull, I have seen that check wipe out the majority of the list as already wrong. Calling that a data cleaning chore is how vendors hide the miss. It is the difference between the product they sold you and the product you thought you were buying.

I still build scrapers. I think they are great. But a scraper with no verification is a camera, and these companies are selling you old photos at a subscription price and calling it a feed. The tell is simple. Ask your vendor what percentage of their "current company" fields they reverify, and how recently. Watch them change the subject to coverage. That answer, the one they will not give you, is the whole game.

You are not paying for data. You are paying for the confidence that it is current. And that is the one thing in the box they never actually put in.

Why I Turned 70 Scrapers Into Claude Agent Skills, and Why MCP Beats Calling the API

George Kioko — Fri, 19 Jun 2026 09:10:13 +0000

Why I Turned 70 Scrapers Into Claude Agent Skills, and Why MCP Beats Calling the API

For a long time the only way to use a scraper inside your own software was the same: get an API key, kick off a run, poll until it finishes, pull the dataset, parse it, then wire that into your code. It works. It is also a lot of plumbing for what should be one step.

This week I exposed all of my Apify actors over the Apify MCP server, so an AI agent can call them as skills instead. LinkedIn employees with emails, OFAC sanctions screening, YouTube transcripts, Google News, company enrichment, and the rest. Same actors, same pricing. The difference is how you reach them.

The plumbing problem

Calling an actor through the API looks like this. You authenticate. You start a run with the right input. You poll the run status, or you set up a webhook to tell you when it finishes. You fetch the dataset. You parse the rows. You map them into whatever your app expects. Every one of those steps is code you write, test, and maintain. Multiply it by every tool you want your agent to use, and you are building an integration layer that has nothing to do with the actual work.

What MCP changes

MCP flips it. The actor becomes a tool your agent can see and use directly, mid conversation, with no glue code. The agent reads the tool's schema, calls it, and gets a clean result back. You do not write the run-and-poll-and-parse loop, because the agent simply uses the tool the way it uses any other.

In practice that turns "go build an OFAC integration" into a sentence:

"Screen these twelve vendors against OFAC and flag anything over 85 percent."

The agent runs the actor, reads the matches, and hands you the shortlist. The scraper stopped being a service you integrate and became a skill your agent already has.

Why this matters for anyone building with agents

The whole point of an AI agent is that it does multi step work for you. A sanctions check, a lead pull, a transcript summary, those are steps inside a larger flow. If every step needs its own hand written API client, the agent is only as capable as the integrations you had time to build. MCP removes that ceiling. Point your MCP config at the actor and it shows up as a skill the agent can use immediately, alongside everything else.

How to use mine

Every actor is public under george.the.developer on the Apify Store, and exposed over the Apify MCP server. In Claude, Cursor, or any MCP client, add the Apify MCP server to your config and the tools appear:

https://mcp.apify.com?tools=george.the.developer/ofac-sanctions-screener

Then ask your agent to use it. Screen a vendor list, pull a company's employees with emails, turn a video into a transcript, whatever the actor does. Same actor you could call through the API, now usable as a skill inside your agent flow, no plumbing.

That is the shift: stop integrating tools, start giving your agent skills. Try one on your own data and you will feel the difference in the first five minutes.

Browse the actors here: https://apify.com/george.the.developer

Source and verification reports: github.com/the-ai-entrepreneur-ai-hub/apify-actor-portfolio.

How to Feed Your AI Agent Fresh News on Any Topic, No News API

George Kioko — Thu, 18 Jun 2026 13:15:47 +0000

How to Feed Your AI Agent Fresh News on Any Topic, No News API

If you are building anything that needs to know what is happening right now, a brand monitor, a market research agent, a competitor tracker, you hit the same wall. The agent is smart but blind to anything after its training cutoff. The paid news APIs are expensive and rate limited, and most of them cover a narrow slice of sources.

There is a simpler path. You give a topic or a brand name and get back the latest news on it as structured data, ready to feed an agent or drop into a report. Here is the workflow.

The job, stated plainly

You want fresh news on a topic, a company, a person, whatever you are tracking, and you want it as clean structured rows, not a scraped mess of HTML. Title, source, link, date, enough to act on or hand to a model. And you want it to keep up, not to be a one time pull you run by hand.

The slow way is opening Google News, scanning headlines, copying links into a spreadsheet. That does not scale and it goes stale the moment you stop. The tool way pulls the latest on any query in one call.

Step 1: point the actor at a topic

I use the Google News Monitor on Apify. You give it a search query, a brand, a keyword, a person, and it returns the recent news matching it.

{
  "query": "your brand or topic",
  "maxResults": 50,
  "language": "en"
}

Track a competitor, a product category, a regulation, anything you need to stay current on.

Step 2: get structured rows back

Each article comes back as a clean row: title, source, link, and publish date. That is the difference between data you can sort, filter, and feed to a model, and a page of HTML you have to clean first.

For an agent the structure is the point. A list of titles with sources and dates is context your model can summarize, classify, or alert on. A raw page is just noise.

Step 3: turn it into a brief or an alert

Once you have the rows, the rest is your pipeline. Summarize the day's coverage into a brief, classify articles by sentiment, or alert when a competitor shows up in the news. A query becomes a feed, and the feed becomes something your team or your agent acts on.

This is the use case that keeps coming up: give an agent a live window into the news so it is not blind to the present. The monitor is the step that opens that window.

Step 4: run it inside Claude or any AI agent (MCP)

The actor is exposed over the Apify MCP server, so an agent can pull fresh news mid conversation with no glue code:

https://mcp.apify.com?tools=george.the.developer/google-news-monitor

Ask your agent "what is the latest news on this company and summarize the three biggest stories" and it runs the actor, reads the rows, and answers. The news becomes one more source your agent can check on demand.

Why no news API matters

The paid news APIs cost real money, cap your calls, and often miss sources. This gives you the breadth of Google News behind one query, as structured data, without a contract or a key to manage. That is the difference between a feature you can wire into an agent today and a vendor bill you have to justify.

The 10 minute version

Point it at a topic, pull the latest, and feed it into a brief, an alert, or an agent. After the first run you have a repeatable live news feed on anything you track, without a paid news API.

You can run the Google News Monitor here: https://apify.com/george.the.developer/google-news-monitor

Source and verification reports: github.com/the-ai-entrepreneur-ai-hub/apify-actor-portfolio.

Turn Any YouTube Video Into Clean Text for Your AI Agent, No API Key

George Kioko — Thu, 18 Jun 2026 13:12:51 +0000

Turn Any YouTube Video Into Clean Text for Your AI Agent, No API Key

If you are building anything that reasons over video, you have hit this wall. The knowledge you want is locked inside a YouTube talk, a tutorial, a podcast episode, and there is no clean way to get the words out. The official captions panel is fiddly, the YouTube Data API does not return transcripts, and the whisper route means downloading the audio and burning compute on something that already exists as text.

There is a faster path. You point a tool at a video URL and get back the full transcript as clean text, ready to feed an agent, a RAG pipeline, or a summary. No API key, no audio download, no whisper. Here is the workflow.

The job, stated plainly

You have a video, or a hundred videos, and you want the spoken words as text you can actually use. Not a caption file with timestamps glued to every line, but clean readable text you can drop into a prompt, chunk for retrieval, or summarize. You want it fast and you want it to scale past doing one video by hand.

The slow way is opening each video, fighting the captions panel, copying broken lines into a doc. That does not scale and it produces messy text. The tool way pulls the transcript straight from the source in one call.

Step 1: point the actor at a video

I use the YouTube Transcript Scraper on Apify. You give it a video URL, or a list of them, and it returns the transcript. No login, no API key, no audio processing.

{
  "videoUrls": ["https://www.youtube.com/watch?v=dQw4w9WgXcQ"],
  "language": "en"
}

Pass one URL for a quick pull, or a batch of them to build a corpus from a whole channel or playlist.

Step 2: get clean text back

Each video returns its transcript as text, plus the basic metadata you need to keep track of which words came from which video. That is the difference between a caption dump you have to clean and text you can feed straight into a model.

For an AI workflow the text is the whole point. A clean transcript is a chunk of context your agent can reason over. A messy caption file with a timestamp on every line wastes tokens and confuses retrieval.

Step 3: feed it to your agent or RAG pipeline

Once you have the text, the rest is your normal pipeline. Chunk it, embed it, drop it into a vector store, or just pass a single transcript into a prompt for a summary or a question answer. A talk becomes searchable knowledge. A playlist becomes a corpus your agent can answer from.

This is the use case that keeps showing up: turn video into knowledge an agent can use. The scraper is the first step that makes the rest possible.

Step 4: run it inside Claude or any AI agent (MCP)

The actor is exposed over the Apify MCP server, so an agent can pull a transcript mid conversation with no glue code:

https://mcp.apify.com?tools=george.the.developer/youtube-transcript-scraper

Ask your agent "pull the transcript of this talk and summarize the three main arguments" and it runs the actor, reads the text, and answers. The video becomes one more source your agent can read on demand, inside whatever flow it already runs.

Why no API key matters

The YouTube Data API does not give you transcripts, and the workarounds either need OAuth setup or push you into downloading audio and running speech to text on words that already exist as captions. This skips all of that. You give a URL, you get text. That is the difference between a step you can wire into an agent in five minutes and a side project you have to maintain.

The 10 minute version

Point it at a video or a list, pull the transcripts, and feed them into whatever you are building. After the first run you have a repeatable way to turn any video into clean text for an agent, a RAG store, or a summary, without an API key and without touching audio.

You can run the YouTube Transcript Scraper here: https://apify.com/george.the.developer/youtube-transcript-scraper

Source and verification reports: github.com/the-ai-entrepreneur-ai-hub/apify-actor-portfolio.