Scraping Social Media with Gemini for sentiment analysis

Jerrod Kim — Mon, 16 Mar 2026 18:09:35 +0000

Scraping Social Media with Gemini for Sentiment Analysis

Hackathon project: using Gemini + a user's browser to analyze real sentiment hidden inside social media comment sections.

Inspiration

If you want to understand how people actually feel about a product, brand, or topic, the best data source is often comment sections.

The problem: scraping social platforms is getting harder every year.

Platforms now deploy:

expensive APIs and paywalls (e.g. Reddit's API changes)
aggressive bot detection
rate limits and scraping prevention

Running scrapers from servers or headless browsers usually gets blocked almost immediately.

So we asked:

What if the scraper wasn't a bot at all — but a real user's browser?

Modern browser automation combined with Gemini's computer-use capabilities makes that possible.

A real browser session comes with some powerful advantages:

✅ already authenticated to social platforms
✅ trusted by anti-bot systems
✅ capable of normal browsing behavior

In other words: the ultimate scraping environment already exists — the user's browser.

What It Does

Textpot leverages a user's browser to explore social media and collect comments for sentiment analysis.

The architecture separates browser control from AI decision making.

The system runs a loop (up to 3 turns) to navigate a page and analyze comments:

Extension captures screenshot
Screenshot POSTed to Cloud Run
Gemini analyzes the screen and returns the next action
Extension executes the action via CDP

The result is a feedback loop where:

the browser acts
Gemini decides what to do next

This allows Textpot to automatically explore comment sections and extract sentiment insights.

Architecture

The system is split into two parts.

1. Chrome Extension (User Machine)

The extension owns the browser.

Responsibilities:

opens the extension popup
attaches to the page via Chrome DevTools Protocol (CDP)
performs actions (click, scroll, keypress)
captures screenshots of the page

Everything runs directly inside the user's local Chrome session.

2. Cloud Run Backend

The backend owns the AI logic.

Responsibilities:

receives screenshots from the extension
sends them to Gemini
stores conversation history across turns
returns the next action to perform

Importantly:

The backend never directly touches the browser.

It only tells the extension what action to perform next.

Why This Architecture Works

Splitting responsibilities between browser and AI backend solves a major scraping problem.

The browser:

has real authentication
has real cookies
behaves like a normal user

Cloud Run simply tells it:

"Click here."

"Scroll down."

"Open this comment thread."

This approach bypasses many of the traditional scraping roadblocks.

Challenges We Ran Into

The first version of Textpot looked very different.

Initially we built it as a web app running a headless browser on Cloud Run.

That approach quickly failed.

Problems included:

bot detection blocking the browser
authentication failures
restricted access to social media pages

The fix was simple but important:

Move the browser to the user.

Once we pivoted to a Chrome extension, the system could use:

real user sessions
real cookies
normal browsing behavior

That solved most of the blocking issues immediately.

What's Next for Textpot

Next steps include:

polishing the extension UX
improving comment extraction
adding deeper sentiment analysis
launching on the Chrome Web Store

Because part of the system runs on Google Cloud Run, we'll also need to figure out a sustainable pricing model.

Final Thoughts

AI-powered browser automation opens up a new way to interact with the web.

Instead of fighting platform restrictions with bigger scrapers, we can:

use real browsers
keep AI in the backend
let models like Gemini decide how to navigate

For sentiment analysis and market research, this could unlock data sources that are otherwise extremely difficult to access.

If you're experimenting with Gemini computer use, browser automation, or AI agents, I'd love to hear how you're approaching it.

My First GKE Experience - GKE Turns 10 Hackathon.

Jerrod Kim — Sun, 21 Sep 2025 00:49:31 +0000

This post is about creating a project for the GKE Turns 10 Hackathon - https://gketurns10.devpost.com/

For the GKE Turns 10 Hackathon, I decided to extend Bank of Anthos with a retirement planning dashboard. (Bank of Anthos is a sandbox project you can run on GKE.)The idea was to give users a way to check their savings goals, get AI-powered advice from Google Gemini, and even look up side jobs through the Adzuna API. And I also did execute!

As a starter..

If you're new to GKE in general, this is a rough explanation of how the GKE deployment process works: You create a docker image of your code/microservice -> Then you push the docker image to Google Cloud's Artifact Registry -> Then you run the "kubectl apply" terminal command to deploy to GKE.

Some stuff I went through:

The app itself wasn’t the hard part—the real challenge was getting it running smoothly on Google Kubernetes Engine (GKE). Along the way, I hit several bumps:
Secrets not set up → My pods wouldn’t start because I forgot to apply the JWT secret. This was in the readme.md in the root directory. I'm the one who missed it.
Cluster too small → Some services stayed in Pending until I scaled the cluster up. Wasn't expecting services to stop working when I lowered the cpu requirements in the yaml files because GKE costs were higher than I had anticipated..!
Docker image mismatch → I built locally on a Mac (ARM) but GKE nodes use AMD64. Quick fix here.
Service exposure confusion → My dashboard worked internally, but I couldn’t access it until I switched the service type from ClusterIP to LoadBalancer. Even as a rookie, I can see this being a rookie mistake.
Each issue was frustrating in the moment, but they helped me get closer to understanding GKE.. hopefully it did:)

So in the end..

Anway, I ended up being able to build a dashboard that:

Hooks directly into the Bank of Anthos frontend
Provides personalized AI retirement advice
Surfaces job listings to boost income
Runs consistently on GKE (GKE is expensive for a solo dev I gotta say. Good thing I got $100 credit from the hackathon but it's running out real fast)

Final Thoughts

I could see how GKE and Kubernetes are essential for large, large projects. As a solo dev, I've been mainly using things like Firebase and maybe Cloudflare. I do feel that I need to up my ante to be on the next level as a dev and a founder. I'm still too uninitiated. Still taking small baby steps:)

This post was created for the purposes of entering the GKE Turns 10 Hackathon.

DEV Community: Jerrod Kim

Scraping Social Media with Gemini for sentiment analysis

Scraping Social Media with Gemini for Sentiment Analysis

Inspiration

What It Does

Architecture

1. Chrome Extension (User Machine)

2. Cloud Run Backend

Why This Architecture Works

Challenges We Ran Into

What's Next for Textpot

Final Thoughts

My First GKE Experience - GKE Turns 10 Hackathon.

As a starter..

Some stuff I went through:

So in the end..

Final Thoughts