DEV Community: Samir Yuja

I Gave Claude the Keys to My Airbnb for Two Weeks. Here's What It Found.

Samir Yuja — Wed, 10 Jun 2026 03:28:18 +0000

Originally published at samiryuja.dev

I connected the official Hospitable MCP server to Claude, pointed it at my Airbnb, and asked it to take a look. Here's what I found.

A language model is only as good as the context it's given. On its own it can talk about short-term rentals in the abstract; it knows nothing about mine.

The MCP closes that gap — it gives the model live, authenticated access to my actual reservations, messages, reviews, and calendar. So instead of guessing, it can look at the real thing. I run a two-room rental I've managed for over a year, and I figured I had it under control. Two weeks of letting an agent actually look turned up a surprising amount that was quietly broken.

Setup

Hospitable Professional plan (the tier that includes MCP access)
Claude Pro
Connect them: Settings → Connectors → add custom connector → OAuth
~50 tools load: reservations, messages, calendar, reviews, the Knowledge Hub, tasks, smart locks
Permissions were monitored the entire time — no messages were ever sent to guests. Everything guest-facing stayed read-only or draft-and-paste: the agent proposes, I decide.

What an AI audit of a real business actually surfaces

Here are the highlights of what it found, in four buckets.

1. Liability

The registered guest booked from overseas, but the person staying was his wife, who had no verified profile of her own. The agent caught it by reading the message thread. That matters because AirCover and host protections are written around the registered guest, and nothing in the structured reservation data reveals a substitution. It only surfaces if someone actually reads what the guest wrote.

2. Revenue opportunities

This bucket changed how I look at my inquiry inbox. Every inquiry is revenue waiting on a response, and the agent treated it that way: it read the full inquiry history and identified exactly where a faster or better-matched reply would have won the booking. One guest asked about renting the whole place; by the time a reply went out hours later, those dates had gone to another property. Another inquiry came in written in Spanish, and the automated rule answered in English because no multilingual template existed. Across the history, it flagged six inquiries where response time or language likely cost the conversion.

None of this is visible on any dashboard. What the agent produced was effectively a conversion punch list: respond faster to whole-home inquiries, add a Spanish template, and answer the recurring questions (parking, check-in) before guests have to ask. Questions are demand — the goal is turning them into bookings.

3. Things quietly broken

The theme in this bucket: the business changed, and the documentation didn't keep up.

The big one was stale door codes. When I first set up the Airbnb, I hardcoded the entry codes into the Knowledge Hub — the reference Hospitable's AI uses to answer guests. Later I synced my Schlage lock with Hospitable, which generates a fresh code for every reservation. The original setup-era codes stayed behind in seven different Knowledge Hub entries, wrong for every guest since. If anyone had asked the AI "what's my door code?", it would have confidently handed them a code that opens nothing.

A typo had been sitting in an active message rule for months: a small formatting error in the review request that fires after every checkout, the final touchpoint of every stay. Easy to fix, impossible to spot without going line by line through every rule, which no one does.

The agent also caught a time-sensitive review window. A recent guest had smoked inside and brought unregistered visitors, and my window to review them was about to close. Once that window shuts, future hosts get no warning, so that was the catch that mattered. It also surfaced a backlog of guest reviews I'd never responded to — lower stakes, but review responses are public and responsiveness factors into Airbnb's ranking, so the backlog went on the list rather than in the trash.

Parking was in this bucket too. It's the single most common question guests ask, and the Knowledge Hub had no car parking entry at all. The Parking section existed, but it only covered bicycles.

4. Features I wasn't using but should be

Some of what it surfaced wasn't broken, just sitting there unused. The Knowledge Hub was thin and partly wrong; now it's corrected and actually filled in. Tasks was completely empty, so I registered myself as a teammate and set up text alerts for upcoming cleanings. Right now that's redundant since it's just me. But the structure exists now, and the day I bring on a housekeeper, I swap in their number and nothing else changes.

How I actually use it

The dramatic catches are one thing; the reason it's still connected is the boring daily stuff.

The first is complaint clustering, because it goes straight at my biggest pain point: a new guest arrives and starts complaining, and in the moment it's hard to tell a one-off from a pattern. I had the agent group every review across Airbnb and Booking.com by theme. The runaway top three:

Bedroom doors didn't lock from the inside — four guests, both rooms. The clear #1. Guests wanted keyed locks, not the privacy locks the rooms came with. The main entrance already had a Schlage deadbolt; the fix was recently replacing the bedroom privacy locks with keyed door knobs — and the clustering confirmed the complaints dropped off after the change.
Location and transit — guests underestimating how far things were.
Surprise at the shared-home setup — a listing-clarity issue, not a property one.

The clustering also corrected an assumption I'd been carrying: I was sure Booking.com guests complained more than Airbnb guests. Totally wrong. The data showed the gap was a scoring-norm artifact, not a real satisfaction difference.

The second is the weekly turnover outlook. One question and I get a clean in-chat calendar of which days have a checkout and a check-in stacked up — the days I can't schedule anything else. Combined with the task alerts, I stopped double-booking myself — committing to a day in the office only to find out it was also a turnover day.

False positives and rough edges

If I only told you the good parts you shouldn't believe me. The honest column:

It flagged a problem that wasn't one. Reviewing my message templates, the agent called out messages where the %smartlock_code% shortcode hadn't resolved. The real explanation: those guests had already checked out, and their codes had expired with the reservation. The lesson is that the agent doesn't automatically know the lifecycle of the data it's reading — what looks broken on a past reservation can be perfectly normal.

Writes can fail silently. Adding an item to a Knowledge Hub that had never been initialized succeeded with no error at all — but nothing actually appeared until I generated the hub in the dashboard. A success response that doesn't result in the thing existing is the kind of surprise that bites you later.

What's next

The obvious next step is automating the loop I ran by hand these two weeks — issue comes in, gets detected, a fix gets drafted for me to approve. I'm still working out exactly what that should look like, so I'll leave it there for now.

The part that stuck with me

What surprised me wasn't any one finding — it was the speed of the audit. Catching a months-old typo without reading every message rule by hand. Clustering a year of reviews into three themes in seconds. Fewer complaints, less of my time spent firefighting, more of it back. The most useful thing an agent did here wasn't anything clever — it was simply looking, systematically, at data I already had, and sorting it into the four things that actually matter: liabilities, revenue opportunities, the things that had quietly snuck past me, and the features I should have been using all along.

Built with the official Hospitable MCP server connected to Claude. Real two-room property, real findings. Guest details kept deliberately vague.

Futbol Report — building a multi-model LLM comparison on AWS Lambda

Samir Yuja — Mon, 01 Jun 2026 13:17:21 +0000

A few months ago I set up a soccer-digest bot that sends me a Telegram message every few days with fixtures, results, transfer news, and manager changes. It started as a tmux session running Claude Code on a small Linux server, fired by a cron job. It worked. It also went down occasionally, and I had no good way to inspect what it was producing.

I wanted to do two things at once: make it more reliable, and turn it into something more interesting than "one bot sending one report." The result is Futbol Report — a scheduled job on AWS that sends the same prompt and search context to four different language models (Claude, Kimi, Qwen, Gemma) every three days, stores all four reports in Redis, and renders them side-by-side on this site with live voting.

This post is about how it works, what I learned from running it, and the deployment war stories — the bits that are usually edited out of "here's my side project" writeups.

What the comparison shows

Same input, four models, no editing. The page has a dropdown to switch between past runs and a vote button under each report. Vote tallies persist.

The point isn't to crown a "best" model. It's to make differences visible on a real, repeated task. Placed next to each other, the models diverge in ways that are easy to see — how faithfully they follow the requested format, what they choose to include or filter out, and how long their reports run.

How the pipeline works

EventBridge Scheduler  (every 3 days)
        ↓
   AWS Lambda  (Python 3.12)
        ├── Brave Search    (~13 queries: fixtures, results, transfers, manager changes)
        └── OpenRouter      (same context → 4 models)
        ↓
   Redis  (Vercel)
        ↓
   Next.js page  (server-rendered comparison + voting)

Every three days EventBridge fires a Lambda function. The Lambda calls Brave Search with about a dozen queries, then sends the same compiled context to four models through OpenRouter. Each model's report goes into Redis under a timestamped key. The Next.js site, deployed on Vercel, reads from the same Redis and renders the comparison page.

A few decisions worth surfacing:

OpenRouter as the inference layer. One API instead of four, and adding or swapping a model is a one-line change.

Server-rendered comparison page. The data only changes every few days, so there's no point fetching it from the browser. The server reads Redis and sends back the finished page. Only the vote button runs in the browser.

Redis with a 90-day TTL on report keys. Redis fit the access pattern — small payloads (a few KB per report) and pure key lookups by timestamp, no queries. The TTL means old reports expire automatically; votes and the run index have no TTL, so voting history is never evicted even if memory fills.

What I learned from running it

1. Search results aren't deterministic. Running the same query thirty minutes apart returns different result sets — that's just how live ranking works. So context has to be held fixed within a run for the comparison to be fair (one Brave call feeds all four models).

2. A simple anti-hallucination clause worked across all four models. After the first run hallucinated fixtures, I added "use ONLY facts present in the provided search results" to the prompt. None of the four models invented data after that — same clause, same effect, across four different labs.

3. Models filter context differently. One run, Brave's results included an out-of-scope Indian Super League match. Three models filtered it out; one led its report with it. Same prompt, same data, different prioritization.

4. Output length varies wildly with model size. Claude and Kimi used most of the available context. Gemma — by far the cheapest model — collapsed the same input into one-line summaries. Cost and level of detail are correlated.

5. Format adherence varies too. Claude followed the prompt's structure most faithfully. Gemma dropped most of it. Qwen and Kimi were in between.

6. The pipeline survived the off-season without code changes. When Serie A ended, fixture queries returned nothing useful. I added two new search categories (transfers, manager changes) and one line to the prompt — "if fixtures are sparse, lead with transfer news." Reports stayed substantive: Allegri sacking at Milan, Guardiola leaving City, World Cup buildup.

The deployment war stories

The interesting part of moving the generator to Lambda was the several hours of debugging in the middle.

Python runtime mismatch. AWS defaulted the new Lambda to Python 3.14. My deployment package was built for 3.12. The error didn't say "wrong Python version" — it said No module named 'pydantic_core._pydantic_core', because the compiled C extension is a cpython-312 .so that won't load under 3.14. Fix: match the runtime to the build target.

Mac vs Linux binaries. Even after pinning the Python version, pydantic-core kept loading the macOS binary into my zip. I was using uv for packaging — uv pip install --only-binary is supposed to fetch a Linux wheel but reliably didn't here. Switching to vanilla python3 -m pip install --platform manylinux2014_x86_64 --only-binary=:all: finally pulled the right artifact. The newer tool I trusted was the problem; the older boring tool worked.

Optional imports for dual environments. python-dotenv is great locally — reads .env, populates os.environ. On Lambda, environment variables come from AWS directly, and python-dotenv is just dead weight that doesn't ship in the runtime. Wrap the import:

try:
    from dotenv import load_dotenv
    load_dotenv()
except ImportError:
    pass  # dotenv not needed on Lambda — env vars are set directly

Same code works in both environments.

Default timeout. Lambda's default timeout is 3 seconds. My pipeline needs about 4 minutes — 13 sequential Brave calls plus four sequential model generations. Bump it to 900 seconds (Lambda's max).

Reaching for the CLI over the browser. I couldn't get the AWS console's "Upload .zip file" button to reliably refresh the deployed code — the SHA-256 hash kept matching the previous upload even after I selected a new file. It probably works fine most of the time; in my case aws lambda update-function-code from the CLI was faster and easier, and it's the path I use now.

Secrets are easy to leak when copy-pasting console output. I rotated three API keys during this project — twice because I pasted command output that included environment variables. Painful in proportion to how preventable it is. Wiring gitleaks into a GitHub Actions workflow is next on the list.

What's next

In priority order:

Telegram delivery from the Lambda — restore the original use case so the digest pings me when a new run finishes.
CI security scanning — gitleaks for secrets and osv-scanner for dependency CVEs, both wired into a GitHub Actions workflow on each repo.
Parallelize the model calls — sequential right now (~4 minutes); concurrent would cut runtime by ~75%.
Full page content for grounding — Brave returns 1-2 sentence snippets, so thin context yields thin reports. Firecrawl or a readability extractor would fix this.
Server-side vote dedup — votes are deduped per-browser via localStorage; clearing storage or going incognito gets around it.

Thanks

To Ryan — for letting me run the original bot on his machine over Tailscale, and for the steady stream of articles and ideas that shaped a lot of the thinking behind this project.

DEV Community: Samir Yuja

I Gave Claude the Keys to My Airbnb for Two Weeks. Here's What It Found.

Setup

What an AI audit of a real business actually surfaces

1. Liability

2. Revenue opportunities

3. Things quietly broken

4. Features I wasn't using but should be

How I actually use it

False positives and rough edges

What's next

The part that stuck with me

Futbol Report — building a multi-model LLM comparison on AWS Lambda

What the comparison shows

How the pipeline works

What I learned from running it

The deployment war stories

What's next

Links

Thanks