DEV Community: Yuuki Yamashita

The day I gave an AI a wallet — building an approval-gated shopping agent with Sonnet 4.6, AgentCore Payments, Rakuten and Stripe

Yuuki Yamashita — Tue, 23 Jun 2026 13:48:22 +0000

TL;DR
I built a PoC that wires up Bedrock AgentCore Payments (x402 + USDC) and Rakuten + Stripe Checkout behind a single Strands Agent — and made sure the agent can't spend a single cent without a human approval card. Production-deployed on Vercel, with an 8-language UI.

Production: https://wallet-agent.vercel.app/

Repo: https://github.com/yama3133/wallet-agent

Why I built this

I have five different talks / CFPs lined up — re:Invent 2026 COM Track, Qiita Tech Festa, iOSDC LT, re:Deploy Security, and a Slack Hackathon — all centred on the same idea: "the day you give an AI a wallet, and what an approval gate looks like."

Slides alone weren't going to carry the message. I needed a shared implementation base that I could spin up live in front of an audience.

The shape of the goal was simple:

User asks something in natural language → the agent picks an option → an approval card pops up → human approves / rejects → on approval, the actual payment runs → result is summarised back to the user.

I wanted to prove this in two axes: pay-per-call paid APIs (microtransactions) and real-world product purchases.

Architecture

Frontend: Next.js 16 on Vercel (approval list + chat + checkout result)
State: DynamoDB — wallet_agent_tasks / approvals / txns, Streams + PITR
Agent: AgentCore Runtime (ARM64 container) + Strands Agent + Claude Sonnet 4.6
Payments — Phase 1: AgentCore Payments → Privy (StripePrivy) → x402 → base-sepolia USDC
Payments — Phase 2: Rakuten Ichiba IchibaItem/Search → Stripe Checkout (test mode)
Localisation: ja / en / zh / ko / fr / it / es / ar (RTL) — 8 languages, localStorage + navigator.language auto-detect, LINE Seed JP Bold as the base font

All of it — code, CloudFormation, demo script — lives in yama3133/wallet-agent.

The agent itself

The agent is just six @tool functions wired into a Strands Agent.

@tool
def search_paid_resources(query: str = "") -> list[dict]: ...  # x402 catalog

@tool
def request_payment_approval(resource_id: str, amount_usd: str, justification: str) -> dict:
    """Write a pending approval to DynamoDB / local JSON and block until a decision lands."""

@tool
def execute_x402_payment(resource_id: str, payment_session_id: str | None = None) -> dict:
    """Drive AgentCore Payments' generate_payment_header through to a successful x402 settle."""

@tool
def search_rakuten_items(keyword: str, max_jpy: int | None = None, hits: int = 5) -> list[dict]: ...

@tool
def request_purchase_approval(item_id: str, title: str, amount_jpy: int, justification: str) -> dict: ...

@tool
def execute_stripe_checkout(item_id: str, title: str, amount_jpy: int, image_url: str = "") -> dict:
    """Create a Stripe Checkout Session and return the URL."""

The trick is that request_*_approval writes a row to DynamoDB and waits. The tool chain literally can't progress until a human flips the row to APPROVED. That single primitive keeps the LLM from going off the rails.

Phase 1 — the AgentCore Payments signer trap

This is where I lost the most time.

ProcessPayment → AccessDeniedException
"Privy credentials are invalid. Please verify the credential configuration."

I could create PaymentManager, PaymentConnector, and PaymentInstrument (an Embedded Crypto Wallet) from boto3 just fine. ProcessPayment was the one call that wouldn't go through.

Looking at the Privy dashboard, the wallet that AWS's CreatePaymentInstrument had created was owned by an internally-generated Privy User, and my Authorization Key was not registered as a signer on any wallet at all.

The fix is to run Privy's official template, privy-io/aws-agentcore-sdk, locally and click through the "Connect agent" UI in a browser. That UI hits Privy's internal API and adds your Authorization Key to additional_signers on the wallet.

git clone https://github.com/privy-io/aws-agentcore-sdk ~/wallet-agent-privy-template
cd ~/wallet-agent-privy-template
# Drop NEXT_PUBLIC_PRIVY_APP_ID / PRIVY_APP_SECRET / NEXT_PUBLIC_PRIVY_SIGNER_ID into .env.local
pnpm dev
# → browse to localhost:3001 → log in → Connect agent

After that, process_payment returns PROOF_GENERATED and the merchant (https://drvd12nxpcyd5.cloudfront.net/market-recap, a public x402 demo endpoint) accepts the payload.

[bedrock_agentcore.payments.manager] Successfully processed payment for user test-user-yama3133
[bedrock_agentcore.payments.manager] Successfully generated payment header for user test-user-yama3133

The lesson: you cannot finish AgentCore Payments setup purely server-side. Design your demo flow with the Privy frontend baked in from day one.

Phase 2 — Rakuten and the Referer pitfall

Phase 2 was a much more pedestrian WebAPI face-plant.

I registered a new Rakuten webservice application, took the Application ID (UUID form), and hit https://openapi.rakuten.co.jp/ichibams/api/IchibaItem/Search/20260401. Got:

{"errors":{"errorCode":403,"errorMessage":"REQUEST_CONTEXT_BODY_HTTP_REFERRER_MISSING"}}

Adding Referer: https://wallet-agent.vercel.app/ didn't help. The actual culprit was User-Agent bot detection — User-Agent: wallet-agent/0.1 is rejected. Swapping to a browser-ish Mozilla/5.0 ... makes the request go through.

The final tools/rakuten.py looks like this:

headers = {
    "User-Agent": (
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
        "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0 Safari/537.36"
    ),
    "Referer": referer,             # WALLET_AGENT_PUBLIC_URL
    "Origin": referer.rstrip("/"),
}
if access_key:
    headers["accessKey"] = access_key
    params["accessKey"] = access_key

From there: a pair of black socks at ¥1,980, a Stripe Checkout Session, test card 4242 4242 4242 4242, and a redirect to https://wallet-agent.vercel.app/checkout/success?session_id=cs_test_... showing "Paid / 1,980 JPY."

DynamoDB and the Vercel frontend

The approval state lives in wallet_agent_approvals on DynamoDB. Local dev falls back to agent/.approvals.json — flipped by WALLET_AGENT_STORAGE=local|dynamo.

The Next.js 16 App Router side is a small handful of route handlers:

// /api/approvals (GET) — list PENDING
const r = await ddb().send(new ScanCommand({
  TableName: TABLES.approvals,
  FilterExpression: "#s = :p",
  ExpressionAttributeNames: { "#s": "status" },
  ExpressionAttributeValues: { ":p": "PENDING" },
}));

// /api/approvals/[id] (POST) — decide
await ddb().send(new UpdateCommand({
  TableName: TABLES.approvals,
  Key: { approval_id: id },
  UpdateExpression: "SET #s = :d, decision = :d, #r = :r, decided_at = :t",
  ExpressionAttributeNames: { "#s": "status", "#r": "reason" },
  ExpressionAttributeValues: { ":d": body.decision, ":r": body.reason ?? "", ":t": String(Date.now()/1000), ":pending": "PENDING" },
  ConditionExpression: "attribute_exists(approval_id) AND #s = :pending",
}));

I bit the error is a DynamoDB reserved word trap once — you need ExpressionAttributeNames: { "#e": "error" } to update it, otherwise you get ValidationException: Invalid UpdateExpression.

8-language UI and LINE Seed JP

apps/web/src/lib/i18n.ts is just a flat dictionary of 8 languages × 31 keys, wired into a tiny useI18n() Context. The Arabic locale flips document.documentElement.dir to "rtl":

useEffect(() => {
  document.documentElement.lang = locale;
  document.documentElement.dir = getDir(locale); // "ltr" | "rtl"
}, [locale]);

The font is LINE Seed JP Bold via next/font/google, exposed as a CSS variable --font-line-seed and dropped into Tailwind's font-sans. It gives Japanese text a friendly rounded-bold feel that reads well alongside the LINE-app universe.

Things I learned

The Privy signer wall is not solvable server-side. Build the "Connect agent" frontend step into the demo on day one.
agentcore configure is interactive by default. With -ni, a hand-rolled ECR repo, and a Dockerfile in the bundle, you can drive it from CI just fine.
Vercel's 60-second Hobby timeout does not play nicely with synchronously invoking a long-running agent. Plan for waitUntil or a polling pattern.
One "human approval card" tool is enough to make the LLM safe-by-construction. The same primitive solves Phase 1 and Phase 2 without modification.

Porting Apex Voice to Windows broke at every single layer (faster-whisper + pystray + pywin32)

Yuuki Yamashita — Mon, 22 Jun 2026 00:27:06 +0000

I ported my macOS voice-typing app Apex Voice to Windows,
aiming for feature parity. Every layer of the stack tripped me up.
Here's the honest log.

Where I started

Apex Voice is the macOS
menu-bar voice-typing app I built on mlx-whisper + rumps + LaunchAgent.
Apple Silicon-only by design, so it doesn't run on Windows as-is.

The plan was a port for parity. The only test environment I had on
hand was Windows 11 ARM64 (evaluation) on UTM. I did not expect
this to be a months-of-effort kind of project. It wasn't months, but
it was every layer.

Repo: github.com/yama3133/apex-voice
Windows README: README_win.md

The stack I picked

The substitutions I started with:

	macOS	Windows
Speech recognition	mlx-whisper	faster-whisper (CPU/int8)
Tray	rumps	pystray
Text insertion	NSPasteboard + osascript Cmd+V	pyperclip + pynput Ctrl+V
Auto-launch	LaunchAgent	Task Scheduler
Caps Lock one-key	Karabiner-Elements	AutoHotkey v2

In my head this was "swap the libraries." In reality, every swapped
layer broke in its own way.

Pain #1: sounddevice doesn't load on ARM64

First blocker. I wanted to use sounddevice for mic input, but on
ARM64 Windows it fails with libportaudioarm64.dll: error 0x7e.
PortAudio doesn't ship an ARM64 binary.

→ Fall back to pyaudio. pyaudio has an ARM64 wheel and just works.
Rewrote the Recorder class around it.

class Recorder:
    """sounddevice doesn't work on ARM64 Windows; use pyaudio."""

    def _start_pyaudio(self):
        import pyaudio
        self._pa = pyaudio.PyAudio()
        self._stream = self._pa.open(
            rate=SAMPLE_RATE, channels=1, format=pyaudio.paFloat32,
            input=True, frames_per_buffer=BLOCK
        )

Pain #2: zero output on startup

Ran python voicetype_win.py and the console showed nothing. The
process was running but looked frozen.

My first guess: pystray is blocking the main thread. Added print
statements everywhere. Still nothing.

The actual cause: the Windows console was choking on bytes written
to stderr. pyaudio and pystray write something to stderr at startup
that the console's code page can't render, and the output buffer
stalls.

Workaround:

python -u voicetype_win.py 2>err.txt

-u unbuffers stdout, 2>err.txt redirects stderr to a file. After
that I could finally see my STEP1 STEP2 ... diagnostic prints.

I spent over an hour debugging "why aren't my flush=True prints
appearing."

Pain #3: pynput can't catch F19

On macOS I use Karabiner to remap Caps Lock → F19, and pynput
listens for F19. I tried to replicate this with PowerToys on Windows.

The PowerToys remap itself worked (the Caps Lock LED no longer
toggled, meaning the original Caps Lock behavior was suppressed).
But pynput didn't catch F19. The log said hotkey registered: <f19>,
but pressing the key did nothing.

I later found that pynput on Windows has known issues with F13 and
above. Switched the hotkey to <ctrl>+<alt>+r and it worked
immediately.

Pain #4: PowerToys ate Ctrl+V

Mid-debug, PowerToys Keyboard Manager somehow hijacked Ctrl+V
itself. Even after stopping Apex Voice, Ctrl+V no longer pasted.
PowerToys' keyboard hook had stacked some leftover state.

Eventually I lost all keyboard input to Notepad and had to
reboot the VM. After that I dropped PowerToys (it felt brittle) and
switched to AutoHotkey v2 for the remap.

#Requires AutoHotkey v2.0
SetCapsLockState("AlwaysOff")
CapsLock::Send("^!r")

SetCapsLockState("AlwaysOff") suppresses the Caps Lock toggle
behavior (the uppercase mode).

Pain #5: Tray clicks steal focus

Click tray → stop recording → Whisper transcribes → paste into
Notepad. The "inserted" log appeared, but nothing showed up in
Notepad. The moment I clicked the tray, focus had moved to the
Command Prompt, and Ctrl+V was pasting there.

Fix: use pywin32 to save the foreground window handle when recording
starts, and restore it right before the paste:

def _toggle(self, *_):
    if not self.recording:
        try:
            import win32gui
            self.inserter._prev_hwnd = win32gui.GetForegroundWindow()
        except Exception:
            pass
        ...

def insert(self, text: str):
    import win32gui
    prev_hwnd = getattr(self, '_prev_hwnd', None)
    if prev_hwnd:
        try:
            win32gui.SetForegroundWindow(prev_hwnd)
            time.sleep(0.15)
        except Exception:
            pass
    # then pyperclip + Ctrl+V

On macOS, osascript handled active-app restoration implicitly, so
this code didn't exist. OS-layer differences exposed.

Pain #6: Bedrock — MissingDependencyException

aws login works, aws sts get-caller-identity returns my user, but
calling Bedrock via boto3 from Python:

botocore.exceptions.MissingDependencyException: Missing Dependency:
Using the login credentials provider requires an additional dependency.
You will need to pip install "botocore[crt]"

Credentials produced by aws login (AWS CLI 2.32.0+) need botocore's
optional crt extra:

pip install "botocore[crt]"

After that, polish/formal/translate/bullets all worked.

Pain #7: Caps Lock doesn't reach the VM in UTM

Running the AutoHotkey script as admin, pressing Caps Lock did
nothing. AHK was never triggered.

Cut to the chase: the Mac → UTM → Windows key path drops Caps
Lock somewhere. Either UTM's keyboard pass-through doesn't relay
it, or macOS absorbs it at the OS level before UTM sees it. Hard to
pin down.

This wouldn't happen on bare-metal Windows. I gave up on Caps Lock
1-key and shipped with <ctrl>+<alt>+r pressed directly as the
hotkey. Control+Option+R on the Mac keyboard arrives in Windows as
Ctrl+Alt+R through UTM, so functionally it works.

Auto-launch on login

Task Scheduler with /sc onlogon /rl highest:

schtasks /create /tn "ApexVoice" /tr "%USERPROFILE%\apex_voice_start.bat" /sc onlogon /rl highest /f
schtasks /create /tn "ApexCapsAHK" /tr "%USERPROFILE%\apex_caps.ahk" /sc onlogon /rl highest /f

Important: if Apex Voice runs as administrator, AutoHotkey must also
run as administrator (Windows blocks key injection from a lower-
privilege process to a higher-privilege one).

End state and performance

Measured on UTM (no bare metal at hand):

Metric	UTM (ARM64)	Estimated (bare-metal Windows)
4s audio recognition	~37s (CPU/int8, large-v3-turbo)	a few seconds
Insertion latency	~250ms	similar
Bedrock post-processing	works	works
Caps Lock 1-key	impossible (UTM limitation)	should work

UTM's virtual CPU is too slow for Whisper at this model size. On
real Windows hardware large-v3-turbo should be practical. Need
bare-metal testers for that.

What I learned

"Just swap the libraries" is a lie. OS permission models, key codes, console code pages, focus management — even Python apps hit all of these at the lower layers.
When console output disappears, suspect stderr. Splitting it off with 2>err.txt made everything visible.
Key remappers that grab modifiers are dangerous. PowerToys Keyboard Manager left leftover state that killed Ctrl+V. AHK v2 has been more stable.
The real cost of cross-platform isn't code volume, it's the number of OS-specific traps you have to step into.

What's next

Verification on bare-metal Windows. UTM is enough for validation but I want to confirm performance and the Caps Lock path on real hardware.
GPU inference. faster-whisper supports CUDA, so an NVIDIA-GPU Windows machine should be dramatically faster.
AgentCore Memory / Browser / Payments on Windows. The code path is shared with macOS but I haven't verified it on Windows yet.

The Windows port is functional on UTM, published, and documented in
README_win.md. If you try it on bare metal, issues/PRs welcome:
github.com/yama3133/apex-voice.

Apex Voice v2 — Killing Hallucinations, Cutting Latency, and Landing on Caps Lock

Yuuki Yamashita — Sun, 21 Jun 2026 14:06:10 +0000

Follow-up to my first post on Apex Voice — the macOS menu-bar
voice-typing app I built with mlx-whisper and Amazon Bedrock.
This is the "make it actually pleasant to use every day" pass.

Where I Left Off

A few weeks ago I shipped v0.2 — local Whisper, Bedrock post-processing,
Strands Agents for multi-step tool calls, AgentCore Memory for
vocabulary learning. It worked. But day-to-day it had three rough edges:

YouTube-style hallucinations — "Thanks for watching, see you in the next video!" appearing mid-sentence when I hadn't said anything.
Latency — noticeable lag between speaking and seeing text.
Hotkey ergonomics — ⌃⌥V worked, but it's a three-finger contortion I never got comfortable with.

v2 is about fixing all three plus adding two real Bedrock AgentCore
features I'd been deferring.

Repo: github.com/yama3133/apex-voice
Companion web: apex-voice-web.vercel.app

1. Killing the Hallucinations (the layered way)

Whisper hallucinations aren't a bug — they're a side effect of how
the model was trained. The Japanese training set is dominated by
YouTube captions, so when Whisper sees silence or noise, the highest-
probability output is a polite YouTube sign-off. There's no single fix.
You have to layer defenses.

Layer 1: push-to-talk, not VAD-driven segmentation.
The original recorder used RMS-based VAD to detect the start and end
of each utterance. That meant any time my mic picked up keyboard
noise or breath, a "phantom utterance" got passed to Whisper —
and Whisper helpfully invented words. I tore it out:

# Listening mode = take everything, no judgment
if not self._speaking:
    self._speaking = True
    self._buf = list(self._pre)
self._buf.append(block)

The user explicitly toggles recording on/off (now via Caps Lock,
see §3), so VAD inside the recording window is redundant. The
gate is now at the boundary, not inside.

Layer 2: Silero VAD as the final gate before Whisper.
Once a recording ends, I run it through Silero VAD. If Silero says
"no speech here," the audio never touches Whisper.

if not _silero_has_speech(audio):
    log("Silero VAD: no speech → drop")
    return ""

I also preload Silero at app startup on a background thread, so
the first utterance doesn't pay the model-load cost.

Layer 3: tighter mlx-whisper decoding.
Three knobs:

kwargs = dict(
    no_speech_threshold=0.7,         # was 0.5 — bias toward silence
    logprob_threshold=-0.5,          # drop low-confidence outputs
    compression_ratio_threshold=2.2, # drop repetition spirals
    temperature=0.0,                 # no fallback ladder
)

Temperature fallback was a hidden latency cost too (more on that
in §2).

Layer 4: STRONG vs WEAK phrase filters.
My old filter rejected outputs that were mostly a known YouTube
phrase. But "次の動画でお会いしましょう" ("see you in the next video")
has the trigger phrase "次の動画" at the front, followed by 9
characters of plausible-looking text. The "mostly" check let it
through.

The fix was simple. Split the phrase list into two:

STRONG: if this phrase appears anywhere, drop. ("see you in the next video", "channel registration", "thank you for watching")
WEAK: only drop if the phrase is the dominant content. ("good night" — could be a real utterance)

for p in STRONG_HALLUCINATION_PHRASES:
    if p in text:
        return True  # immediate kill

2. Cutting the Latency

I added timing logs to every stage to find the real bottleneck:

audio=4.8s vad=56ms whisper=1499ms → 19 chars

Whisper is the bottleneck. Always. So I attacked it three ways.

Async clipboard restore. The clipboard dance was blocking:

self._pb_set(text)
ok = self._paste()  # this is what the user feels
# THEN sleep 150ms, THEN restore...

Moved restore to a background thread. The user-perceived insertion
time dropped from ~450ms (first time) to ~150ms steady-state.

Quantized Whisper model. Switched the default from
mlx-community/whisper-large-v3-turbo to its 4-bit quantized
sibling whisper-large-v3-turbo-q4. Honest result:

	turbo (default)	turbo-q4
Model size	~800MB	~350MB
Whisper time (4s audio)	~1500ms	~1500ms
Accuracy (subjective)	high	same or slightly better

MLX is already heavily optimized, so the speed delta is ~0%.
But the memory and disk halve, accuracy didn't degrade in my
testing, and there's no reason not to take the win.

The failed experiment. I also tried
mlx-community/distil-whisper-large-v3 — theoretically 2× faster.
Japanese accuracy collapsed. Distil-Whisper is English-tuned and
it shows. Reverted in 30 seconds. Worth saying out loud:
always measure end-to-end, not just throughput.

End state: ~1.5s for a 4-second utterance, on M-series Mac.
That's the floor with this model.

3. Caps Lock as the One-Key Hotkey

The original hotkey was ⌃⌥V via pynput.GlobalHotKeys. Functional,
but I never warmed to the three-finger combo. I wanted one key
that I could mash without thinking — but every reasonable single-key
candidate has a problem:

Letter keys (single v, a, etc.) — kills normal typing
Right Cmd / Right Option — pynput can't distinguish left/right
fn (globe key) — macOS hides this from userspace
Caps Lock — taken by macOS for its toggle behavior
F13–F19 — perfect, but most keyboards don't have them physically

The fix: rebind Caps Lock to F19 at the OS layer with
Karabiner-Elements, then make Apex Voice listen for F19. Karabiner
is a battle-tested OSS keyboard remapper for macOS. The config is
a one-rule JSON file:

{
    "description": "Caps Lock -> F19 (for Apex Voice)",
    "manipulators": [{
        "type": "basic",
        "from": {
            "key_code": "caps_lock",
            "modifiers": { "optional": ["any"] }
        },
        "to": [{ "key_code": "f19" }]
    }]
}

Apex Voice ships this rule and a karabiner://import?url=… link
that adds it with one click. The user-facing result: tap Caps
Lock once to start recording, tap again to stop. Best UX upgrade
in the whole project.

There's a small subtlety here for cross-platform thinking. The
same hotkey config value (<f19>) works on Windows too, because
pynput.GlobalHotKeys parses identically. On Windows the user
would remap Caps Lock → F19 with PowerToys Keyboard Manager.
Different tool, same shape — no code branches.

4. AgentCore Browser for Real Web Search

The agent's "web search and summarize" tool was originally
requests + BeautifulSoup against Google's HTML results. It works
right up until Google's bot detection notices and blocks the
serverless IP. Predictable.

The fix: a two-stage fetch.

1) requests + BeautifulSoup (fast, free)
   ↓ fail / too short / Google blocked
2) AgentCore Browser (managed Chromium + Playwright over CDP)

AgentCore Browser is a managed headless Chromium environment on AWS,
addressed via boto3 + an authenticated WebSocket. You connect
Playwright with connect_over_cdp and drive it like any browser
session. Browsers with bot detection, JS-required pages, and Google's
search results all work, because it really is a real browser.

client = BrowserClient(region=BEDROCK_REGION)
client.start()
ws_url, headers = client.generate_ws_headers()
with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(ws_url, headers=headers)
    page = browser.contexts[0].new_page()
    page.goto(url, wait_until="domcontentloaded", timeout=30000)
    text = page.evaluate("() => document.body.innerText")

Sessions are started per-request and torn down — cost over comfort,
since web summarization is a low-frequency action.

5. AgentCore Payments (x402) — Letting the Agent Pay

This is the headline new capability. The agent can pay for things.

AgentCore Payments is
the Bedrock service for AI-agent-initiated payments via embedded
crypto wallets (Coinbase CDP or Stripe Privy). It speaks
x402, the "HTTP 402 Payment Required"
protocol revived for the agent era — a paid API responds with 402
and a payment manifest, the agent generates a payment header and
retries.

The Apex Voice tool wires it into the existing approval flow:

@tool
def pay_for_paid_resource(
    url: str, max_amount_usd: float, description: str = ""
) -> str:
    """Pay for a paid HTTP resource via x402.

    1) GET the URL → expect HTTP 402
    2) Create a PaymentSession with a spend cap
    3) generate_payment_header for the 402 body
    4) Re-GET with the payment header
    """

The flow:

voice: "pay this API up to 5 cents"
  → Whisper transcript
  → Strands Agent picks pay_for_paid_resource
  → Guardrail check (per-request / per-day caps)
  → Human approval dialog (or Slack approval via Aegis)
  → PaymentManager.create_payment_session
  → generate_payment_header
  → HTTP retry with header
  → result inserted at cursor

Setting it up isn't trivial — you need a PaymentManager + Connector
configured with Coinbase CDP or Stripe Privy credentials in the AWS
console, and an embedded wallet funded for the user. Not a quick demo,
but the code path is real and the approval flow is structurally what
"giving an AI agent a wallet" should look like.

The New Architecture

Everything composes into something that genuinely runs all day.

(see architecture diagram below)

The pipeline is local-first: mic → recording → Silero VAD → mlx-whisper
→ insertion. Bedrock features layer on top per-utterance only when the
mode demands them.

What I Learned (v2 Edition)

For hallucinations, layered defense beats any single fix. Pre- filter (Silero VAD), in-flight params (logprob, no_speech), post- filter (phrase lists). No single layer is enough.
Measure before optimizing, even when "obvious." I was sure the quantized model would be faster. It wasn't. MLX already had the win baked in.
The right primitive beats a clever hack. Three-key combo vs. Caps Lock + Karabiner — same goal, but one is invisible to the user.
AgentCore is more than Memory. Browser solves a real bot- detection problem; Payments turns "give the agent a wallet" from a thought experiment into a real code path with human approval baked in.

What's Next

A real x402 endpoint to demo against. Right now the Payments code path is functional but I'm exercising it with a synthetic 402 responder. I want to wire it to a real paid API.
Windows port. mlx-whisper → faster-whisper, rumps → pystray. The hotkey config (<f19>) is already portable; Windows users rebind via PowerToys.
Vocabulary UI. AgentCore Memory has been quietly learning proper nouns for weeks. I have no UI to inspect or curate it.

If you tried v0.2 and got bitten by the hallucinations or hotkey,
v2 is the version to come back to. Issues and PRs:
github.com/yama3133/apex-voice.

Building a chat-based Marp slide generator with Next.js, Amazon Bedrock, and Lambda

Yuuki Yamashita — Fri, 19 Jun 2026 17:03:10 +0000

Overview

I built an app that turns natural language into Marp slides through a chat interface.

Live: marp-ai-app.vercel.app
Repo: yama3133/marp-ai-app (Public)

You can ask "Make 8 slides on AI agent use cases" and the app generates the deck. Then keep chatting: "Make slide 3 more concise" and it edits in place. Exports to PDF and editable PPTX. There is also a "My Themes" feature that lets you paste any Marp CSS, save it to localStorage, and switch on the fly.

This post is a tour of the architecture and the rough edges I hit.

Architecture

Browser
   ↓
Next.js 16 (Vercel) ─ /api/generate → Amazon Bedrock (Sonnet 4.6, us-east-1)
                    └ /api/export   → Lambda (arm64, ap-northeast-1)
                                       ├ marp-cli + Chromium     → PDF
                                       └ marp-cli + LibreOffice  → editable PPTX
                                       └ S3 (1-day lifecycle)     → presigned URL
Auth: Vercel OIDC Federation → IAM Role (keyless, no access keys)

Role	Tech
Frontend / API	Next.js 16 (App Router, Route Handlers)
Hosting	Vercel
LLM	Amazon Bedrock Converse API (`us.anthropic.claude-sonnet-4-6`, us-east-1)
Export	AWS Lambda (container, arm64, ap-northeast-1)
File delivery	Amazon S3 + presigned URLs (1-day lifecycle)
Auth	Vercel OIDC Federation → IAM Role (keyless)
Preview	marp-core (rendered in the browser)

1. Multi-turn chat with Bedrock Converse

I wanted a plain chat experience — user message, assistant reply, follow-up edits — so I pass the conversation directly to the Bedrock Converse API's messages[].

Message shape

type ChatMessage = {
  id: string;
  role: "user" | "assistant";
  content: string;
  markdown?: string;
};

Each assistant message holds both content (the short conversational reply) and markdown (the Marp deck). When sending to the API, I concatenate them into a single message:

function buildApiMessages(messages: ChatMessage[]): ApiMessage[] {
  return messages.map((m) => ({
    role: m.role,
    content:
      m.role === "assistant" && m.markdown
        ? `${m.content}\n\n${m.markdown}`.trim()
        : m.content,
  }));
}

System prompt: reply first, then Markdown

I want both a chat reply and a Marp deck back, so the system prompt enforces the format:

- Start with a 1-2 sentence natural reply.
- If generating or editing slides, follow the reply (no blank line) with Marp Markdown.
- Never wrap the Markdown in code fences (```

).
- On edit requests, re-output the entire deck (no diffs, no patches).

The server splits on the first ---\nmarp: occurrence:


ts
function parseResponse(text: string): { message: string; markdown: string } {
  const trimmed = text.trim();
  let idx = trimmed.search(/(?:^|\n)---\s*\n\s*marp:/);
  if (idx === -1) idx = trimmed.search(/(?:^|\n)---\s*\n/);
  if (idx === -1) return { message: trimmed, markdown: "" };
  const splitAt = idx === 0 ? 0 : idx + 1;
  return {
    message: trimmed.slice(0, splitAt).trim(),
    markdown: trimmed.slice(splitAt).trim(),
  };
}

Full-deck re-outputs are easier to keep coherent than diffs (frontmatter, page breaks, closing slide all stay consistent). Since the chat history is already in Bedrock's context, vague follow-ups like "shorter please" still work.

2. Making preview and export look the same

Marp themes are CSS. The challenge:

Preview: marp-core running in the browser
Export: marp-cli running in Lambda

If the theme CSS or font override drifts between the two, you get the classic "preview looks fine but the exported file looks different" bug.

One source of truth for theme CSS

src/lib/themes.ts defines six custom theme CSS strings. Both sides consume them:


ts
// Preview side (marp-core)
for (const t of CUSTOM_THEMES) {
  if (t.css) marp.themeSet.add(t.css);
}

// Export side (Lambda: marp-cli)
if (themeCss) {
  const themePath = path.join(work, "theme.css");
  await writeFile(themePath, themeCss, "utf8");
  args.push("--theme-set", themePath);
}

Font override via a `<style>` block in the Markdown

Marp treats <style> tags inside the Markdown as deck-wide CSS. Inject the same override into both marp-core and marp-cli:


ts
export function fontStyleBlock(font: FontId): string {
  const stack = getFontStack(font);
  if (!stack) return "";
  return `<style>
section,
section :is(h1, h2, h3, h4, h5, h6, p, li, blockquote, th, td, a, strong, em) {
  font-family: ${stack} !important;
}
</style>`;
}

3. User-defined themes (localStorage)

I wanted to let users paste any Marp CSS and use it as a theme. The trick is making sure the /* @theme name */ header is always present — if the user forgot, generate one from the label and prepend it:


ts
export function ensureThemeHeader(
  css: string,
  fallbackLabel: string,
): { name: string; css: string } {
  const existing = parseThemeName(css);
  if (existing) return { name: existing, css };
  const name = slugifyThemeName(fallbackLabel);
  return { name, css: `/* @theme ${name} */\n${css}` };
}

Validate against builtin names → save to localStorage → list under a "My Themes" optgroup → on export, send the CSS via userThemeCss so Lambda's existing --theme-set path picks it up. No Lambda changes required.

4. Lambda container: Chromium + LibreOffice in one box

One Lambda handles both formats:

marp-cli + Chromium → PDF (Marp's default path)
marp-cli + LibreOffice → editable PPTX (--pptx --pptx-editable)

Three Lambda-specific gotchas worth flagging.

Chromium dies with `Connection closed`

Lambda has a read-only filesystem, tiny /dev/shm, and forbids sandboxing — Chromium's zygote/multi-process model falls over. Marp doesn't expose low-level browser args, so route through a wrapper script via --browser-path:


dockerfile
RUN printf '#!/bin/sh\nexec /usr/bin/chromium \
  --no-sandbox --disable-dev-shm-usage --disable-gpu \
  --disable-software-rasterizer --single-process --no-zygote "$@"\n' \
  > /usr/local/bin/chromium-marp \
  && chmod +x /usr/local/bin/chromium-marp
ENV CHROME_PATH=/usr/local/bin/chromium-marp

marp-cli hangs waiting on stdin

In Lambda's child_process.spawn, stdin is an open pipe by default, and marp blocks waiting for input. Always pass --no-stdin:


js
const args = ["--no-stdin", "--allow-local-files"];

M PLUS Rounded 1c font family name mismatch

The TTF's internal family name is Rounded Mplus 1c, so listing only 'M PLUS Rounded 1c' in CSS won't match. List both:


css
font-family: 'M PLUS Rounded 1c','Rounded Mplus 1c',sans-serif;

5. Multi-stage build: 3.18 GB → 2.39 GB

My first Dockerfile was a single stage at 3.18 GB. The culprit: the toolchain needed for aws-lambda-ric's native build (g++ / cmake / automake / python3 / libcurl4-openssl-dev) was still in the runtime image.

Split into three stages — runtime dropped to 2.39 GB (-790 MB, -25%):


dockerfile
# 1. Font fetcher
FROM debian:bookworm-slim AS fonts
RUN apt-get install -y --no-install-recommends curl ca-certificates
COPY install-fonts.sh /tmp/
RUN sh /tmp/install-fonts.sh

# 2. node_modules builder (aws-lambda-ric native build)
FROM node:22-bookworm-slim AS builder
RUN apt-get install -y --no-install-recommends \
      g++ make cmake autoconf automake libtool pkg-config python3 \
      libcurl4-openssl-dev ca-certificates
COPY package.json ./
RUN npm install --omit=dev && npm cache clean --force

# 3. Runtime
FROM node:22-bookworm-slim AS runtime
RUN apt-get install -y --no-install-recommends \
      chromium libreoffice-impress fonts-noto-cjk fonts-noto-color-emoji fontconfig
COPY --from=fonts /usr/share/fonts/truetype/marpfonts /usr/share/fonts/truetype/marpfonts
RUN fc-cache -f /usr/share/fonts/truetype/marpfonts
COPY --from=builder /build/node_modules ./node_modules

Key points:

fonts stage uses curl to grab 7 font families — runtime never sees curl
builder stage owns the toolchain; only node_modules is copied to runtime
Runtime only carries chromium + libreoffice-impress + fonts-noto-cjk + fontconfig

6. Keyless AWS from Vercel via OIDC Federation

I didn't want to store AWS access keys in Vercel env vars. Vercel's OIDC Federation lets you exchange a short-lived token for AWS credentials.

The IAM trust policy locks scope and project:


json
{
  "Effect": "Allow",
  "Principal": {
    "Federated": "arn:aws:iam::761018866498:oidc-provider/oidc.vercel.com/yuuki-yamashitas-projects"
  },
  "Action": "sts:AssumeRoleWithWebIdentity",
  "Condition": {
    "StringEquals": {
      "oidc.vercel.com/yuuki-yamashitas-projects:aud":
        "https://vercel.com/yuuki-yamashitas-projects",
      "oidc.vercel.com/yuuki-yamashitas-projects:sub":
        "owner:yuuki-yamashitas-projects:project:marp-ai-app:environment:production"
    }
  }
}

On the Next.js side, @vercel/functions/oidc does the heavy lifting:


ts
import { awsCredentialsProvider } from "@vercel/functions/oidc";

export function awsCredentials() {
  if (process.env.VERCEL) {
    return awsCredentialsProvider({
      roleArn: process.env.AWS_ROLE_ARN!,
    });
  }
  return undefined; // local: default AWS credential chain
}

Locally I fall back to ~/.aws/credentials via the default chain.

Wrap-up

Multi-turn chat editing falls out of Bedrock Converse messages[] if you let the model re-output the full deck on every turn
Match preview and export by sharing theme CSS and injecting font overrides through a <style> block in the Markdown
For Chromium + LibreOffice in one Lambda: --single-process --no-zygote --no-sandbox and --no-stdin are non-negotiable
Multi-stage build saved 790 MB
Vercel → AWS goes through OIDC Federation, no keys involved

Repo: yama3133/marp-ai-app
Live: marp-ai-app.vercel.app

Feedback welcome.

Voice Typing Anywhere on macOS — I Built Apex Voice with mlx-whisper, Amazon Bedrock, and Strands Agents

Yuuki Yamashita — Fri, 19 Jun 2026 14:22:06 +0000

A weekend project that turned into a daily-driver: a macOS menu-bar app that lets you talk into any input field — Slack, browser, Notes, anywhere — powered by local Whisper and Amazon Bedrock.

TL;DR

I built Apex Voice, an open-source macOS voice typing tool. It listens through your microphone, transcribes speech offline with mlx-whisper, and inserts the result wherever your cursor is. With Amazon Bedrock layered on top, it can also polish the text, translate it, or execute agent actions like "add a reminder" or "summarize this page for me."

Repo: github.com/yama3133/apex-voice
Companion web app: apex-voice-web.vercel.app

Why I Built It

macOS already has built-in dictation, and there are great commercial tools like Aqua Voice. So why bother?

Built-in dictation doesn't reliably work in every app, and Japanese accuracy is uneven.
Aqua Voice is polished but closed and paid.
I wanted to own the stack: pick my model, my post-processing, my agent tools. And to learn by building.

What It Does

The core loop:

Mic → VAD → mlx-whisper → (optional Bedrock post-process) → Clipboard → ⌘V into any app

On top of that:

Post-process modes powered by Claude Haiku 4.5 on Bedrock: polish, formal, translate, bullets.
Agent mode via Strands Agents: one utterance can trigger multiple tool calls — add a reminder, open a calendar event, fetch a webpage and summarize it.
Vocabulary learning with Amazon Bedrock AgentCore Memory: proper nouns and domain terms accumulate over time and get injected as Whisper's initial_prompt, so accuracy improves with use.

Architecture

The whole thing runs as a Python process managed by launchd. Local-only by default; Bedrock features kick in when you enable them.

The main pipeline (mic → whisper → insertion) is fully offline. Bedrock and AgentCore Memory are auxiliary — they make the experience richer but the app works without them.

The AWS Side

Three Bedrock-shaped pieces:

Piece	Role
Amazon Bedrock (Claude Haiku 4.5)	Post-processing (rewrite/translate/bullets), agent classification, page summarization
Strands Agents	Defines tool schemas and orchestrates multi-step calls to Claude
Amazon Bedrock AgentCore Memory	Persists extracted vocabulary across sessions; injected as Whisper prompt hints

I picked Haiku 4.5 because the post-processing happens every utterance. Sub-second latency matters more than top-tier reasoning here. For the agent mode, Haiku is still strong enough to pick the right tool from ~10 options reliably.

The companion web app (apex-voice-web) runs on Vercel with a Python serverless function that calls Bedrock for URL classification and summarization. It uses Upstash Redis for a live history feed. The macOS app fires history entries to it via a non-blocking POST.

The Hard Part Wasn't AI. It Was Packaging.

I lost almost a full day to py2app.

The plan was obvious: setup.py py2app → dist/Apex Voice.app → drop it in /Applications → done. Reality:

[12:12:22] 認識エラー: bad local file header:
  '/Users/.../Apex Voice.app/Contents/Resources/lib/python312.zip'

py2app bundles your dependencies into a python312.zip, and mlx-whisper's native extensions don't survive being zipped. I tried zip_include_packages: [] — not a valid py2app option. I tried a shell-script launcher inside the .app — macOS warned the user about needing Rosetta. I tried a hand-compiled arm64 launcher binary — that worked, but every iteration meant re-granting Accessibility permission because the bundle signature changed.

The fix: stop building a .app entirely.

I switched to a LaunchAgent plist (com.yamashita.apexvoice.plist) that points directly at the venv's Python and the script. Drop it in ~/Library/LaunchAgents/, launchctl load, done. With KeepAlive: true, it auto-restarts on crash. The "restart" menu item just calls rumps.quit_application() and lets launchd bring it back.

Two small touches kept it feeling like a proper app:

import setproctitle
setproctitle.setproctitle("Apex Voice")

So Activity Monitor shows "Apex Voice" instead of "python3.12". And for the menu-bar icon, I rendered SF Symbols (waveform.and.mic / mic.fill) to PNG once and loaded them as a template image — they auto-adapt to light/dark mode.

What I Learned

Don't fight the OS packaging story when you have a better path. A LaunchAgent + venv beat py2app on every axis: simpler, more stable, easier to update.
Pick the model for the latency profile, not the leaderboard. Haiku 4.5 wins here because users feel every 500 ms in a voice typing loop.
Bluetooth mic quality is an OS problem, not an app problem. When the mic engages HFP mode, sample rate drops to 8 kHz across the board. No app-level fix exists on macOS. (Windows handles this slightly better in some driver combos, but the underlying HFP/A2DP tradeoff is universal.)
Accessibility-driven keystroke injection (osascript-Cmd-V) is the right primitive. It works in every app I've tried — Slack, browsers, native editors. No per-app integration needed.

What's Next

Vocabulary learning UI — show the user what AgentCore Memory has learned.
Windows port — swap mlx-whisper for faster-whisper, rumps for pystray. The core loop is OS-agnostic.
Approval-gated agent actions — wire it into Aegis, a Slack-based approval plane I'm building for AI agents.

Try It

git clone https://github.com/yama3133/apex-voice.git
cd apex-voice
/opt/homebrew/bin/python3.12 -m venv .venv
.venv/bin/pip install -r requirements.txt
# Run directly:
.venv/bin/python voicetype.py
# Or install as a LaunchAgent (auto-start, auto-restart):
# Edit com.yamashita.apexvoice.plist to point at your clone path
cp com.yamashita.apexvoice.plist ~/Library/LaunchAgents/
launchctl load ~/Library/LaunchAgents/com.yamashita.apexvoice.plist

You'll need an Apple Silicon Mac (mlx is Apple Silicon only) and AWS credentials if you want post-processing and agent features.

If you build something similar, or hit the same py2app wall I did, I'd love to hear about it. Code, issues, and PRs welcome on GitHub.

I gave my AI agent a boss: a human-approval gate in Slack, over MCP

Yuuki Yamashita — Thu, 11 Jun 2026 14:49:11 +0000

AI agents can now act, not just suggest. They issue refunds, run migrations, message customers. That's powerful — and a little terrifying. "Autonomous" should not mean "unsupervised." The moment an agent can spend money or drop a production table, someone needs to be able to say "wait — not like that."

So I built Aegis: a human-approval control plane for AI agents. Before an agent does anything high-risk, it asks a human for approval in Slack, and waits.

▶️ 90-second demo: https://youtu.be/c1jqPDPo6AU
💻 Code: https://github.com/yama3133/aegis-slack-app

The shape of the idea

The gate has to be decoupled from the agent — I didn't want to wire approval logic into every agent or framework. The Model Context Protocol (MCP) is the perfect seam: Aegis is just an MCP server with three tools, and any agent can adopt it without changing its reasoning.

// src/mcp-server.ts
server.registerTool("request_approval", {
  description:
    "Request human approval in Slack before executing a high-risk action. " +
    "ALWAYS call this before destructive, financial, or externally visible actions.",
  inputSchema: {
    agent: z.string(),
    action: z.string(),
    args: z.record(z.string(), z.unknown()),
    risk: z.enum(["low", "medium", "high", "critical"]),
    reason: z.string().optional(),
  },
}, async (input) => {
  const req = await postApprovalRequest(slack, input, DEFAULT_CHANNEL);
  return text({ request_id: req.id, status: req.status, auto_approved: req.autoApproved });
});

Three tools in total: request_approval, check_approval, and wait_for_approval. The agent calls request_approval before a risky action, then blocks on wait_for_approval until a human decides.

The agent side

In the demo, the agent is an Amazon Bedrock model (Claude Sonnet 4.6) running a Converse API tool-use loop. The only thing that makes it "safe" is a system prompt and the MCP tools:

Before ANY high-risk action (refunds, deletions, external messages, anything
financial or destructive), you MUST call request_approval ... then call
wait_for_approval until you get a terminal status.
If approved with edited arguments, you MUST use the returned arguments.
If denied or expired, do NOT perform the action.

The agent pauses on its own. No orchestration framework required — just a tool call.

The approval card

When a request comes in, Aegis posts a Block Kit card to Slack. A human gets the agent, the action, the risk level, the exact arguments, and four buttons:

✅ Approve — the agent proceeds
❌ Deny — the agent safely aborts
✏️ Edit & Approve — the part I'm proudest of
❓ Request Info

The feature I didn't expect to love: Edit & Approve

A plain yes/no felt too blunt. Real reviewers don't just stop an agent — they correct it. So Edit & Approve opens a modal with the agent's arguments as editable JSON:

// src/blocks.ts
export function editModal(req) {
  return {
    type: "modal",
    callback_id: "aegis_edit_modal",
    title: { type: "plain_text", text: `Edit #${req.id}` },
    submit: { type: "plain_text", text: "Approve edited" },
    blocks: [{
      type: "input",
      element: {
        type: "plain_text_input",
        multiline: true,
        initial_value: JSON.stringify(req.args, null, 2),
      },
    }],
  };
}

In the demo, a reviewer lowers a refund from $1,200 to $800 and approves. The agent then executes the corrected amount and reports the change. Control, not just a veto.

Not everything needs a human

If every $45 goodwill refund paged a person, nobody would use this. So Aegis ships a small policy engine:

// src/policy.ts
if (multi.risks.includes(risk) || amount >= multi.minAmountUsd) {
  return { decision: "human", approvalsRequired: 2 };   // e.g. drop a prod table
}
if (auto.enabled && auto.risks.includes(risk) && amount <= auto.maxAmountUsd) {
  return { decision: "auto_approve" };                  // e.g. a $45 refund
}
return { decision: "human", approvalsRequired: 1 };

Auto-approve low-risk actions (🤖 instantly).
N-of-M for critical ones — dropping a production table needs two approvers.
TTL — anything left pending simply expires. Fail-safe by default.

Context, so it's a decision and not a rubber stamp

The thing I learned: a good approval is less about a button and more about context. Two touches:

Plain-language summary of every action via Amazon Bedrock (Claude Haiku 4.5), so reviewers don't read raw JSON.
Related Slack messages pulled onto the card with Slack's Real-Time Search API (assistant.search.context) — the relevant conversation is right there, with permalinks.

The fiddly bit: assistant.search.context needs a fresh action_token, which only arrives on a Slack assistant/mention event and lives for minutes — but approval requests originate outside any Slack event. I cache the latest token so out-of-band cards can still search.

How it all fits together

Any agent → request_approval over MCP → Aegis (policy + context + Block Kit) → Slack → human → the decision returns to the agent via wait_for_approval. Every step is written to an audit log.

What I'd tell past me

MCP is the right abstraction for guardrails. Because the gate is decoupled, it works with any agent.
"Human in the loop" is a UX problem, not just a yes/no. Summary + context + the ability to edit are what make it usable.
Put the control surface where people already are. Slack means zero new tooling for the approver.

Built for the Slack Agent Builder Challenge. Stack: TypeScript · Slack Bolt (Socket Mode) · MCP · Amazon Bedrock · Slack Real-Time Search API.

⭐ Code & setup: https://github.com/yama3133/aegis-slack-app
▶️ Demo: https://youtu.be/c1jqPDPo6AU

I built a web app that overlays Niconico-style scrolling comments onto your slides (PowerPoint-ready, keyless OIDC)

Yuuki Yamashita — Mon, 08 Jun 2026 00:13:34 +0000

What if comments scrolled across your talk slides like on Niconico Douga (a Japanese video site famous for comments flying over the video)? That idea turned into a web app that overlays scrolling comments onto an existing .pptx.

Live: https://nico-comment-app.vercel.app
Source: https://github.com/yama3133/nico-comment-app

What it does

Upload a .pptx
Pick which slides get comments (all / 1,3,5-7)
Type comments with colors, tune the scroll speed, etc.
Hit "Generate" and download a .pptx with the comments baked in

Run the slideshow in PowerPoint / Keynote and the comments scroll right-to-left.

First hurdle: how to animate inside PowerPoint

My first attempt used PowerPoint's native motion-path animation — writing the XML that moves a text box from right to left directly via python-pptx.

It failed three times in a row:

1st: all comments stacked in the center (wrong initial position)
2nd: nothing moved in the slideshow — it just froze
3rd: PowerPoint reported "there is a problem with the content" and "repaired" it by deleting the animation

The root cause was hand-written animation XML that didn't match PowerPoint's schema — and, more importantly, I was shipping it without being able to verify playback in my own environment. Lesson: don't ship "it should work" for something you can't verify.

The fix: overlay a transparent animated GIF

I switched approaches: generate a transparent animated GIF of scrolling comments and overlay it full-bleed on the slide.

PowerPoint / Keynote auto-play animated GIFs in slideshow mode (built-in)
A GIF lets me extract frames and verify it actually moves
No more animation-XML schema headaches

Generating the GIF with Pillow is just drawing comments frame by frame:

for f in range(n_frames):
    t = f / fps
    img = Image.new("RGBA", (width, height), (0, 0, 0, 0))
    d = ImageDraw.Draw(img)
    for it in items:
        x = width - speed * (t - it["start"])  # right -> left
        d.text((x, it["y"]), it["text"], font=font,
               fill=it["color"], stroke_width=stroke, stroke_fill=(0,0,0,255))
    # quantize to a palette with transparency, then append the frame

Then python-pptx's add_picture overlays the GIF over each slide. The pptx diff turned out to be just 4 spots ("register GIF in Content_Types", "add GIF to media", "add the relationship", "add a <p:pic> to the slide XML"), so it's reproducible in the browser or on the server.

Architecture

Frontend (Next.js / Vercel): pptx parsing (JSZip) and live preview (Canvas) happen entirely in the browser — you can preview without sending the deck anywhere
Backend (AWS Lambda, Tokyo): the heavy GIF generation runs in a container Lambda with Pillow + python-pptx, bundling Noto Sans JP for Japanese text
Storage (S3): uploads go straight to S3 via a presigned PUT, dodging Vercel/Lambda payload limits and handling larger decks. Inputs/outputs auto-delete after 24h

Keyless with Vercel OIDC Federation

I initially planned to expose a Lambda Function URL and call it directly, but public access was blocked by the environment's guardrails (403 Forbidden).

So I moved to Vercel OIDC Federation. The OIDC token issued to the Vercel function is exchanged via AWS AssumeRoleWithWebIdentity for short-lived credentials to invoke Lambda. The win: no long-lived access keys stored anywhere.

import { awsCredentialsProvider } from "@vercel/oidc-aws-credentials-provider";

const lambda = new LambdaClient({
  region: "ap-northeast-1",
  credentials: awsCredentialsProvider({ roleArn: process.env.AWS_ROLE_ARN }),
});

On the AWS side, register Vercel's issuer (https://oidc.vercel.com/<team>) as an OIDC provider, and scope the IAM role's trust policy to a specific project's production environment.

A note on rights

Niconico's feature of comments scrolling over a video (the "comment delivery system") is reportedly patented by its operator.

What this tool does is overlay pre-written comments as a visual effect; each comment is embedded as a transparent GIF and plays locally, without any network. That differs from a system where an unspecified crowd posts comments over the internet that are synchronized in real time via a server. Still, the right call depends on usage — for commercial/streaming use you should verify the rights yourself, and displaying real-time incoming comments during a live stream is out of scope for this tool. The app includes a notice page about this.

Takeaways

To make PowerPoint animate reliably, overlaying an animated GIF beat native animation
Choosing a method you can verify yourself is the fastest path in the end
Connecting to AWS keylessly via Vercel OIDC is great even for hobby projects

Give it a try → https://nico-comment-app.vercel.app

Building Fridge AI Health: An AI-Native Full-Stack App with Vercel and Amazon Aurora

Yuuki Yamashita — Sat, 06 Jun 2026 18:37:34 +0000

Snap a photo of your fridge, and an AI plans meals around your health goals, tracks your nutrition, and coaches you every week — in 7 languages.

🔗 Live demo: https://fridge-ai-app.vercel.app
💻 Source: https://github.com/yama3133/fridge-ai-menu

What I built

Fridge AI Health turns a single photo of your fridge into a personalized, health-aware meal plan.

The flow is simple:

Take a photo of the inside of your fridge.
The app recognizes the ingredients (OCR + vision).
An AI proposes menus tailored to your health goals (calories, protein, sodium limit, fiber), each with a nutrition breakdown.
You tap "Ate this" to log a meal.
A nutrition dashboard visualizes your daily progress and a 7-day trend.
The AI Health Coach analyzes your accumulated logs and gives concrete weekly advice.

It started as a simple "what can I cook with this?" demo, and grew into a continuous health-management tool — which is exactly where a real database earns its keep.

The stack

Frontend / hosting: Next.js (Pages Router) on Vercel
Database: Amazon Aurora PostgreSQL, provisioned through the Vercel Marketplace AWS integration
AI: Amazon Bedrock — Claude Sonnet 4.5 (us-east-1) for both menu generation and the weekly coach
Vision: Google Cloud Vision (TEXT_DETECTION) for reading product labels
ORM / auth: Drizzle ORM + NextAuth (Google OAuth) with the Drizzle adapter
Charts / i18n: Recharts, plus a small homemade i18n layer (7 languages)

The data model is intentionally relational: users, accounts, sessions, health_profiles, meal_logs, and coach_advices. The coach feature is the whole reason a database matters — it reads back what you've eaten over the last 7 days, aggregates it, and feeds that to the model. The value isn't in storing data; it's in what the AI does with the accumulated data.

Why this fits "frontend in minutes, scalable backend"

The hackathon's pitch is "build a frontend in minutes and a scalable backend." That mapped almost perfectly to how this came together:

The UI (camera capture, menu cards, dashboard, coach) is plain Next.js + Recharts.
The backend scalability comes for free from Aurora PostgreSQL behind Vercel serverless functions.
The "AI-native" part is Bedrock doing the heavy lifting on both the menu and the coaching side.

The hard parts (where I actually spent my time)

The happy path is boring to read about. Here's what actually cost me hours.

1. Aurora DSQL looked perfect — until foreign keys

My first instinct was Aurora DSQL: serverless, PostgreSQL-compatible, scales to zero. Great fit for a hackathon.

Then I checked the compatibility docs. DSQL does not support foreign key constraints (along with triggers, views, sequences at the time, etc.). My schema leans heavily on ON DELETE CASCADE foreign keys — every app table references users. Migrating would have meant ripping out referential integrity and re-implementing it in the application layer.

For a one-week build, that was too much risk. I switched to Aurora PostgreSQL, kept my schema as-is, and moved on. Lesson: "PostgreSQL-compatible" is a spectrum — always check the unsupported-features list before you commit your schema.

2. Passwordless DB access with Vercel OIDC + RDS IAM

The Vercel Marketplace Aurora integration doesn't hand you a DATABASE_URL with a password. Instead it sets up RDS IAM authentication via Vercel's OIDC federation — your serverless function assumes an AWS role and generates a short-lived token that acts as the DB password. No secrets hardcoded anywhere. Very nice... once it works.

My lib/db ended up supporting both modes — a connection string for local Docker, and IAM tokens for production:

import { Signer } from '@aws-sdk/rds-signer'
import { awsCredentialsProvider } from '@vercel/functions/oidc'

const signer = new Signer({
  hostname: process.env.DATABASE_PGHOST,
  port: Number(process.env.DATABASE_PGPORT),
  username: process.env.DATABASE_PGUSER,
  region: process.env.DATABASE_AWS_REGION,
  credentials: awsCredentialsProvider({
    roleArn: process.env.DATABASE_AWS_ROLE_ARN,
    clientConfig: { region: process.env.DATABASE_AWS_REGION },
  }),
})

const pool = new Pool({
  host: process.env.DATABASE_PGHOST,
  user: process.env.DATABASE_PGUSER,
  database: process.env.DATABASE_PGDATABASE,
  port: Number(process.env.DATABASE_PGPORT),
  password: () => signer.getAuthToken(), // fresh token per connection
  ssl: { rejectUnauthorized: false },
})

3. "No OpenIDConnect provider found"

First production connection attempt:

No OpenIDConnect provider found in your account for
https://oidc.vercel.com/<team>

The IAM role existed (the integration created it), but the OIDC identity provider it trusts did not. The role's trust policy pointed at oidc.vercel.com/<team> as a federated principal — and that provider simply wasn't registered in my AWS account.

The fix was one command:

aws iam create-open-id-connect-provider \
  --url "https://oidc.vercel.com/<team>" \
  --client-id-list "https://vercel.com/<team>"

Right after that, my migration endpoint reported current_user: postgres, can_create: true, and the tables went in. There's also a subtle gotcha worth flagging: the Aurora instance is not publicly accessible, so you can't just psql into it from your laptop — the Vercel-function-over-OIDC path is the way in. I ran migrations through a tiny, secret-protected /api/admin/migrate endpoint (and deleted it once the schema was applied).

4. Bigger model ≠ better product

For the AI, I assumed "use the most powerful model available." So I tried swapping Claude Sonnet 4.5 for an Opus model.

The result was instructive: Opus over-engineered the recipes. It happily designed elaborate dishes that required konjac, burdock root, daikon — ingredients that weren't in the fridge — which blew up the shopping list and quietly broke the app's core premise of cook with what you already have. It was also slower.

Sonnet 4.5 stayed closer to the detected ingredients, ran faster, and matched the product's intent better. I reverted. The "best" model is the one that fits the job, not the one with the highest benchmark.

Features I'm happy with

Health-aware generation. The user's profile (target calories, protein, sodium cap, fiber, goal) is injected straight into the prompt, so the menus actually respect "I'm cutting" vs "I'm bulking."
Structured shopping list. Each menu splits ingredients into in your fridge (blue) vs to buy (orange), and the page aggregates every "to buy" item into one shopping list. This also fixed an earlier inconsistency where the model would sometimes tag missing items and sometimes not — structuring the output removed the ambiguity.
The AI Health Coach. This is the centerpiece. It pulls the last 7 days from meal_logs, computes daily averages vs targets, and asks Claude for a summary, what's going well, what to improve, and concrete suggestions for tomorrow — returned as JSON and stored in coach_advices. Watching it say "your protein is great but you're ~340 kcal short and low on fiber — add whole grains tomorrow," grounded in real logged data, is the moment the whole app clicks.
7 languages. UI strings and the AI-generated menu text both switch between Japanese, English, Chinese, Korean, French, Spanish, and Portuguese — the language code is passed into the Bedrock prompt so the recipe names and descriptions come back localized.

What I'd do next

Move nutrition numbers from model estimates to a real nutrition API for accuracy.
Persist and chart the coach's weekly advice history as a trend.
Add household/shared fridges — a natural multi-user extension now that the relational schema and auth are in place.

Takeaways

Check the database's unsupported features before designing your schema. It saved me from a painful DSQL migration mid-build.
Passwordless IAM + OIDC is worth the setup. Once the OIDC provider is registered, you get short-lived credentials with zero secrets in your env.
Let the product pick the model. A more powerful model made the experience worse here.
A database becomes interesting when AI reads it back. Logging meals is mundane; an AI coach reasoning over a week of logs is the feature people remember.

Vercel handled the frontend and serverless glue, Aurora PostgreSQL handled the durable, relational core, and Bedrock turned stored data into advice. That combination is exactly the "AI-native full-stack" loop this hackathon is about.

Built for the H0 Hackathon (Vercel × AWS Databases).

Detect AI writing, then rewrite it human — Bedrock + Strands Agents + AgentCore Gateway

Yuuki Yamashita — Sat, 06 Jun 2026 14:03:29 +0000

Live demo: https://ai-text-checker-pi.vercel.app
Code: https://github.com/yama3133/ai-text-checker

I built AI Text Checker (Japanese name: 「AI文章判定くん」): paste some text and it scores how "AI-like" the writing is with reasons, then rewrites it to read as if a human wrote it. It works in Japanese and English, accepts paste / .txt / .md / .pdf, and is also exposed as an MCP tool so you can call it from Claude Code and other agents.

This post is less a feature tour and more a write-up of the decisions and gotchas — including one honest design choice that shaped the whole thing.

The honest premise: you cannot "detect AI" reliably

There is no technology today that definitively decides whether a text was written by a human or an AI. Dedicated detectors have high false-positive rates, especially on Japanese and on AI text that a human lightly edited. Asking an LLM to output a binary "AI or human" verdict is even shakier.

So I deliberately did not build an "AI detector." I built an AI-likeness editor:

❌ No binary "AI/human" verdict.
✅ An AI-likeness score (an estimate) plus the specific phrases that read AI-ish (mechanical connectives, safe generalities, formulaic closers, uniform rhythm, lack of specifics).
✅ A rewrite that keeps the meaning but sounds human.

The UI says this in plain text: it is an estimate, it can be wrong, don't use it as proof of authorship. That framing plays to what an LLM is actually good at — editorial feedback and rewriting — instead of pretending it can do something it can't.

Stack

Next.js 16 (App Router) + Tailwind, deployed on Vercel
Amazon Bedrock, Claude Sonnet 4.6 via the Converse API (us-east-1)
Strands Agents (TypeScript SDK) for the "rewrite until it passes" loop
Amazon Bedrock AgentCore Gateway to expose the tool over MCP

Pick a model you can actually invoke

I wanted Opus 4.8, but on this account it returned AccessDenied (403). A model showing up in list-foundation-models / list-inference-profiles does not guarantee you can invoke it. I checked candidates with a one-line Converse "ping" and landed on Sonnet 4.6:

aws bedrock-runtime converse --region us-east-1 \
  --model-id us.anthropic.claude-sonnet-4-6 \
  --messages '[{"role":"user","content":[{"text":"ping"}]}]' \
  --inference-config '{"maxTokens":5}'

The core analysis is a single Converse call with a system prompt that returns strict JSON (aiLikenessScore, label, summary, findings[], humanized). One detail that mattered for bilingual support: the commentary follows the UI language, but the rewrite must stay in the source language. The prompt says, in effect, "write the labels and reasons in English, but keep the rewrite in the same language as the input — never translate it." Without that line, English input sometimes came back rewritten into Japanese.

"Rewrite until it passes" with Strands Agents

A single rewrite might still read AI-ish. So there's a second mode: an agent that scores, rewrites, re-scores, and repeats until the AI-likeness drops below a threshold (or it hits a max number of rewrites). That is a genuine agent loop — a perfect fit for Strands.

The TypeScript SDK (@strands-agents/sdk) gives you an Agent, a tool() helper (Zod schemas), a BedrockModel, and structuredOutputSchema for typed results:

import { Agent, BedrockModel, tool } from "@strands-agents/sdk";
import { z } from "zod";

const scoreTool = tool({
  name: "score_ai_likeness",
  description: "Score how AI-like a text is (0-100). Call after every rewrite.",
  inputSchema: z.object({ text: z.string() }),
  callback: async (i) => JSON.stringify(await scoreOnce(i.text)),
});

const agent = new Agent({
  model: new BedrockModel({
    region: "us-east-1",
    modelId: "us.anthropic.claude-sonnet-4-6",
    maxTokens: 4096,
  }),
  tools: [scoreTool],
  structuredOutputSchema: z.object({
    finalScore: z.number(),
    iterations: z.number(),
    humanized: z.string(),
    notes: z.string(),
  }),
  systemPrompt: `Rewrite to read human. Score with score_ai_likeness,
    rewrite, re-score, repeat until below ${threshold} or ${maxRewrites} times.`,
});

const res = await agent.invoke(userPrompt);
return res.structuredOutput; // typed

In practice the model often nails it in one rewrite — a 92/100 sample dropped to 12/100, and the structured output told me exactly what changed.

Gotcha — bundling. Strands pulls in many optional peer deps. In Next.js, mark it external so the bundler leaves it alone at runtime:

// next.config.ts
const nextConfig = {
  serverExternalPackages: ["@strands-agents/sdk", "pdf-parse-fork"],
};

Gotcha — Vercel timeouts. The loop makes several Bedrock calls and can take 20–45s. Vercel Hobby caps functions at 60s, so a long auto-rewrite can time out. Fine locally; budget for it in production.

Exposing it over MCP with AgentCore Gateway

I wanted the tool callable from Claude Code, so I put it behind AgentCore Gateway as an MCP server. The shape:

MCP client (Claude Code) → AgentCore Gateway (MCP / JWT)
   → Lambda (detect_ai_style) → Bedrock (Sonnet 4.6)

The Lambda is a single Node file (the Node 22 runtime already bundles the AWS SDK v3, so no dependencies to package). The Gateway target points at the Lambda and carries the tool schema.

The big surprise: AgentCore Gateway's authorizerType is CUSTOM_JWT — and that's the only option. There is no anonymous gateway. So an MCP client must present a bearer JWT. I set up a Cognito user pool with a client_credentials (machine-to-machine) app client, and pointed the gateway's authorizer at Cognito's discovery URL:

aws bedrock-agentcore-control create-gateway \
  --name ai-text-checker-gateway --protocol-type MCP \
  --authorizer-type CUSTOM_JWT \
  --authorizer-configuration '{"customJWTAuthorizer":{
     "discoveryUrl":"https://cognito-idp.us-east-1.amazonaws.com/<POOL_ID>/.well-known/openid-configuration",
     "allowedClients":["<CLIENT_ID>"]}}' \
  --role-arn <GATEWAY_ROLE_ARN>

Two more sharp edges:

The Gateway invokes the Lambda with the tool arguments as the event, and passes the tool name via context as ${targetName}___${toolName}. With a single tool you can ignore the routing entirely.
The target toolSchema does not accept enum in property definitions — only type, properties, required, items, description. I moved the allowed values into the description.

Once it's up, the flow from a client is plain MCP JSON-RPC (initialize → tools/list → tools/call), with Authorization: Bearer <token>. The tool appears as detect___detect_ai_style. Tokens are short-lived, so I keep the client secret out of the repo and fetch a fresh JWT at runtime via the AWS CLI.

Takeaways

For "is this AI?", be honest: ship an estimate + editorial feedback, not a verdict.
Verify model access with a real Converse call before you build on a model id.
Strands makes an iterative, tool-using rewrite loop a few lines of typed code — but mark it external in Next.js.
AgentCore Gateway is a clean way to turn a Lambda into an MCP tool, but plan for mandatory JWT auth (Cognito), not an anonymous endpoint.

Try it: https://ai-text-checker-pi.vercel.app — and tell me where the score is wrong. It will be, sometimes. That's the point.

Never oversell, anywhere on Earth — building a global drop platform on Amazon Aurora DSQL

Yuuki Yamashita — Fri, 05 Jun 2026 17:01:22 +0000

Built for the H0 hackathon (Hack the Zero Stack with Vercel v0 and AWS Databases). #H0Hackathon

When a limited drop goes live, thousands of people press Buy in the same second. The classic failure modes — overselling, double-booking, the site falling over — are really one failure: losing track of a single number under global concurrency. Sell 101 units of a 100-unit drop and you've broken a promise to a real customer.

I kept watching limited drops oversell, double-book, and crash — and I wanted to know whether a database could make that impossible by design, not just patched over with queues bolted on top. So I built DROPZERO, a drop platform that sells exactly its stock and not one unit more, no matter how many buyers hit it at once or which region they come from. The whole thing runs on Amazon Aurora DSQL with a Next.js frontend on Vercel. Live demo: https://dsql-drop-app.vercel.app

Why Aurora DSQL

A single-region database can keep a stock counter honest with a transaction. The hard part is doing it across regions without giving up correctness. Most distributed setups force a choice: strong consistency or low latency or multi-region writes — pick two.

Aurora DSQL is the reason DROPZERO can refuse that trade-off. It's a serverless, distributed SQL database with multi-Region, active-active clusters that are strongly consistent. Two regional endpoints, one logical database, synchronous commit quorum. That means a buyer in Tokyo and a buyer in Seoul racing for the last unit are resolved against the same truth — exactly one wins, and the counter never goes negative. This is the property the entire product is built on.

The stack (and zero hardcoded secrets)

Frontend: Next.js (scaffolded with v0), deployed on Vercel.
Auth: Vercel OIDC federation → assume an AWS IAM role → mint a DSQL auth token per connection. No long-lived credentials anywhere in the app.
DB: Amazon Aurora DSQL.

The connection is refreshingly boring — which is the point:

import { AuroraDSQLPool } from "@aws/aurora-dsql-node-postgres-connector";
import { awsCredentialsProvider } from "@vercel/oidc-aws-credentials-provider";

const pool = new AuroraDSQLPool({
  host: process.env.PGHOST!,
  region: process.env.AWS_REGION!,
  user: "admin",
  database: "postgres",
  port: 5432,
  customCredentialsProvider: awsCredentialsProvider({
    roleArn: process.env.AWS_ROLE_ARN!,        // injected by Vercel
    clientConfig: { region: process.env.AWS_REGION! },
  }),
});

The first time I tried DSQL I burned an evening on "auth problems" — which turned out to be unrelated credentials. Lesson: let Vercel's OIDC integration own the token lifecycle and don't hand-roll DSQL tokens.

A deliberate, DSQL-native data model

Aurora DSQL is PostgreSQL-compatible, but it is not vanilla Postgres. Three constraints shaped the schema:

No foreign keys — integrity is enforced in the app.
No sequences / SERIAL — primary keys are UUIDs (gen_random_uuid() works as a server-side default).
One DDL statement per transaction — migrations run statement-by-statement.

The schema is tiny on purpose:

products(id uuid PK, name, drop_name)
inventory(id uuid PK, product_id, stock) ← the entire drop hinges on this one row
orders(id uuid PK, product_id, user_ref, status, region)

Every buyer on Earth is racing for the same stock cell. That's the whole game.

The core: never oversell

A purchase is a single transaction — check and decrement — and the only interesting part is what happens when two of them collide. Aurora DSQL uses optimistic concurrency control: it detects the conflict at commit time and rejects the loser with SQLSTATE 40001 (or DSQL's OC000 / OC001). So the app must retry.

const OCC = new Set(["40001", "OC000", "OC001"]);

async function purchase(productId, userRef, region, maxAttempts = 40) {
  for (let attempt = 1; ; attempt++) {
    const client = await pool.connect();
    try {
      await client.query("BEGIN");
      const upd = await client.query(
        "UPDATE inventory SET stock = stock - 1 WHERE product_id = $1 AND stock > 0",
        [productId],
      );
      if (upd.rowCount === 0) { await client.query("ROLLBACK"); return "sold_out"; }
      await client.query(
        "INSERT INTO orders (product_id, user_ref, status, region) VALUES ($1,$2,'confirmed',$3)",
        [productId, userRef, region],
      );
      await client.query("COMMIT");           // a conflict surfaces here
      return "confirmed";
    } catch (e) {
      await client.query("ROLLBACK").catch(() => {});
      if (OCC.has(e.code) && attempt < maxAttempts) {
        await backoffWithJitter(attempt);     // exponential + jitter
        continue;                             // retry re-reads the latest stock
      }
      throw e;
    } finally {
      client.release();
    }
  }
}

The elegant part: a retry re-reads the latest snapshot. Once stock hits 0, the WHERE stock > 0 matches no rows and the request cleanly returns sold out. The conflict becomes a correct answer, not an error.

Proving multi-region consistency

I created a multi-Region peered cluster: Tokyo (ap-northeast-1) and Seoul (ap-northeast-2) as active peers, with Osaka (ap-northeast-3) as the witness.

A gotcha worth saving you the hour I lost: I assumed the witness had to be a US region (a claim you'll find repeated online). us-east-1 and us-west-2 were both rejected. The witness must live in the same region set as the peers — for an APAC cluster, that's Osaka:

aws dsql create-cluster --region ap-northeast-1 \
  --multi-region-properties '{"witnessRegion":"ap-northeast-3"}'
# then create the Seoul peer and link the two ARNs

The demo writes to both regional endpoints at the same instant. Stock = 1, one purchase to Tokyo, one to Seoul: exactly one confirms, the other gets sold-out, and both endpoints report the same final stock. Strong consistency, across regions, with zero application-side coordination.

Does it hold at scale? (k6)

I pointed k6 at both regional endpoints: 100 units in stock, 3,000 concurrent purchases.

Retry budget	confirmed	sold_out	errors	final stock
8 attempts	100	2,810	90	0
40 attempts	100	2,900	0	0

Two takeaways:

Oversell never happened — confirmed is exactly 100 and final stock is exactly 0 in both runs. The 90 "errors" in the first run were failed purchases (conflicts that exhausted retries), never extra sales.
Hot-row contention needs a real retry budget. A single stock row under 50 concurrent VUs generates a lot of OCC conflicts; bumping the retry ceiling turned those 90 errors into clean sold-outs. Idempotent, retryable transactions are not optional here — they're the design.

Global from the first commit

A worldwide drop should read like home wherever you are. The UI ships in 8 languages — English, Japanese, Chinese, Korean, Spanish, French, Portuguese, and Arabic — including full right-to-left layout for Arabic (the whole interface mirrors; numbers and IDs stay LTR).

What I'd tell the next builder

Lean on the managed auth path. Vercel OIDC → IAM → DSQL token means no secrets in the app and no token plumbing.
Respect DSQL's dialect. No FKs, no sequences, one DDL per transaction, fixed Repeatable Read isolation. Design with it, not around it.
Treat OCC retries as a feature, not error handling. They're how "never oversell" actually works.
Multi-region witness = same region set. Don't trust the "US-only" folklore; trust the API error.

DROPZERO sells exactly what's in stock — to one buyer or to a million, in Tokyo or São Paulo — because Aurora DSQL gives you a single, strongly consistent source of truth across regions, and the rest is a small, careful transaction.

Live: https://dsql-drop-app.vercel.app · Architecture: https://dsql-drop-app.vercel.app/architecture

#H0Hackathon — built with Amazon Aurora DSQL and Vercel.

From static slides to a postable video: finishing my LINE-style chat generator with GitHub Copilot

Yuuki Yamashita — Wed, 03 Jun 2026 11:42:35 +0000

This is a submission for the GitHub Finish-Up-A-Thon Challenge.

What I Built

A while back I built a small web tool that turns a block of conversation text into a vertical (9:16), iPhone-style LINE chat slide — with the message bubbles animating in one by one. It was made for talks: paste a conversation, download an animated .pptx, present it. I even wrote it up here on DEV.

But it always had a missing half. A .pptx is great on a projector and awkward everywhere else — you can't really post it. What I actually wanted, and never finished, was to turn the same conversation into a video I could drop into a social post, where the bubbles pop in like a chat happening live.

This challenge was the push to finish that. The app now exports the conversation as a vertical mp4 (and webm), with bubbles fading in one at a time and a "typing…" indicator before each incoming message — all in the browser, no server involved.

Demo

App: https://line-chat-app-nine.vercel.app
Repo: https://github.com/yama3133/line-chat-app

The Comeback Story

Where it was

pptx generation only (python-pptx + hand-written animation XML), already shipped and documented.
There was an animated preview in the browser, but the only thing you could save was a pptx.
The "export it as a video" idea sat untouched for months.

What I added

Rebuilt the LINE / iPhone UI on a <canvas>. The pptx version draws with python-pptx shapes and the old browser preview was CSS — neither can be recorded as a video, so the UI had to be redrawn on canvas.
Accurate bubble wrapping with ctx.measureText, instead of the character-width guess the python version had to rely on.
The one-by-one fade-in animation, driven by a single renderFrame(ctx, opts, t) that draws the exact state at time t (the same function is reused for recording).
Recording with MediaRecorder. I expected to need ffmpeg.wasm for mp4 — but Chrome's MediaRecorder supports video/mp4;codecs=avc1.42E01E (H.264) directly, so mp4 comes out with no extra dependency.
A reliability fix that surprised me: a naive requestAnimationFrame record loop silently stalls when the tab isn't focused. I switched to setTimeout + canvas.captureStream(0) + track.requestFrame(), so recording keeps going even in a background tab.
A "typing…" indicator before each incoming bubble, for a more chat-like feel.

Still on the list

GIF export (handy for Slack / LINE) via ffmpeg.wasm or gif.js.

My Experience with GitHub Copilot

I leaned on Copilot for the part that's tedious but well-defined: drawing the LINE / iPhone UI on canvas. The approach was comment-driven — I wrote each function's spec as a comment first, then let Copilot fill in the body:

// Draw the phone frame (black, rounded) filling the logical area, then the
// inner screen (chat background) inset by L.phone.pad. Return {x, y, w, h}.
function drawPhoneFrame(ctx) { /* Copilot */ }

Where it shone:

Repetitive shape code — the 4 signal bars, the 3-dot menu, the battery with its little nub — came out correct from a one-line comment.
It picked up my existing constants (V for colors, L for layout) and reused them instead of hardcoding values.
Asking it to implement all the TODO(Copilot) functions at once filled the whole UI layer in a single pass.

Where I took over: the bubble-wrapping math, the animation timing, and the background-recording fix were things I designed and wrote by hand.

The split that worked for me: Copilot builds the UI scaffolding fast; I own the logic. My one practical tip is to treat comments as the spec — the more precise the comment, the better the completion.

Generating animated LINE-style chat slides with python-pptx + raw XML (and shipping it on Vercel)

Yuuki Yamashita — Tue, 02 Jun 2026 15:20:32 +0000

I was putting together a talk and had one of those half-baked ideas that you can't shake off: what if I showed an iPhone with a LINE chat screen, and the messages popped in one by one, like a real conversation happening live? The problem is, building that by hand in PowerPoint sounds miserable — laying out every bubble, then setting up an entrance animation for each one, one at a time. So instead I built a little tool that takes a chunk of conversation text and spits out a .pptx, and then I turned it into a web app and put it on Vercel.

This post focuses on the three things that actually gave me trouble:

Building an iPhone / LINE-style UI out of nothing but shapes in python-pptx
Working around the fact that python-pptx has no animation support — by hand-writing the timing XML and injecting it into the slide
Wiring up "conversation in, pptx out" with a static HTML front end and a Vercel serverless function (including the deploy that bit me)

Here's what came out of it:

And the slides it generates look like this:

The overall shape of it

It's nothing fancy. This is the whole file list:

.
├─ index.html        front end (input form + live preview)
├─ api/generate.py   conversation JSON → pptx (build_pptx) + handler
├─ requirements.txt  python-pptx
├─ vercel.json       builds (static + @vercel/python)
└─ dev_server.py     for local testing

All the real generation logic lives in Python (python-pptx), and the front end is plain HTML/CSS/JS. On Vercel, api/generate.py runs as a serverless function and index.html is served as a static file.

Building the iPhone / LINE UI out of shapes

Set the slide to a tall 9:16 aspect ratio, and from there it's just a matter of stacking rounded rectangles, ellipses, and text boxes.

SLIDE_W = Inches(7.5)
SLIDE_H = Inches(13.333)   # 7.5 : 13.333 = 9 : 16
prs.slide_width  = SLIDE_W
prs.slide_height = SLIDE_H

The stacking order goes: phone frame (black rounded rect) → screen (chat background) → green header → Dynamic Island → status bar (clock, signal, battery) → input bar. For something like the Dynamic Island, a black rectangle with its corner roundness (adjustment) cranked up to 0.5 already reads as the real thing.

isl = slide.shapes.add_shape(MSO_SHAPE.ROUNDED_RECTANGLE, ...)
isl.adjustments[0] = 0.5   # fully rounded into a "pill" shape

For the header and the input bar, I used ROUND_2_SAME_RECTANGLE ("rounded on the top two corners only" / "bottom two only") so they don't fight with the rounded corners of the screen. The input bar is just rotated 180° so its rounded side faces down.

One thing that quietly mattered: Japanese fonts. If you only set a:latin, Japanese text can fall back to some other font, so I push the same typeface into a:ea (East Asian) as well. That kept things consistent.

rPr = run._r.get_or_add_rPr()
for tag in ("a:latin", "a:ea", "a:cs"):
    el = rPr.find(qn(tag)) or rPr.makeelement(qn(tag), {})
    el.set("typeface", "Hiragino Sans")

Sizing the bubbles by guesswork

Each bubble gets a width and height estimated from its text, then placed. python-pptx can't measure rendered text, so I just approximate the display width as full-width characters = 1.0 and half-width = 0.5: short lines get a one-line width, longer ones wrap at a maximum width. Pretty crude, but it works.

def measure_bubble(text):
    char_w = BUBBLE_FONT_PT / 72.0           # one full-width char ≈ 0.208in
    cap = int((MAX_BUBBLE_W - PAD*2) / (char_w * 1.06))  # full-width chars per line
    ...
    if longest <= cap and "\n" not in text:
        width = longest * char_w + PAD*2 + 0.18   # slack so it doesn't wrap
        lines = 1
    else:
        width = MAX_BUBBLE_W
        lines = ceil(longest / cap)
    height = lines * LINE_H + PAD_Y*2 + 0.06
    return width, height, lines

At first I didn't add the slack (that trailing + 0.18) and used the exact width. The result: a short phrase like おつかれさま！ ("hey, nice work today!") would wrap onto two lines and look weirdly stretched. The actual rendered character width comes out a bit wider than my estimate, so giving it a little breathing room settled things down.

For the arrangement I copied real LINE and made "bottom-aligned" the default (newest message sits right above the input bar). I just sum up the height of the whole conversation first and set the start position to CHAT_BOTTOM - block_h.

The main event: no animation API, so write the XML yourself

This is where I got stuck the longest. python-pptx can create shapes and text just fine, but it gives you no way to touch animations (entrance effects and the like) through its API. So what do you do? Animations are stored as an OOXML element called <p:timing>, so I build that element myself and append it to the end of an already-generated slide.

PowerPoint animations are a tree of time nodes. Under mainSeq (the sequence that advances on click), you hang a <p:par> for each effect. What I wanted was "on click, bubbles appear one after another at 0.8s intervals," so:

The first effect is nodeType="clickEffect" (fires on click)
Every effect after that is nodeType="withEffect" with a delay, staggered in time

There's also an "After Previous" option, but it works by waiting for the previous effect to finish, and the behavior felt flaky. So I leaned on absolute offsets of i × 800ms instead. That turned out to be both more predictable and, honestly, less work.

The effect on each bubble is "Float In" — it fades in while drifting up slightly from below. I use <p:set> to make it visible, <p:anim> to move ppt_y (vertical position) from a touch below up to its real spot, and <p:animEffect filter="fade"> to fade it in, all running together.

def _effect_par(eid, spid, delay, node_type, grp):
    return f'''<p:par ...>
  <p:cTn id="{eid}" presetID="42" presetClass="entr" presetSubtype="8"
         fill="hold" grpId="{grp}" nodeType="{node_type}">
    <p:stCondLst><p:cond delay="{delay}"/></p:stCondLst>
    <p:childTnLst>
      <p:set> ... style.visibility=visible ... </p:set>
      <p:anim calcmode="lin" valueType="num">
        <p:cBhvr additive="base">
          <p:cTn id="{eid+2}" dur="{EFFECT_MS}" fill="hold"/>
          <p:tgtEl><p:spTgt spid="{spid}"/></p:tgtEl>
          <p:attrNameLst><p:attrName>ppt_y</p:attrName></p:attrNameLst>
        </p:cBhvr>
        <p:tavLst>
          <p:tav tm="0"><p:val><p:strVal val="ppt_y+0.04"/></p:val></p:tav>
          <p:tav tm="100000"><p:val><p:strVal val="ppt_y"/></p:val></p:tav>
        </p:tavLst>
      </p:anim>
      <p:animEffect transition="in" filter="fade"> ... </p:animEffect>
    </p:childTnLst>
  </p:cTn>
</p:par>'''

spid is shape.shape_id. For an incoming message I wanted the bubble and the avatar to appear together, so I give them the same grpId and the same delay. Once the <p:timing> string is assembled, I parse it with lxml and just append it to the slide.

timing = etree.fromstring(timing_xml.encode("utf-8"))
slide._element.append(timing)

A nice bonus: once a shape has an entrance effect, PowerPoint handles "keep it hidden until it fires" on its own. So I didn't have to write any code to hide the initial state myself.

One more note on checking my work. I was eyeballing layout by exporting to PDF with LibreOffice — but LibreOffice's PDF export only renders the final state of an animation. It's perfectly fine for catching layout problems, but to verify the motion itself I had to fall back to running an actual slideshow in PowerPoint/Keynote.

Turning it into a web app

Because I'd already factored the generation into a single build_pptx(data) -> bytes function, the web side only needed to POST the conversation as JSON. I wrote the serverless function with the standard-library BaseHTTPRequestHandler and return the bytes from build_pptx directly.

class handler(BaseHTTPRequestHandler):
    def do_POST(self):
        data = json.loads(self.rfile.read(length) or b"{}")
        pptx = build_pptx(data)
        self.send_response(200)
        self.send_header("Content-Type",
            "application/vnd.openxmlformats-officedocument.presentationml.presentation")
        self.send_header("Content-Disposition", 'attachment; filename="line_chat.pptx"')
        self.end_headers()
        self.wfile.write(pptx)

Input is just one text area

I didn't get clever with the input UI — it's a single text area. The only rules are "L: is the other person, R: is you, a blank line starts a new slide." Keeping it this loose is actually what makes it pleasant: you just dump in whatever conversation pops into your head.

L: Hey, nice work today!
L: There's something I want to run by you
R: What's up?

L: This thing's been eating up my time lately
R: Oh, I totally get that

Parsing is a regex applied line by line. Any line without an L:/R: prefix gets joined onto the previous bubble with a newline (i.e. multi-line bubbles).

const m = /^([LRlr])[:：]\s?(.*)$/.exec(raw);
if (m) { last = {side: m[1].toUpperCase(), text: m[2]}; slide.push(last); }
else if (last) { last.text += "\n" + raw.trim(); }   // multi-line bubble

A live preview in the browser

Having to download the file just to see the result is a miserable loop, so I rebuilt the same look in HTML/CSS and added a preview-and-play feature right in the browser. The colors and the bottom-alignment match the generated pptx, and for playback I just add CSS transitions (opacity and translateY) one after another with setTimeout. It ends up feeling about the same as the animation inside the pptx.

Two things that bit me on deploy

A requirements.txt makes Vercel think it's a "Python app"

My very first vercel --prod died immediately with this:

The pattern "api/generate.py" defined in `functions`
doesn't match any Serverless Functions inside the `api` directory.

The cause was the requirements.txt sitting at the project root. With that present, Vercel decides the whole project is a "Python app" and stops picking up api/generate.py as an individual function. I fixed it by spelling out builds in vercel.json, telling Vercel to build the static HTML and the Python function separately.

{
  "version": 2,
  "builds": [
    { "src": "index.html",      "use": "@vercel/static" },
    { "src": "api/generate.py", "use": "@vercel/python" }
  ],
  "routes": [
    { "src": "/api/generate", "dest": "/api/generate.py" },
    { "src": "/",             "dest": "/index.html" }
  ]
}

After that, python-pptx installed itself from requirements.txt, and both the function and the static serving worked exactly as I'd hoped.

You can't protect production on the free plan

Once it was live, I wanted to put it behind some access control, so I tried enabling ssoProtection (Vercel Authentication) via Vercel's REST API. This came back:

{ "code": "invalid_sso_protection",
  "message": "Vercel Authentication is not available on your plan for production deployments" }

Turns out the Hobby (free) plan doesn't let you put Vercel Authentication / Password Protection on production deployments (you need Pro). Preview deployments can be protected on the free plan. If you want to keep production hidden while staying free, you're looking at rolling your own gate — Edge Middleware, or Basic auth inside the Python function. For now I left it open.

Wrapping up

After doing all this, the part I'm most glad I pushed through was writing the animation XML by hand. I'd nearly written off animations as impossible with python-pptx, but it turns out that hand-assembling a <p:timing> element and appending it is all it takes to get bubbles floating in, one by one, on a real PowerPoint. And the dead-simple withEffect + absolute delay approach was more than enough.

Factoring the generation into a single function also paid off more than I expected — moving from a CLI to a web app was basically copy-paste.

Being able to type out a conversation and get an animated chat slide back made prepping for talks noticeably less of a chore. The code is up on GitHub if you want to poke around — happy to hear what you'd build with it.

DEV Community: Yuuki Yamashita

The day I gave an AI a wallet — building an approval-gated shopping agent with Sonnet 4.6, AgentCore Payments, Rakuten and Stripe

Why I built this

Architecture

The agent itself

Phase 1 — the AgentCore Payments signer trap

Phase 2 — Rakuten and the Referer pitfall

DynamoDB and the Vercel frontend

8-language UI and LINE Seed JP

Things I learned

Links

Porting Apex Voice to Windows broke at every single layer (faster-whisper + pystray + pywin32)

Where I started

The stack I picked

Pain #1: sounddevice doesn't load on ARM64

Pain #2: zero output on startup

Pain #3: pynput can't catch F19

Pain #4: PowerToys ate Ctrl+V

Pain #5: Tray clicks steal focus

Pain #6: Bedrock — MissingDependencyException

Pain #7: Caps Lock doesn't reach the VM in UTM

Auto-launch on login

End state and performance

What I learned

What's next

Apex Voice v2 — Killing Hallucinations, Cutting Latency, and Landing on Caps Lock

Where I Left Off

1. Killing the Hallucinations (the layered way)

2. Cutting the Latency

3. Caps Lock as the One-Key Hotkey

4. AgentCore Browser for Real Web Search

5. AgentCore Payments (x402) — Letting the Agent Pay

The New Architecture

What I Learned (v2 Edition)

What's Next

Building a chat-based Marp slide generator with Next.js, Amazon Bedrock, and Lambda

Overview

Architecture

1. Multi-turn chat with Bedrock Converse

Message shape

System prompt: reply first, then Markdown

2. Making preview and export look the same

One source of truth for theme CSS

Font override via a <style> block in the Markdown

3. User-defined themes (localStorage)

4. Lambda container: Chromium + LibreOffice in one box

Chromium dies with Connection closed

marp-cli hangs waiting on stdin

M PLUS Rounded 1c font family name mismatch

5. Multi-stage build: 3.18 GB → 2.39 GB

6. Keyless AWS from Vercel via OIDC Federation

Wrap-up

Voice Typing Anywhere on macOS — I Built Apex Voice with mlx-whisper, Amazon Bedrock, and Strands Agents

TL;DR

Why I Built It

What It Does

Architecture

The AWS Side

The Hard Part Wasn't AI. It Was Packaging.

What I Learned

What's Next

Try It

I gave my AI agent a boss: a human-approval gate in Slack, over MCP

The shape of the idea

The agent side

The approval card

The feature I didn't expect to love: Edit & Approve

Not everything needs a human

Context, so it's a decision and not a rubber stamp

How it all fits together

What I'd tell past me

I built a web app that overlays Niconico-style scrolling comments onto your slides (PowerPoint-ready, keyless OIDC)

What it does

First hurdle: how to animate inside PowerPoint

The fix: overlay a transparent animated GIF

Architecture

Keyless with Vercel OIDC Federation

A note on rights

Takeaways

Building Fridge AI Health: An AI-Native Full-Stack App with Vercel and Amazon Aurora

Font override via a `<style>` block in the Markdown

Chromium dies with `Connection closed`