DEV Community: Yu-Wei Simon Liu (Simon Liu)

open-doc: Letting Antigravity and Other Coding Agents Fully Own Document Layout and Generation

Yu-Wei Simon Liu (Simon Liu) — Wed, 19 Aug 2026 03:28:17 +0000

GitHub — open-doc
https://github.com/simonliu-ai-product/open-doc

1. Introduction

Over the past year, I think most of us have handed more and more work over to coding agents — writing code, looking things up, running tests. They do all of that pretty well. But there is one thing I never got right: asking an agent to produce a report I could hand off as-is.

Agents are actually good writers. Ask one for a quarterly review, a technical evaluation, or a project proposal and the content quality is fine. The problem starts right after the words — the layout. Every approach I tried got stuck in the same place:

Have the agent write Markdown, then convert to PDF. The output has no concept of a "page". Tables get sliced in half across a page break, captions get separated from their figures, the table-of-contents page numbers don't line up. You end up tuning CSS instead of reading content.
Have the agent write HTML, then print it. Every document reinvents the page layout from scratch. Where are the A4 margins, when should it break, how do page numbers carry across — you re-derive all of it every time, and then the next document starts over.
Have the agent produce Word. No need to elaborate here. The odds of the formatting falling apart are roughly 100%.

After thinking about it long enough, you realize the problem isn't that the agent isn't smart enough — it's that the division of labor is wrong. What agents are genuinely good at is content, and layout happens to be the half that isn't allowed to be wrong. A report's page size, margins, break positions, and consecutive page numbers leave no room for creativity: they're either right or wrong. Handing those to an agent that has to guess afresh every time was never a reasonable idea.

So the sensible arrangement is to let the framework lock down the parts that can't be wrong, and let the agent handle only what it's actually good at. The project that made me see this clearly was somebody else's.

2. A Look at open-slide

To be clear up front: open-slide is @1weiho's work, not mine. I'm a user.

GitHub - 1weiho/open-slide: A slide framework built for agents.
https://github.com/1weiho/open-slide

It bills itself as "a slide framework built for agents". Every slide is a fixed 1920 × 1080 canvas written as a React component, and the framework takes care of scaling, navigation, hot reload, presenter mode, and speaker view. You don't have to learn a restrictive DSL, because the page is a component — want a chart, drop in a chart; want an animation, write an animation.

npx @open-slide/cli init my-slide

I heard about the project at COSCUP and went home to try it. In practice, the feeling is: as long as you can describe the slide you want in plain language, the agent writes React and the result shows up in your browser immediately.

But what actually made me stop and dig in wasn't "writing slides in React" — that isn't new. What I found interesting was how it handles the question of how the agent is supposed to know how to use this tool.

When you init an open-slide project, the folder contains more than code: there's an AGENTS.md, plus a handful of skill documents under .agents/skills/. When your coding agent opens the project, it already knows what the file contract looks like, how much content safely fits on one slide, and what it shouldn't touch.

In other words, the manual travels with the project, not with the conversation. That design became the starting point for all of open-doc.

3. What open-slide Taught Me — Building Tools, and Making Coding Agents Understand How to Use Them

I put open-slide through its paces and read the source. This section is about what it taught me regarding the difference between tools built for agents and tools built for people.

I. The manual belongs in the repo, not in the prompt

This is the most counterintuitive point and the one with the biggest impact. We're used to writing "how to use this tool" as a prompt pasted at the top of a conversation. The problem is that prompts go stale, get truncated, and never get updated — and a different person or a different agent means pasting it all over again.

open-slide's approach is to turn that knowledge into files in the repo. A skill is just a Markdown file that says "when you write this kind of file, here's what you must follow", and the scaffolder generates it into the user's project. Three benefits fall out of this: it's versioned (the framework changes, the skill changes with it), it's project-scoped (anyone who opens the repo can see it), and it's reviewable (a skill is part of the source, so it goes through PRs).

II. Lock the parts that can't be wrong into the framework

open-slide's canvas is always 1920 × 1080. That isn't a limitation — that is the product. Because the canvas is fixed, the agent never has to guess how big the slide is, how large the text should be, or whether the content will fit. It just writes content, and "does it fit" gets answered by the framework through measurement, which is vastly more reliable than an agent guessing.

I'd put it even more bluntly: every choice you take away from the agent is one class of error you no longer have to verify. Layout is right-or-wrong with no creative latitude — it was never something to leave to guesswork.

III. The agent needs to know where you're currently looking

This one I only appreciated after using it for real. You're looking at slide 7 in your browser, you turn to the agent and say "the spacing on this page is too tight" — which page is "this page"? The agent doesn't know. It can only ask you back, or guess, and a wrong guess means it edits something else entirely.

open-slide's fix is direct: on every navigation, the dev server writes "where the user is right now" into a file, paired with a skill that tells the agent to read it. Deictic references like "this page" or "this element" then resolve into a concrete file path and line number. It looks like a small thing, but it's the bridge between the screen the human is looking at and the file the agent is editing.

Those three lessons came straight from open-slide. But once I actually built a system of my own, I hit two more problems it hadn't shown me — humans and agents editing the same file at the same time, and the same operation having more than one entry point. I'll cover both in the next section.

4. My Open Source Project: open-doc

Carrying those three lessons, I spent the last few days building open-doc — and the two problems I just teased are exactly what I ran into along the way.

If open-slide is Google Slides for agents, then open-doc is Google Docs. Same concept, different medium: a deck is a 1920 × 1080 canvas, while a document is a stack of A4 sheets that has to survive contact with a printer.

npx @open-document/cli init my-docs
cd my-docs
pnpm dev

(Side note: the @open-doc scope on npm was already taken, so the packages are `@open-document/` — but the CLI command and the project name are still open-doc.)*

open-doc's document viewer — page thumbnails on the left, a real A4 sheet in the middle, footer and page number filled in by the framework

Real page dimensions

Every page component renders as an actual sheet of paper. A4 (794 × 1123 px @96dpi), Letter, A5, and Legal are all supported, in portrait or landscape. What you see on screen is what's in the PDF, because the @page size matches — nothing gets re-scaled at print time.

This is something I called out explicitly when writing the skills: authors write CSS px, but paper is measured in mm. 1pt is roughly 1.333px, so 14px body text prints at about 10.5pt — while 11px, which looks okay on screen, prints at 8pt and is unreadable.

The file contract

A document is a folder plus an index.tsx:

// docs/q3-review/index.tsx
import type { DocMeta, DocPage } from '@open-document/core';

const Cover: DocPage = () => <div>…</div>;
const Summary: DocPage = () => <div>…</div>;

export const meta: DocMeta = {
  title: 'Q3 Review',
  pageSize: 'A4',
};

export default [Cover, Summary] satisfies DocPage[];

No front-matter DSL, no hierarchy of config files. One component is one printed sheet.

Auto-pagination that knows what can't be split

Fixed pages are right for covers, tables of contents, and section dividers — pages where the layout is the content. Body text is the opposite: paginating it by hand usually yields eleven pages that are each 60% full. So body content can be wrapped in flow(), and the framework measures every block in the real DOM before packing it into pages: headings don't get stranded at the bottom, captions stay with their figures, tables move as a unit. That pagination logic is a pure function, so it has unit tests. Pagination rules should be verifiable, not folklore.

A table of contents and page numbers that maintain themselves

Just write real <h1> / <h2> and you get an outline sidebar automatically. Drop in <TableOfContents /> and the TOC page fills itself in, with page numbers that are correct in both the viewer and the exported file. Footers use useDocPageNumber() / useDocPageCount(), so nothing has to be renumbered by hand.

Edit right on the page, or leave a note for the agent

Inspect mode lets you click an element on the page and edit its text directly; the change is written back to source via AST replacement. But the feature I use more often is the other one: you can leave a note for the agent right on the page. It's stored in the source as an @doc-comment marker, and later you tell the agent "apply comments" — the skill walks through them one by one, makes the edits, and clears the markers. This is far more precise than screenshotting a page and saying "fix this bit", because the annotation is anchored to a source line number.

Inspect mode — click any element to rewrite its text, or leave a note for the agent

Humans and agents will edit the same file

This is one I only discovered by building it, and one traditional tools mostly don't have to face. While you're clicking around in the browser editing text, the agent is editing the same source. Whoever writes last clobbers the other — silently, with no warning at all.

My fix is that every write API takes an expected parameter. You send along "what I just read", and if what's on disk no longer matches, the write is rejected (409) rather than applied. Once the agent gets a 409, it re-reads and re-decides — which it's perfectly capable of doing, as long as you give it the chance.

Export and deploy

PDF goes through the browser's print pipeline at real page dimensions, and waits for fonts and images to load and the TOC to be filled before serializing — so you don't get the classic "chart is blank after export" problem. HTML export is self-contained and printable (documents with assets are bundled into a zip). open-doc build also produces a fully static site you can drop onto Vercel, Cloudflare Pages, or any static host.

An MCP server for any agent framework

pnpm add -D @open-document/mcp
open-doc dev --mcp
# ➜ MCP:   http://localhost:5273/mcp
# ➜ Local: http://localhost:5273/

There are 19 tools in total, covering document CRUD, precise single-paragraph text replacement, themes, assets, and folders. It's stateless Streamable HTTP — clients just call it, no session handshake required.

Which brings me to the second thing I ran into while building this: the same operation has more than one entry point. open-doc can be driven from the browser UI or from MCP, and I nearly ended up writing two sets of logic. I eventually pulled every operation that touches disk into src/ops/, leaving the dev server's routes and the MCP tools as thin adapters over it.

The payoff is that rules are written once — the 409 conflict check, path safety, id validation are all identical — so you never get a gap where "the UI blocks it but MCP doesn't". That matters especially in systems with agents in them, because the path an agent takes is often the one you manually test least.

Built-in skills

The project you init ships with these skill documents, placed in both .agents/skills/ and .claude/skills/ so either flavor of agent can see them:

Skill	What it does
`create-doc`	Draft a document from scratch: establish the topic, the audience and the source material first, ask a few scoping questions, plan the pages, and only then start writing
`doc-authoring`	Technical reference: the file contract, the page canvas, print-safe type sizes, the vertical budget that decides where pages break, tables, charts, assets
`create-theme`	Generate a reusable family style, including a palette, a type scale, and components you can drop straight in
`apply-comments`	Walk through the notes you left in Inspect mode, complete each one, and clear the markers
`current-doc`	Resolve "this page" and "this element" by reading the cursor file the dev server writes

5. Demo: Driving open-doc with Antigravity

Everything above is design. Now let's actually run it. I'm using Google Antigravity as the coding agent environment here, for a simple reason: open-doc's interface to agents is a generic convention (AGENTS.md plus .agents/skills/, with MCP as an option), so it isn't tied to any one vendor's agent. The flow is the same with other tools.

If you want background on Antigravity, here's something I wrote earlier — this article won't cover any AI-coding details, just the content and the results:

Experiencing AI Building a Personal Website Directly with the Google Antigravity IDE
https://medium.com/@simon3458

Step 1: Create a workspace

Run the following to scaffold the folder:

npx @open-document/cli init q3-report
cd q3-report
pnpm install
pnpm dev

The open-doc workspace created by the CLI

Open http://localhost:5273 and you'll see an empty workspace plus a getting-started document.

Step 2: Open the folder in Antigravity

The first thing Antigravity does after opening the project is read AGENTS.md, which sits in the root of the scaffolded project. It says that only docs/<id>/ may be written to, that no new dependencies should be added, and lists the available skills and what each is for. In other words, you don't have to explain what open-doc is to the agent — the project explains itself.

Antigravity reading the project's AGENTS.md and skills

Step 3: Ask for a report in one sentence

My prompt was roughly:

Write me a Q3 infrastructure review report for an audience of engineering leads.
The data is in ~/workspace/data/q3-metrics.csv — don't make up any numbers.

The create-doc skill first pins down where the material comes from. That behavior is deliberately written into the skill: when no data is provided it should ask, not invent a set of plausible-looking numbers. Then it asks a few scoping questions (page limit, TOC or not, tone), plans out the pages, and only then starts writing files. The browser hot-reloads the moment it's done, and you're looking at that stack of A4 pages directly.

Antigravity generating the document

Step 4: Leave a note on the page and let the agent come back to it

This is the loop I use most. Rather than describing "the paragraph under the subheading on page three" in a chat box, just open Inspect, click that paragraph, leave a note, then go back to Antigravity and say "apply comments".

Leaving a note in Inspect mode, and the result after the agent applies it

Step 5: Export the PDF

Pick PDF from the Download menu in the top right, and out comes a real A4 file with consecutive page numbers and correct TOC page references — no round trip through Word to fix things up.

The exported PDF

Optional: hook up MCP

If you'd rather have another agent framework (Google ADK, for instance) drive it through tool calls than edit files directly, add --mcp to start the MCP service at http://localhost:5273/mcp. The agent can then use tools like list_documents, read_document, and write_text, and the page in the browser updates live, because the MCP server runs on the dev server itself.

6. Conclusion

The biggest thing I got out of building open-doc wasn't the tool itself — it was getting clear on one idea: the hard part of building tools for agents isn't the features, it's making the agent know how to use them.

And "making the agent know how to use them" isn't about writing a longer prompt. It's three concrete pieces of engineering work: put the knowledge in the repo so it's versioned, project-scoped, and reviewable; lock the parts that can't be wrong into the framework so the agent only has to handle what it's actually good at; and give the agent the context it's missing, like which page you're currently looking at, or whether this file has been changed by someone else since it last read it.

None of these three have much to do with how smart the LLM is. They're design problems — and I expect more and more tools will have to face the same ones.

Finally, thanks again to @1weiho for open-slide. Virtual-module-based document discovery, the scaffolder, and treating skills as documentation were all learned from there. Good design gets picked up and carried forward by the next person.

open-doc is at v0.2.0, MIT licensed. Try it, open issues, send PRs — and I'd love to hear what kind of documents you end up producing with it.

GitHub — open-doc
https://github.com/simonliu-ai-product/open-doc

I am Simon

Hi, I'm Simon Liu, an AI solutions specialist and a Google Cloud AI Google Developer Expert (GDE). I hope to help enterprises solve problems by adopting AI technologies. If this article was useful to you, please give it a reaction and follow me so you can catch what I write next. Feel free to leave a comment on my LinkedIn and discuss AI topics with me — I hope this was helpful!

My Personal Website: https://simonliuyuwei.my.canva.site/link-in-bio

This article was originally published in Traditional Chinese on Medium.

[Open Source] ADEval — A Tool for Evaluating Tool-Use Capabilities of Google ADK AI Agents

Yu-Wei Simon Liu (Simon Liu) — Tue, 10 Mar 2026 04:58:31 +0000

I. Introduction: Why Building a "Stable" AI Agent is Hard

As developers, we all know that scaffolding an AI Agent using Google’s Agent Development Kit (ADK) or various LLM frameworks is relatively straightforward. The real challenge, however, lies in ensuring the Agent's behavior is predictable and stable.

In real-world business scenarios, an Agent might call the right tool today but deviate tomorrow due to slight prompt variations, model updates, or context interference. Relying solely on manual chat testing is inefficient and fails to cover critical edge cases.

This is why I built ADEval — a systematic evaluation tool designed specifically for AI Agents. It empowers developers to gain deep control over Agent behavior through a dual-track approach: Automation and Visualization.

II. Github: ADEval

ADEval provides an intuitive Web UI and a powerful CLI, allowing you to systematically test your Agent's Question-Tools-Answer (Q-Tools-A) flow. It supports experiment management, batch testing, and comprehensive tracing.

Github Repository GitHub - ap-mic-inc/ADEval: Google ADK Evaluation Service
Project Documentation ADEval Documentation

III. Core Philosophy: The Q-Tools-A Validation Framework

When evaluating an AI Agent, comparing the final text response (Answer) is simply not enough. A high-quality Agent must call the right Tools at the right time with the correct parameters.

ADEval is designed around the Q-Tools-A logic:

Question: The input prompt, specific User ID, and necessary Session State (persistence).
Tools: Automatically validates if the Agent invoked the expected tools. We support Smart Argument Comparison — even if the JSON parameter order differs, ADEval marks it as a match if the values and logic are consistent.
Answer: Ensures the final response meets business requirements via keyword matching or semantic checks.

Below is the workflow logic diagram I designed for ADEval, ensuring a clear path from start to output:

ADEval Workflow Logic Diagram (Mermaid Chart)

IV. Dual Mode: From Debugging to Production Automation

ADEval provides both an intuitive Web UI and a powerful CLI, allowing you to switch seamlessly between manual debugging and CI/CD automation.

1. Web UI: Visual Tracing & Real-time Debugging

Observing an Agent's thought process is critical. The ADEval Web dashboard features a powerful "Playground" where you can input questions and observe results in real-time.

ADEval Playground Interface showing testing panels

The standout feature is Visual Tracing. We transform complex API response event streams into a "Dark Terminal Style" viewer. You can expand raw JSON with one click to pinpoint exactly where a tool call failed or deviated.

Visual Tracing Dark Terminal View showing JSON events

2. CLI Tool: Powerhouse for CI/CD & Batch Execution

Once your experiments are defined, you no longer need a browser. ADEval offers a full-fledged command-line tool perfect for automation scripts:

Global Configuration (adeval config): Set default API URLs and developer credentials to save time.
Quick Test (adeval test): Perform stress tests or logic validation directly against the Agent without creating an experiment.
Batch Execution & Reporting (adeval run / export): Execute entire experiment sets and receive precise statistical reports.

CLI output showing 'adeval run' statistical table

V. Deep Dive into Features

Experiment Management & Batch Evaluation: Import dozens or hundreds of test cases via CSV files. ADEval provides real-time progress bars and pass-rate statistics during execution.
Local Data Ownership: Privacy and performance are priorities. All experiment data, logs, and configs are stored locally in the .adeval/ folder. Your data stays on your machine.
Smart Comparison Logic: In tool validation, we support "Order-Independent" comparison. If an Agent calls get_weather(city="Taipei", unit="c"), ADEval accurately judges it even if the internal parameter sequence differs.

VI. Getting Started

ADEval is open-source and ready to use. You can install it easily via Python:

# Clone the repository
git clone https://github.com/ap-mic-inc/ADEval.git
cd ADEval

# Install in editable mode
pip install -e .

After installation, simply run adeval ui to launch the web interface or check out adeval --help for the full CLI suite.

VII. Conclusion

Building an AI Agent that can chat naturally is just the beginning. Ensuring its stability and predictability in complex business scenarios is where the real engineering challenge lies. ADEval was born to solve the core pain points of testing, debugging, and maintaining Agent systems.

In summary, ADEval brings three key values to developers:

Precise Behavioral Control: Move beyond text matching to rigorous Tool-use validation.
Flexible Workflow: Covers the entire lifecycle from visual debugging to automated regression testing.
Security & Efficiency: Localized storage for privacy and smart matching to reduce false positives.

If you are building high-quality enterprise AI applications with Google ADK, ADEval is your indispensable testing companion. Give us a Star (⭐) on GitHub or submit an Issue/PR to help us build a stronger AI evaluation ecosystem!

I am Simon
Hi everyone, I am Simon Liu, an AI Solutions Expert and Google Developer Expert (GDE) in GenAI. I am dedicated to helping enterprises implement AI technologies to solve real-world problems. If this post was helpful, please follow me on Medium or connect with me on LinkedIn.

My Personal Website: https://simonliuyuwei.my.canva.site/link-in-bio

[ Open Source Project] open-translate — Offline Translation Web Service Powered by TranslateGemma

Yu-Wei Simon Liu (Simon Liu) — Sun, 18 Jan 2026 16:26:31 +0000

I’ve provided a Google Colab test script for this project. You can apply for Hugging Face and ngrok tokens to test it. Welcome to use it!

On January 15, 2026, Google released a new model named TranslateGemma. It can perform translation tasks for 55 languages, including Traditional Chinese and English, based on its specific training. It even supports image input for translation output.

I wondered if it was possible to create an offline "Google Translate-like" web service so that corporate confidential data could be processed in a non-networked environment. Thus, this project was born. This article will introduce TranslateGemma and my personal project.

I. Introduction to TranslateGemma

1. Background & Overview

TranslateGemma is a specialized LLM for translation tasks developed by Google DeepMind, built on the Gemma 3 architecture. It aims to provide the strongest translation capabilities in the open-source community. Unlike general chatbots, TranslateGemma focuses on language conversion. Weights are publicly available on Hugging Face and Vertex AI for local or cloud deployment.

2. Architecture & Training

Its core advantage comes from a unique "two-stage training" process:

Supervised Fine-Tuning (SFT): Utilizing high-quality human translation data and synthetic data generated by Gemini.
Reinforcement Learning (RL): Further guided by translation reward models like MetricX-QE and AutoMQM to align with human preferences and semantic precision.
Model Sizes: Available in 4B (mobile), 12B (laptop/workstation), and 27B (cloud).

3. Key Features

Supported Languages: 55 core languages with training on nearly 500 language pairs. > PS: According to the tech report, Traditional Chinese-related pairs include EN/Cantonese -> Trad. Chinese and Trad. Chinese -> Cantonese.
Multimodal Potential: Inherits Gemma 3's vision capabilities, enabling "visual translation" to understand text context in images (signs, menus, etc.).
High Efficiency: The 12B version often outperforms larger unspecialized models in translation quality. > Reminder: Max Context Window is 2k.

4. Performance & Applications

Excelled in authority benchmarks like WMT24++.

The 12B version's quality (MetricX) even surpasses the general Gemma 3 27B model, proving the effectiveness of specialized training.
Its flexibility makes it the best choice for balancing lightweight design and high quality in the open-source community.

5. Information

Google Blog: https://blog.google/innovation-and-ai/technology/developers-tools/translategemma/
Huggingface: https://huggingface.co/collections/google/translategemma
Tech Report: https://arxiv.org/pdf/2601.09012

II. Open Translate — Modern Interface for TranslateGemma

Open Translate was created to bridge the gap in localization and privacy.

1. Technology Stack

(1) Backend: FastAPI for high-performance async API. Uses Hugging Face transformers to call the translategemma-4b-it model with CUDA acceleration.
(2) Frontend: React (Vite) + Bootstrap for a clean, modern UI with real-time previews.
(3) Database: SQLite (SQLAlchemy) for translation history logs.

2. Highlights & Localization

Multimodal Support: Image translation interface for screenshots, signs, or documents.
Trad. Chinese Optimization: Integrated OpenCC to convert outputs into Taiwan-style phrasing and Traditional Chinese characters.
Privacy & Security: Supports full offline deployment via Docker to ensure data remains internal.

3. Quick Start

Google Colab: One-click script with Node.js/Python setup and ngrok access. https://colab.research.google.com/github/simonliu-ai-product/open-translate/blob/main/open_translate_project_workflow.ipynb
Docker Compose: Single command to run locally on NVIDIA GPUs.

3. Github

Link: https://github.com/simonliu-ai-product/open-translate/tree/main

III. Conclusion

The release of TranslateGemma proves that specialized small models can punch above their weight.

Democratizing Compute: 4B models provide professional quality on home laptops, lowering the barrier to entry.
Multimodal is Future: Translation moves beyond text to direct visual understanding, changing how we interact with the world.
Open Source Value: Combining Google's models with modern web frameworks enables rapid problem-solving.

Open Translate is just a starting point. I will continue to optimize localization and explore integration with AI Agents. Welcome to download the source code on GitHub, give it a Star, or test it via Colab!

I am Simon

Hi everyone, I am Simon Liu, an AI Solutions Expert and a Google Developer Expert (GDE) in GenAI. I look forward to helping enterprises implement AI technologies. If this article was helpful, please give it a "Clap" on Medium and follow my account. Feel free to leave comments on my LinkedIn!

My Personal Website: https://simonliuyuwei.my.canva.site/link-in-bio

[AI Agent] TPU-Based AI Agent Development: Integrating the Twinkle AI Open Source Model (gemma-3–4B-T1-it) with Google ADK Tools

Yu-Wei Simon Liu (Simon Liu) — Mon, 12 Jan 2026 02:44:31 +0000

Special Thanks: I would like to express my gratitude to APMIC and the Twinkle AI community for their assistance, which made the completion of this article possible.

Original Chinese Post: https://medium.com/@simon3458/twinkleai-gemma-3-t1-4b-adk-agent-d3309665f448

As Large Language Model (LLM) technology enters a stage of maturity, the focus of developers has shifted from simple "conversation generation" to "AI Agents" capable of autonomous planning and execution. However, creating an Agent that understands Taiwan's local culture and can accurately execute complex tool calls presents two major challenges: first, general-purpose models often lack understanding of local regulations and context; second, the high cost of GPU computing power limits the widespread adoption of applications.

This article will guide you through exploring: building AI Agent application services by combining the matrix computation advantages of Google TPU, Twinkle AI's gemma-3–4B-T1-it open-source model (optimized specifically for the Taiwanese context), and the Google ADK (Agent Development Kit).

We will start with the underlying architecture of TPUs to explain why they are accelerators for AI inference. Next, we will introduce how Twinkle AI solves "alignment drift" and strengthens Function Calling capabilities. Finally, we will conduct a hands-on walkthrough using Google Colab, stacking and integrating these technologies to build an AI Agent from scratch that is responsive, understands Taiwanese linguistic habits, and can actually query stock information.

I. Introduction to Google TPU

1. What is a TPU (Tensor Processing Unit)?

Google TPU is a "Domain-Specific Architecture" (DSA) integrated circuit tailored specifically for machine learning workloads. Unlike traditional processors that need to handle various general-purpose tasks, the core design of the TPU adopts a "Systolic Array" architecture. This design mimics the way a heart beats, allowing data to flow rhythmically between thousands of arithmetic units within the chip. This architecture enables the TPU to significantly reduce frequent memory access when performing matrix multiplication—the core operation of neural networks—thereby breaking through the "von Neumann bottleneck" of traditional computer architectures and achieving extremely high computational density and efficiency.

2. The Difference Between TPU and GPU

The fundamental difference between the two lies in the philosophical opposition of "Specialization" vs. "Generalization."

GPU (Graphics Processing Unit): essentially a general-purpose parallel processor designed for graphics rendering. It retains a massive amount of control logic and cache to handle complex instruction streams, giving it high flexibility and a powerful CUDA software ecosystem suitable for highly variable research and applications.
TPU: sacrifices generality (it cannot efficiently handle non-matrix operations) and removes hardware units irrelevant to AI, dedicating all released resources to matrix operation units. This gives TPUs higher computational efficiency when processing large-scale static computation graphs in specific formats (such as bfloat16). However, the development barrier is higher than GPUs when dealing with dynamic control flows or custom operators, usually relying on the XLA compiler and JAX framework for optimization.

II. Introduction to the Twinkle AI gemma-3–4B-T1-it Model

(Image Source: Huggingface)

gemma-3-4B-T1-it is a 4B parameter model launched by Twinkle AI based on the Google Gemma 3 architecture. It aims to solve the "alignment drift" problem caused by uneven data in mainstream foundation models and to practice the concept of "Sovereign AI."

The model has been deeply optimized for the Taiwanese context, correcting:

Vocabulary Misuse/Appropriation: (e.g., distinguishing between terms for "quality" and "mass").
Legal and Institutional Hallucinations: (Citing current laws of the Republic of China rather than laws of the PRC).
Cultural Meme Disconnects: (Understanding internet slang from communities like PTT and Dcard).

Through Gemma 3's "Local-Global Hybrid Attention Mechanism" and a 128K token context window, T1–4B-it achieves deep cultural alignment at a lightweight scale, positioning itself as a language model focused on Agent workflows and local needs.

regarding dataset selection and ecosystem collaboration, T1–4B-it adopts a rigorous data strategy. Training data includes lianghsun/tw-reasoning-instruct (reasoning instructions) designed for the Taiwan context, nvidia/Nemotron (instruction following), lianghsun/tw-contract-review-chat (contract review), and Chain of Thought (CoT) data prepared by Kerg (such as tw_mm_R1). We thank APMIC for providing critical computing support for the infrastructure to make this possible.

For architecture designed to strengthen Function Calling, T1–4B-it specifically introduces the Hermes Tool-call Parser format for training, equipping it with powerful Agent capabilities. The model can handle four levels of complex calling scenarios: single function, multiple functions, parallel functions, and parallel multiple functions. In the BFCL evaluation, the model achieved an overall accuracy of 84.5%, with performance on multiple Abstract Syntax Trees (AST) reaching as high as 89%. This demonstrates that at the 4B parameter magnitude, it possesses tool usage and automated execution capabilities surpassing many 7B or 13B models.

Example: Using Google ADK to handle parallel function processing simultaneously.

For detailed information, please visit HuggingFace:

twinkle-ai/gemma-3-4B-T1-it

III. Hands-on: Launching an AI Agent Service on Google Colab via VLLM and Google ADK

If you are not familiar with Google ADK tools, you can read this article first: https://medium.com/@simon3458/google-adk-tools-intro-202504-3181fd6ab567

(Thanks to Twinkle AI community friend Thomas for assisting with the graphics!)

The main goal of this project is to deploy a Twinkle AI Gemma 3 T1 4B model in a Google Colab TPU v5e-1 environment and transform it into an AI Agent capable of executing specific tasks.

Google Colab Notebook Link

1. Environment Preparation and Dependency Installation

Hardware Setup: Confirm the current execution environment is Google TPU v5e-1, hardware designed specifically to accelerate machine learning workloads.

Core Packages: Install the vLLM inference engine that supports TPU acceleration; this is key to making the model run fast. Simultaneously install OpenAI SDK and LiteLLM for subsequent API connection and forwarding.

2. Launching vLLM Inference Service

Load Model: Start the vLLM server via terminal commands and load the Twinkle AI Gemma-3–4B-T1-it model.

Enable Advanced Features: Configure parameters at startup to enable the model's "Auto Tool Choice" and "Hermes Tool Parser," giving the model the ability to understand and call external tools.

Verify Service Status:

Check if the model API is successfully online.
Perform simple conversation tests to confirm the model responds normally.
Critical Test: Test if the model can correctly parse "Function Calling" (e.g., asking about database structures to confirm the model returns the correct tool execution request).

3. Setting up the LiteLLM API Bridge

Configure Forwarding Rules: Create a configuration file to forward standard API requests to the backend vLLM service. This step is to standardize the model interface for compatibility with Google ADK.

Start Proxy Service: Launch the LiteLLM proxy server in the background and monitor it until the service is fully ready (model list appears).

4. Integrating Google ADK (Agent Development Kit)

Get Agent Example: Download a pre-written Stock Query Agent example project from GitHub.

Install Agent Dependencies: Install the Python packages required for the Agent project.

Set Environment Variables: Configure the keys and API addresses needed for the Agent connection, pointing them to the LiteLLM service we just set up.

Run and Test Agent: Launch the Google ADK command-line interface and actually converse with the Agent (e.g., asking for TSMC's stock price). At this point, the Agent will automatically determine the need, call the stock query tool, retrieve data, and finally use the Gemma model to generate a natural language response.

5. (Optional) Establishing a Remote Development Tunnel

Setup ngrok: Use the ngrok tool to expose the API service inside Colab to the public internet.

Local Connection: This allows developers to develop the ADK frontend or logic on their local machine while leaving the heavy model inference computations to run on the TPU in Colab, achieving an efficient "Local Development, Cloud Inference" model.

This process demonstrates the complete integration from underlying model deployment and mid-layer API forwarding to upper-layer Agent application logic, utilizing Google Colab's TPU computing power to build intelligent AI applications.

IV. Conclusion

This hands-on exercise is not only a display of a technology stack but also verifies the huge potential of combining "Specialized Hardware" with "Localized Small Models."

Through the specialized acceleration of Google TPU v5e, we proved that even a lightweight model at the 4B parameter level, when paired with high-quality localized instruction fine-tuning (such as the efforts of Twinkle AI Gemma-3-T1-it) and an appropriate inference framework (vLLM + Google ADK), can demonstrate logical reasoning and tool usage capabilities that transcend its size class.

This solution offers three important insights for developers:

Compute is no longer a high wall: TPUs provide an efficient alternative to GPUs. Through platforms like Colab, developers can access powerful matrix computing resources with a lower barrier to entry.
Localization is crucial: The performance of the Twinkle AI model proves that models which solve "cultural disconnects" and "regulatory hallucinations" are better suited for actual business and life scenarios—an advantage general-purpose models struggle to replace.
Standardization of Agent Development: The introduction of Google ADK and standardized APIs (LiteLLM) evolves Agent development from "hand-crafting Prompts" to modular engineering practices, significantly improving development efficiency and stability.

With the open-sourcing of the Google Gemma 3 architecture and the ubiquity of TPU cloud resources, we are on the eve of a blossoming of AI applications. I hope this tutorial helps developers in various fields quickly build intelligent assistants that understand local languages and solve real problems, truly realizing the democratization and innovation of AI technology.

I am Simon

Hello everyone, I am Simon Liu (Liu Yu-wei), an AI Solutions Expert and currently a Google Developer Expert (AI Role). I look forward to helping enterprises implement Artificial Intelligence technologies to solve problems.

If this article was helpful to you, please give it a clap on Medium and follow my personal account so you can read my future articles at any time. You are welcome to leave comments on my LinkedIn to provide feedback and discuss AI-related topics with me. I look forward to being of help to everyone!

My Personal Website:

https://simonliuyuwei.my.canva.site/link-in-bio