50 Forms in 30 Days: What I Learned Building Assessment Forms With an AI Agent

This is the last post in the series. Over the past 30 days, I used Claude Desktop and the formlm-cli MCP server to build 50 assessment forms — everything from a 3-question mood check to a 40-item leadership 360 evaluation. Some were for production, some were tests, some were experiments to see how far I could push the AI.

Here's what worked, what didn't, and what I'd do differently.

The Numbers

Over 30 days:

• Forms created: 50 (12 production, 38 testing/experimentation)
• Total fields added: ~640 (average ~13 fields per form)
• MCP tool calls: ~3,200 (average ~64 per form)
• AI-induced bugs: 7 (details below)
• Time per form: 30 seconds for simple forms, 3-5 minutes for complex ones
• Manual fixes needed: 23 forms needed at least one manual correction after AI build

The manual fix rate (46%) is the number I want to drive down. It means roughly half the forms the AI built weren't ready without human intervention. The bugs weren't catastrophic — but they were real.

The 7 Bugs

Here's every AI-induced bug I hit, ranked by severity:

1. The field_update Options Wipe (Critical)

Claude replaced an entire options list with a single option, destroying the other options' scores. This is the find-then-set pattern story from article 12. Fixed by adding field_set_property and enforcing the find-first pattern.

2. The Staging/Production Profile Leak (High)

Claude published a form to staging when I asked for production. This is the multi-profile story from article 18. Fixed by making the environment visible in every response — still in progress.

3. The assess form clear Attempt (High)

Claude tried to bulk-delete all fields in a form. Blocked by the server-side whitelist. This is the whitelist story from article 19. No fix needed — the whitelist caught it.

4. The Duplicate Fields (Medium)

Claude re-added all fields after a session restart, creating duplicates. This is the idempotency story from article 15. Fixed by enforcing find-before-add.

5. The Boolean Trap (Medium)

Claude couldn't set required: false — it kept omitting the parameter, which defaulted to true. This is the boolean trap story from article 14. Fixed by making the schema description explicit about the default.

6. The Reverse Scoring Confusion (Low)

Claude inverted option scores when it should have used scale-level reversal. This is the scoring story from article 17. Fixed by being explicit about what kind of "reverse" I mean in prompts.

7. The Forgotten App ID (Low)

Claude lost track of the app ID mid-session and tried to add fields without it. This is from the stress test in article 13. Fix idea: maintain a "current app" state in the MCP server — not implemented yet.

Four of the seven bugs were caught before reaching production. Three made it to staging. None made it to production users. The whitelist was the hardest backstop — everything else was caught by manual review.

What Worked Well

Schema-Driven Discovery

Claude consistently called field_schema and field_config before adding fields. It treated the .describe() strings like documentation — reading them, understanding them, and making decisions based on them. This is the single most important design decision I made: making every parameter self-documenting through zod schemas.

Natural Language to Field Structure

Claude was excellent at translating assessment frameworks (PHQ-9, Maslach Burnout, Big Five) into field structures. It knew the items, the scoring ranges, and the dimensional groupings. Give it a framework name and it would produce the right fields with the right options and scores. This saved hours of manual work.

Safe Defaults

The share_publish tool hardcodes formDay: 3650000 (never expires) and formPerm: 1 (public read). The AI never had to decide on expiry or permissions — it just published. Safe defaults eliminated an entire class of potential mistakes.

The Whitelist

Every single time Claude tried something it shouldn't have (the clear attempt, the delete-all variant), the whitelist caught it. Not one dangerous command slipped through. The three-token subcommand extraction was simple, fast, and effective.

What Didn't Work

State Management Across Sessions

Claude doesn't remember what it did in a previous session. When I asked it to "continue building," it started from scratch — re-creating fields that already existed. The find-before-add pattern helped, but the fundamental issue is that MCP tools are stateless. There's no "current app" or "last field added" concept.

Complex Scoring Configuration

The MCP tools handle form construction well. They don't handle scoring at all. For any form that needed dimensional scoring, reverse items, or score cutoffs, I had to configure the Scale module manually. This was the biggest gap — about 70% of the forms I built needed manual scoring configuration after the AI finished the form structure.

Boolean Parameters

The required, unique, and shareable boolean parameters were consistently problematic. Claude's instinct to omit rather than explicitly set false caused silent bugs in multiple forms. The schema description fix helped, but it's a band-aid on a deeper design issue.

Environment Awareness

Claude had no concept of "staging" vs "production." It used whatever profile was active and didn't know the difference. The URL in the response was the only signal, and Claude didn't flag it when the environment didn't match my request.

Patterns That Emerged

Over 30 days, I noticed consistent patterns in how Claude interacted with the tools:

Claude always checks before adding. Before the first field_add, Claude calls field_schema and field_config. Every single time. This is good — it means the schema descriptions are doing their job.

Claude batches similar operations. When adding 10 fields, Claude doesn't ask for confirmation between each one. It adds all 10 in sequence. This is fast, but it means if the first field is wrong, the next 9 are probably wrong too (same pattern, same mistake).

Claude verifies after building. After adding fields, Claude calls field_list to check. This is a self-verification step that catches missing fields but doesn't catch scoring errors (since field_list shows structure, not scoring logic).

Claude gets creative when blocked. When Claude couldn't find a bulk-delete tool, it tried shell commands. When it couldn't set required: false, it tried omitting the parameter. The AI doesn't give up — it finds workarounds. Some of those workarounds are clever. Some are dangerous. The whitelist is what keeps the dangerous ones in check.

What I'd Do Differently

If I were starting over, I'd make these changes:

1. Add a field_add_many batch tool. Reduce 10 round-trips to 1. But define clear error semantics — if one field in the batch fails, return the error for that field and let the AI decide whether to retry.

2. Make the "current app" implicit. After app_create or app_get, set the app ID as the active context. Subsequent field operations default to the active app. This eliminates the "forgotten app ID" bug.

3. Expose scoring configuration via MCP. This is the biggest gap. Without it, the AI can only build the form shell — not the assessment engine. I'm working on this, but it requires careful whitelist design (AI can set scoring, but not reset it).

4. Add environment labels to every response. "Published on staging.formlm.me" not just the URL. Make the environment impossible to miss.

5. Make boolean parameters required, not optional. If required must be true or false, never undefined, Claude can't fall into the "omit means default" trap. The extra parameter in every call is worth the safety.

The Meta-Lesson

The biggest thing I learned over 30 days is this: designing tools for AI agents is different from designing tools for humans.

When I design a CLI for humans, I optimize for ergonomics — sensible defaults, helpful flags, clear error messages. Humans read the docs, understand the context, and make informed decisions.

When I design an MCP server for AI agents, I optimize for explicitness — no hidden defaults, no ambiguous parameters, no operations that require context the AI doesn't have. Every tool needs to be self-contained: the description tells the AI what it does, the schema tells the AI what to pass, and the response tells the AI what happened.

The .describe() string on every zod field isn't just documentation. It's the instruction manual the AI reads before deciding to use the tool. A vague description ("Mark as required") leads to vague behavior. An explicit description ("Defaults to true if omitted. Set to false explicitly for optional fields.") leads to correct behavior.

This series has been about the journey from "the code works" to "the AI can use the code correctly." The first part — writing working code — took a few weeks. The second part — making it AI-safe — has taken 30 days and counting, and I'm still finding edge cases.

What's Next

The formlm-cli MCP server now has 6 layered tools + 6 knowledge resources, a server-side whitelist, explicit schema descriptions, and the patterns (find-then-set, find-before-add) that make AI interactions safe. The CLI is open source. The platform is live at formlm.me.

We're not there yet. But 50 forms in 30 days has proven that the foundation works. The tools are solid. The patterns are reliable. The whitelist holds. The AI can build forms.

Now it can build assessments too.

---

This concludes the 20-part series on building AI-native tooling for FormLM. The CLI and MCP server are open source at github.com/formlm/cli. The platform is at formlm.me. Thanks for reading.