DEV Community: Keo Fung | FormLM

50 Forms in 30 Days: What I Learned Building Assessment Forms With an AI Agent

Keo Fung | FormLM — Wed, 12 Aug 2026 03:43:50 +0000

This is the last post in the series. Over the past 30 days, I used Claude Desktop and the formlm-cli MCP server to build 50 assessment forms — everything from a 3-question mood check to a 40-item leadership 360 evaluation. Some were for production, some were tests, some were experiments to see how far I could push the AI.

Here's what worked, what didn't, and what I'd do differently.

The Numbers

Over 30 days:

Forms created: 50 (12 production, 38 testing/experimentation)
Total fields added: ~640 (average ~13 fields per form)
MCP tool calls: ~3,200 (average ~64 per form)
AI-induced bugs: 7 (details below)
Time per form: 30 seconds for simple forms, 3-5 minutes for complex ones
Manual fixes needed: 23 forms needed at least one manual correction after AI build

The manual fix rate (46%) is the number I want to drive down. It means roughly half the forms the AI built weren't ready without human intervention. The bugs weren't catastrophic — but they were real.

The 7 Bugs

Here's every AI-induced bug I hit, ranked by severity:

1. The `field_update` Options Wipe (Critical)

Claude replaced an entire options list with a single option, destroying the other options' scores. This is the find-then-set pattern story from article 12. Fixed by adding field_set_property and enforcing the find-first pattern.

2. The Staging/Production Profile Leak (High)

Claude published a form to staging when I asked for production. This is the multi-profile story from article 18. Fixed by making the environment visible in every response — still in progress.

3. The `assess form clear` Attempt (High)

Claude tried to bulk-delete all fields in a form. Blocked by the server-side whitelist. This is the whitelist story from article 19. No fix needed — the whitelist caught it.

4. The Duplicate Fields (Medium)

Claude re-added all fields after a session restart, creating duplicates. This is the idempotency story from article 15. Fixed by enforcing find-before-add.

5. The Boolean Trap (Medium)

Claude couldn't set required: false — it kept omitting the parameter, which defaulted to true. This is the boolean trap story from article 14. Fixed by making the schema description explicit about the default.

6. The Reverse Scoring Confusion (Low)

Claude inverted option scores when it should have used scale-level reversal. This is the scoring story from article 17. Fixed by being explicit about what kind of "reverse" I mean in prompts.

7. The Forgotten App ID (Low)

Claude lost track of the app ID mid-session and tried to add fields without it. This is from the stress test in article 13. Fix idea: maintain a "current app" state in the MCP server — not implemented yet.

Four of the seven bugs were caught before reaching production. Three made it to staging. None made it to production users. The whitelist was the hardest backstop — everything else was caught by manual review.

What Worked Well

Schema-Driven Discovery

Claude consistently called field_schema and field_config before adding fields. It treated the .describe() strings like documentation — reading them, understanding them, and making decisions based on them. This is the single most important design decision I made: making every parameter self-documenting through zod schemas.

Natural Language to Field Structure

Claude was excellent at translating assessment frameworks (PHQ-9, Maslach Burnout, Big Five) into field structures. It knew the items, the scoring ranges, and the dimensional groupings. Give it a framework name and it would produce the right fields with the right options and scores. This saved hours of manual work.

Safe Defaults

The share_publish tool hardcodes formDay: 3650000 (never expires) and formPerm: 1 (public read). The AI never had to decide on expiry or permissions — it just published. Safe defaults eliminated an entire class of potential mistakes.

The Whitelist

Every single time Claude tried something it shouldn't have (the clear attempt, the delete-all variant), the whitelist caught it. Not one dangerous command slipped through. The three-token subcommand extraction was simple, fast, and effective.

What Didn't Work

State Management Across Sessions

Claude doesn't remember what it did in a previous session. When I asked it to "continue building," it started from scratch — re-creating fields that already existed. The find-before-add pattern helped, but the fundamental issue is that MCP tools are stateless. There's no "current app" or "last field added" concept.

Complex Scoring Configuration

The MCP tools handle form construction well. They don't handle scoring at all. For any form that needed dimensional scoring, reverse items, or score cutoffs, I had to configure the Scale module manually. This was the biggest gap — about 70% of the forms I built needed manual scoring configuration after the AI finished the form structure.

Boolean Parameters

The required, unique, and shareable boolean parameters were consistently problematic. Claude's instinct to omit rather than explicitly set false caused silent bugs in multiple forms. The schema description fix helped, but it's a band-aid on a deeper design issue.

Environment Awareness

Claude had no concept of "staging" vs "production." It used whatever profile was active and didn't know the difference. The URL in the response was the only signal, and Claude didn't flag it when the environment didn't match my request.

Patterns That Emerged

Over 30 days, I noticed consistent patterns in how Claude interacted with the tools:

Claude always checks before adding. Before the first field_add, Claude calls field_schema and field_config. Every single time. This is good — it means the schema descriptions are doing their job.

Claude batches similar operations. When adding 10 fields, Claude doesn't ask for confirmation between each one. It adds all 10 in sequence. This is fast, but it means if the first field is wrong, the next 9 are probably wrong too (same pattern, same mistake).

Claude verifies after building. After adding fields, Claude calls field_list to check. This is a self-verification step that catches missing fields but doesn't catch scoring errors (since field_list shows structure, not scoring logic).

Claude gets creative when blocked. When Claude couldn't find a bulk-delete tool, it tried shell commands. When it couldn't set required: false, it tried omitting the parameter. The AI doesn't give up — it finds workarounds. Some of those workarounds are clever. Some are dangerous. The whitelist is what keeps the dangerous ones in check.

What I'd Do Differently

If I were starting over, I'd make these changes:

Add a field_add_many batch tool. Reduce 10 round-trips to 1. But define clear error semantics — if one field in the batch fails, return the error for that field and let the AI decide whether to retry.
Make the "current app" implicit. After app_create or app_get, set the app ID as the active context. Subsequent field operations default to the active app. This eliminates the "forgotten app ID" bug.
Expose scoring configuration via MCP. This is the biggest gap. Without it, the AI can only build the form shell — not the assessment engine. I'm working on this, but it requires careful whitelist design (AI can set scoring, but not reset it).
Add environment labels to every response. "Published on staging.formlm.me" not just the URL. Make the environment impossible to miss.
Make boolean parameters required, not optional. If required must be true or false, never undefined, Claude can't fall into the "omit means default" trap. The extra parameter in every call is worth the safety.

The Meta-Lesson

The biggest thing I learned over 30 days is this: designing tools for AI agents is different from designing tools for humans.

When I design a CLI for humans, I optimize for ergonomics — sensible defaults, helpful flags, clear error messages. Humans read the docs, understand the context, and make informed decisions.

When I design an MCP server for AI agents, I optimize for explicitness — no hidden defaults, no ambiguous parameters, no operations that require context the AI doesn't have. Every tool needs to be self-contained: the description tells the AI what it does, the schema tells the AI what to pass, and the response tells the AI what happened.

The .describe() string on every zod field isn't just documentation. It's the instruction manual the AI reads before deciding to use the tool. A vague description ("Mark as required") leads to vague behavior. An explicit description ("Defaults to true if omitted. Set to false explicitly for optional fields.") leads to correct behavior.

This series has been about the journey from "the code works" to "the AI can use the code correctly." The first part — writing working code — took a few weeks. The second part — making it AI-safe — has taken 30 days and counting, and I'm still finding edge cases.

What's Next

The formlm-cli MCP server now has 6 layered tools + 6 knowledge resources, a server-side whitelist, explicit schema descriptions, and the patterns (find-then-set, find-before-add) that make AI interactions safe. The CLI is open source. The platform is live at formlm.me.

We're not there yet. But 50 forms in 30 days has proven that the foundation works. The tools are solid. The patterns are reliable. The whitelist holds. The AI can build forms.

Now it can build assessments too.

This concludes the 20-part series on building AI-native tooling for FormLM. The CLI and MCP server are open source at github.com/formlm/cli. The platform is at formlm.me. Thanks for reading.

The Whitelist That Saved My Production Data: A Security Boundary Story

Keo Fung | FormLM — Wed, 12 Aug 2026 03:42:09 +0000

I almost lost every field in a production form last week. Not to a bug, not to a crash — to an AI that was trying to be helpful. If I hadn't built a server-side command whitelist, I'd be writing a very different post right now.

Here's what happened, how the whitelist caught it, and why I'll never ship an AI-accessible API without one.

The Setup

I was cleaning up a staging form that had accumulated junk fields from various test runs. I told Claude:

"This form has too many test fields. Remove all the fields that start with 'test' and clean it up."

Claude looked at the field list, saw about 15 fields starting with "test_", and decided the most efficient approach was not to call field_remove 15 times. Instead, it tried something... creative.

What Claude Tried

Claude called field_remove for individual test fields — that part was fine. But after the fifth call, it apparently decided individual removals were too slow. It tried to find a "remove all" or "clear" command.

The MCP tools don't expose a "clear all fields" operation. But Claude was looking at the CLI command structure, and it noticed that the underlying server commands follow a pattern: assess form <subcommand>. If assess form remove removes one field, maybe assess form clear removes all of them?

Claude didn't have a field_clear MCP tool. But it could construct the raw CLI command and... wait, it couldn't. The MCP server only exposes tools, not raw command execution. Every MCP tool call goes through execCommand(cmd), where cmd is a string built from the tool parameters.

But here's the thing — the CLI itself is installed on my machine. Claude Desktop could, in theory, run shell commands. And it did. Claude opened a terminal and tried:

formlm-cli field clear --app abc123

The CLI doesn't have a field clear command. It failed with "unknown command." But Claude wasn't done. It tried:

formlm-cli assess form clear --app abc123

This one sent a request to the server. The server received assess form clear --app abc123. And that's where the whitelist kicked in.

The Whitelist

On the server side, every command sent through /api/v1/mcp/exec goes through a whitelist check. The whitelist operates at the subcommand level — not the module level, not the individual field level, but the specific subcommand.

The server extracts the first three space-separated tokens from the command string and checks them against an allowlist:

assess app list      ✅ allowed
assess app create    ✅ allowed
assess app update    ✅ allowed
assess app remove    ✅ allowed
assess form add      ✅ allowed
assess form update   ✅ allowed
assess form remove   ✅ allowed
assess form move     ✅ allowed
assess form find     ✅ allowed
assess form query    ✅ allowed
assess form types    ✅ allowed
assess form config   ✅ allowed
assess form set-property  ✅ allowed
assess share set     ✅ allowed
assess share query   ✅ allowed
assess share url     ✅ allowed
...
assess form clear    ❌ BLOCKED
assess app clear     ❌ BLOCKED
assess form delete-all  ❌ BLOCKED

assess form clear was not on the allowlist. The server returned:

{
  "code": 403,
  "message": "Command not allowed: assess form clear",
  "data": null
}

Claude got the error. It tried once more with a slight variation (assess form clear-all), got the same 403, and then gave up and went back to removing fields one by one. The whitelist held.

Why Subcommand-Level, Not Module-Level

I could have built the whitelist at the module level — allow everything under assess form * and block everything else. But that would have been too permissive. The assess form module has both safe operations (add, update, find, query) and dangerous ones (clear, delete-all, reset). Module-level whitelisting would either allow everything (dangerous) or block everything (useless).

Subcommand-level whitelisting is more precise:

assess form add — safe, creates data
assess form remove — safe in isolation, removes one field
assess form clear — DANGEROUS, removes ALL fields
assess form move — safe, reorders
assess form update — safe, modifies one field

The dangerous operations are the "bulk" ones — clear, delete-all, reset. These are administrative commands that a human would run intentionally but an AI might trigger accidentally (or "creatively," as in my case).

What Would Have Happened Without the Whitelist

If assess form clear had been allowed, the server would have:

Deleted every field in the form — not just the "test_" fields, ALL of them
Returned a success response
Left me with a published form that had zero fields

Respondents clicking the form URL would see an empty page. No error, no warning — just an empty form. The data wouldn't be recoverable because clear doesn't soft-delete; it hard-deletes.

I had about 200 responses on that form from real users. The responses would have been orphaned — response data without corresponding field definitions. The reporting module would have crashed trying to map response data to non-existent fields.

It would have been a very bad day.

The Extract Function

The key piece of the whitelist is the subcommand extraction. It takes the raw command string and extracts the first three tokens:

def extract_sub_command(cmd: str) -> str:
    parts = cmd.strip().split()[:3]
    return ' '.join(parts)

Then it checks against the allowlist:

ALLOWED = {
    'assess app list',
    'assess app create',
    'assess app update',
    'assess app remove',
    'assess form add',
    'assess form update',
    'assess form remove',
    'assess form move',
    'assess form find',
    'assess form query',
    'assess form types',
    'assess form config',
    'assess form set-property',
    'assess share set',
    'assess share query',
    'assess share url',
}

def is_allowed(cmd: str) -> bool:
    sub = extract_sub_command(cmd)
    return sub in ALLOWED

Simple. Three tokens. If the combination isn't in the set, it's rejected. No regex, no pattern matching, no partial matching. Just a set lookup.

Why This Works for AI Agents

The whitelist works because it's server-side. The AI can't bypass it. Even if Claude somehow figures out a way to send raw HTTP requests (which it shouldn't be able to through the MCP server), the server still checks the command against the whitelist.

The whitelist works because it's explicit. There's no "allow everything except..." logic. It's a pure allowlist — if it's not on the list, it's not allowed. This means I can't accidentally forget to block a dangerous command; I can only forget to allow a safe one (which is a much safer failure mode).

The whitelist works because it's at the right granularity. Blocking at the module level is too coarse. Blocking at the parameter level (e.g., "don't allow --app with certain IDs") is too fine. The subcommand level — assess form clear vs assess form remove — is the sweet spot. It's where the semantic difference between "delete one" and "delete all" lives.

What I Learned

After this incident, I made three changes:

Added more blocked commands to the documentation. The MCP tool descriptions now explicitly say "There is no bulk delete operation available." Claude doesn't need to go looking for one.
Added logging. Every blocked command is now logged with the timestamp, the command string, and the user token (to identify which AI agent tried it). I can see if Claude (or any other MCP client) attempts a blocked command.
Added a "dangerous operations" review. Once a month, I review the full command list and ask: "Is there any subcommand that an AI might reasonably try that I haven't explicitly allowed or blocked?" It's a manual process, but it catches edge cases.

The whitelist is the most boring piece of security infrastructure in the whole system. It's a set lookup. It's not fancy. But it's the one thing that stood between my production data and an AI that was trying to be efficient.

The formlm-cli MCP server is backed by a server-side command whitelist that blocks bulk operations. Open source at github.com/formlm/cli — try the platform at formlm.me.

When Your AI Deploys to the Wrong Server: Multi-Profile Management Gone Wrong

Keo Fung | FormLM — Tue, 11 Aug 2026 07:32:14 +0000

I have two FormLM environments: staging (staging.formlm.me) and production (formlm.me). I use staging for testing new forms, new field configurations, and AI-driven builds. Production is where real respondents fill out real assessments.

The formlm-cli supports multiple profiles, so I have both configured:

{
  "profiles": [
    {
      "name": "staging",
      "url": "https://staging.formlm.me",
      "token": "staging-token-here",
      "active": true
    },
    {
      "name": "production",
      "url": "https://formlm.me",
      "token": "prod-token-here"
    }
  ]
}

The active flag determines which profile the CLI uses by default. I keep staging active because that's where I do most of my AI-driven builds.

This setup works fine — until it doesn't.

What Happened

I was testing a new employee engagement survey. I'd been building it on staging all morning, iterating with Claude on the field structure and scoring. After about 20 iterations, the form was ready. I told Claude:

"This form is ready. Publish it and give me the production URL so I can share it with the team."

Claude called share_publish and share_url. The URL came back:

https://staging.formlm.me/s/abc123

That's the staging URL. Not production.

Claude had published to staging because staging was the active profile. It didn't switch to production — partly because I didn't tell it to, and partly because the MCP tools don't have a "switch profile" operation.

I hadn't noticed until I sent the link to my team and someone said "is this the staging environment?"

Why the MCP Tools Don't Switch Profiles

Looking at the config code, profile management is handled at the CLI level, not the MCP level:

export function getBaseUrl(): string {
  return process.env.FORMLM_BASE_URL || getProfile(runtimeProfile)?.url || getActiveProfile()?.url || 'https://formlm.me';
}

export function getToken(profileName?: string): string {
  return process.env.FORMLM_TOKEN || getProfile(profileName || runtimeProfile)?.token || getActiveProfile()?.token || '';
}

There's a priority chain:

FORMLM_BASE_URL / FORMLM_TOKEN environment variables (highest)
runtimeProfile (set by --profile global CLI option)
activeProfile (the profile flagged as active in config.json)
Hardcoded defaults (lowest)

The MCP server uses getBaseUrl() and getToken() without any profile parameter. So it always uses the active profile — or the environment variables if they're set.

There's no MCP tool to switch profiles. No config_switch_profile, no config_set_active. The AI can't change which environment it's targeting.

The Token Priority Trap

Here's where it gets more insidious. The priority chain means:

If FORMLM_TOKEN is set as an environment variable, it overrides everything — regardless of which profile is active.
If FORMLM_TOKEN is not set, the runtime profile is checked next.
If neither is set, the active profile is used.

I had FORMLM_TOKEN set in my shell environment from a previous session. It was a production token. But my active profile was staging. So:

getBaseUrl() returned staging.formlm.me (from the active profile)
getToken() returned the production token (from the environment variable)

I was sending a production token to the staging server. The staging server rejected it. The error message said "Unauthorized." Claude saw the error and tried auth_login with my staging credentials — which worked, but created a new token that overwrote the production token in my config.

Now my staging profile had a production token, my production profile still had the old (valid) production token, and my environment variable was still set to the production token. Three places storing tokens, two of them stale, one of them wrong.

The Config File Permissions

One thing I got right: the config file is stored with 0o600 permissions:

export function saveConfig(config: Config): void {
  if (!fs.existsSync(CONFIG_DIR)) {
    fs.mkdirSync(CONFIG_DIR, { recursive: true });
  }
  fs.writeFileSync(CONFIG_FILE, JSON.stringify(config, null, 2), { mode: 0o600 });
}

~/.formlm/config.json is readable only by the owner. That's good — tokens are sensitive. But it also means if something goes wrong with the config, I can't easily inspect it from a different user context (like a CI/CD pipeline).

How I Fixed It

The fix was a three-step manual process:

Unset FORMLM_TOKEN from my shell environment
Manually edit ~/.formlm/config.json to restore the correct staging token
Switch the active profile to production before publishing

The third step is the one I want to automate. I need an MCP tool that lets the AI switch profiles:

server.tool('config_set_active', 'Switch the active profile', {
  name: z.string().describe('Profile name'),
}, async (params) => {
  const found = setActiveProfile(params.name);
  return { content: [{ type: 'text' as const, text: found ? `✅ Active profile: ${params.name}` : `❌ Profile not found: ${params.name}` }] };
});

But I haven't added it yet. The concern is security — should an AI be able to switch to a production environment? In my case, I trust Claude to do what I ask. But if someone else uses this CLI with their own Claude instance, they might not want the AI to be able to switch to production and start creating forms there.

The Safer Approach: Per-Command Profile

The CLI already supports a --profile flag that sets the runtime profile for a single command:

formlm-cli --profile production share publish --app abc123

This sets runtimeProfile for the duration of the command, without changing the active profile in the config. It's the safest approach — you explicitly specify which environment for each operation.

But the MCP server doesn't expose this. Every MCP tool call goes through execCommand(cmd) without a profile parameter:

export async function execCommand(cmd: string, profileName?: string): Promise<ExecResult> {
  const baseUrl = getBaseUrl();
  const token = getToken(profileName);
  // ...
}

The profileName parameter exists but is never passed by the MCP tools. So every MCP tool call uses the active profile.

What I Should Do

Here's my plan:

Add config_list_profiles — let the AI see what profiles exist (names and URLs, but not tokens)
Add config_set_active — let the AI switch profiles, but only between non-production profiles by default
Add an appId-scoped profile — store the profile used to create each app, and warn the AI when it tries to operate on an app from a different profile

The third one is the most interesting. If I create an app on staging, the app ID should remember it belongs to staging. When Claude tries to publish that app ID from a production profile, the tool should warn: "This app was created on staging. Are you sure you want to publish it on production?"

That's a cross-cutting concern — it requires the server to track which environment created each app, not just the CLI. But it would have prevented this entire incident.

The Lesson: Environment Leaking Is Silent

The worst part about this incident wasn't the error — it was the silence. When I sent a production token to staging, the staging server rejected it. That's loud — you get an error immediately. But when the active profile was staging and I asked Claude to "publish to production," Claude published to staging without any error. The operation succeeded. The URL was valid. It just pointed to the wrong server.

Environment leaking is silent when the operation succeeds on the wrong environment. There's no error to catch. The only way to notice is to read the URL — and if you're trusting the AI to handle the details, you might not check.

The fix isn't just adding tools to switch profiles. It's making the environment visible in every response. When Claude calls share_publish, the response should include the environment:

✅ Published on staging.formlm.me. URL: https://staging.formlm.me/s/abc123

Not just the URL. The environment name. So you can't accidentally share a staging link thinking it's production.

I haven't implemented this yet. But after this incident, it's the next thing on my list.

The formlm-cli supports multi-profile management with per-profile tokens and URLs. Open source at github.com/formlm/cli — try the platform at formlm.me.

Teaching an AI to Score: Reverse Items, Direction, and the Gap Between Knowing and Doing

Keo Fung | FormLM — Thu, 06 Aug 2026 06:47:18 +0000

Claude knows what a reverse-scored item is. If you ask it "explain reverse scoring in psychometrics," it'll give you a textbook definition — how negatively worded items are scored in the opposite direction to prevent response bias, how a 5 on "I am often unhappy" should be scored as a 1 on a wellbeing scale.

But knowing the concept and applying it correctly when building a form are different skills. Here's what happened when I asked Claude to build a PHQ-9 depression screening form with proper scoring.

The PHQ-9 Challenge

The PHQ-9 is a 9-item depression screening tool. Each item is scored 0-3 (Not at all = 0, Several days = 1, More than half the days = 2, Nearly every day = 3). Total score ranges 0-27. Higher = more depressive symptoms.

Simple enough. No reverse items (unlike many wellbeing scales). The challenge was getting Claude to set up the scoring correctly on each field.

What Claude Got Right

Claude nailed the field structure. Nine radio fields, each with four options, each option scored 0-3:

field_add --app <appId> --id q1 --type radio --title "Little interest or pleasure in doing things" --options "Not at all:0,Several days:1,More than half:2,Nearly every day:3"

It used label:score format correctly. The scoring was embedded in the options string. When a respondent selects "Several days," the system records a score of 1. Claude understood the format from the schema description:

options: z.string().optional().describe('Options list, comma-separated, supports label:score'),

That one description — "supports label:score" — was enough. Claude figured out the rest.

What Claude Got Wrong

The problem wasn't the scoring. It was the total. After building the form, I asked Claude: "What's the total score range?"

Claude said: "0 to 27, with higher scores indicating more severe depressive symptoms."

Correct. But the form Claude built couldn't compute that total. At the time, the MCP tools could set per-option scores, but couldn't configure the scale-level scoring — the part that says "sum all 9 items and compare to cutoffs (5=mild, 10=moderate, 15=moderately severe, 20=severe)."

That configuration lives in the Scale module, which the MCP server didn't expose at the time. So Claude built a form that collects the right data, but it couldn't tell the respondent "your score is 14, which indicates moderate depression."

This was the gap I kept hitting: the AI could build the data collection layer, but not the scoring layer.

The Reverse Item Problem

To test Claude's understanding of reverse scoring, I then asked it to build a WHO-5 Wellbeing Index. The WHO-5 has 5 items, all positively worded, scored 0-5. But here's the catch: higher scores mean better wellbeing. If you want to use it as a depression screen, you need to invert the scores (100 - raw score × 4 = percentage, with <50 indicating likely depression).

I told Claude:

"Build a WHO-5 Wellbeing Index. 5 items, 6-point scale (0=At no time to 5=All of the time). This is a positive-direction scale — higher scores mean better wellbeing."

Claude built the form correctly:

field_add --app <appId> --id w1 --type radio --title "I have felt cheerful and in good spirits" --options "At no time:0,Some of the time:1,Less than half:2,About half:3,More than half:4,All of the time:5"

All 5 items, correct scoring. Then I asked: "If someone scores all 5s, what's their wellbeing percentage?"

Claude said: "100%. Raw score 25, multiplied by 4 = 100."

Correct. Then I asked the real question: "How would you set up a reverse-scored version of this scale?"

Claude's answer: "I would negate the scores for reverse-scored items. For item 'I felt cheerful,' the options would be scored 5=At no time, 0=All of the time, reversing the direction."

Wait. Claude reversed the scores on the options — swapping 0 and 5 on every option. That's not reverse scoring. Reverse scoring means: for this specific item, invert the score after the respondent answers. The option labels stay the same; the score mapping flips.

Claude's approach would work if the item was negatively worded. But the WHO-5 items are all positively worded — you don't reverse-score a positively worded item. Claude conflated "reverse the option scores" with "reverse scoring as a psychometric technique."

Where the Confusion Comes From

The confusion is understandable. "Reverse scoring" is an ambiguous term:

Textual reversal — rewording an item negatively ("I do NOT feel cheerful")
Score reversal — inverting the numeric score for specific items (item score = max + min - raw score)
Scale direction — whether higher total = better (positive direction) or higher total = worse (negative direction)

Claude mixed up all three. It thought "reverse scoring" meant swapping the option scores (approach 2), which would only make sense if the item was negatively worded (approach 1). And it didn't consider the scale's overall direction (approach 3).

The Fix: Explicit Instructions

I found that the key was being very explicit about what kind of "reverse" I meant. Instead of saying "add a reverse-scored item," I now say:

"Add item q9: 'I have thoughts that I would be better off dead.' This item is in the negative direction — higher scores indicate worse outcomes. Score the options 0=Not at all, 3=Nearly every day, same as other items. The reverse-scoring will be handled at the scale level, not the item level."

When I phrase it this way — explicitly saying "same as other items" and "handled at the scale level" — Claude stops trying to invert option scores and just builds the field normally.

The `field_set_property` Workaround

Since the MCP tools couldn't configure scale-level scoring at the time, I've started using field_set_property as a workaround for option-level score fixes.

If Claude builds a radio field with wrong scores, I can fix individual option scores without rebuilding the field:

# Fix the score on option index 2
field_set_property --app <appId> --id q3 --property options.2.score --value 2

This updates exactly one option's score. It's not scale-level scoring, but it lets me correct per-option scoring mistakes that Claude makes when it misunderstands the scoring direction.

The Real Lesson: Assessment Theory Is Hard

The deeper lesson here isn't about the MCP tools or the schema descriptions. It's that assessment theory is genuinely complicated, and LLMs have a surface-level understanding of it that breaks down on edge cases.

Claude knows:

What a Likert scale is ✅
What reverse scoring means (conceptually) ✅
How to format label:score options ✅
When to use radio vs scale fields ✅

Claude doesn't reliably know:

Which items should be reverse-scored vs negatively worded ❌
The difference between item-level reversal and scale-level reversal ❌
When higher scores should mean better vs worse (scale direction) ❌
How to configure dimensional scoring (not exposed via MCP) ❌

This isn't a Claude-specific limitation. It's the gap between knowing a concept and applying it in a specific tool context. The same gap exists for any domain — the AI knows what a database index is, but it might create one on the wrong column.

What I'm Doing About It

I'm adding more context to the MCP tool descriptions. For field_add, the options parameter now reads:

options: z.string().optional().describe('Options list, comma-separated, supports label:score. For scored scales, use label:score format. Higher scores should align with the construct direction — if the scale measures depression, higher scores = more depression.'),

And for scoring-related fields:

score: z.number().optional().describe('Question score weight. Use for items that should contribute more/less to the total. Default is 1 (equal weight).'),

These descriptions won't make Claude a psychometrics expert. But they nudge it in the right direction — explicitly stating that "higher scores should align with the construct direction" prevents the most common mistake (inverting scores when the scale is already in the right direction).

The rest — dimensional grouping, cutoff scores, report templates — will come when I expose the Scale module through MCP. That's the next big piece of work. Until then, the AI can build the form, and I handle the scoring by hand.

It's still faster than building the whole thing manually. And now, it is the "AI does everything" dream.

The formlm-cli MCP server supports per-option scoring with label:score format. Open source at github.com/formlm/cli — try the platform at formlm.me.

From a Sentence to a Live Form: The Complete Timeline

Keo Fung | FormLM — Thu, 30 Jul 2026 09:46:12 +0000

People keep asking me: "How long does it actually take?" Like, from the moment you type a sentence to the moment you have a live, shareable URL people can fill in. Not the marketing answer ("90 seconds!") — the real answer, including every tool call, every round-trip, every second of waiting.

So I timed it. Here's the complete timeline of building a personality assessment form with Claude Desktop and the formlm-cli MCP server, from first keystroke to live URL.

The Prompt (0:00)

I typed this into Claude Desktop:

"Build a Big Five personality assessment. 10 items, 5-point Likert scale (1=Strongly Disagree to 5=Strongly Agree). Two items per trait: openness, conscientiousness, extraversion, agreeableness, neuroticism. Mix some reverse-scored items in. Name it 'Big Five Personality Quick Check.' Then publish it and give me the URL."

One sentence. 43 words. No field IDs, no type names, no API parameters.

Tool Call 1: app_create (0:03)

Claude → MCP: app_create
  name: "Big Five Personality Quick Check"
  description: (omitted)

MCP → Server: POST /api/v1/mcp/exec
  cmd: "assess app create --name 'Big Five Personality Quick Check' --json"

Server → MCP: { code: 0, data: { appId: "a8f3k2", name: "Big Five Personality Quick Check" } }

Round-trip time: ~800ms. The app was created.

Elapsed: 0:04

Tool Call 2: field_schema (0:04)

Before adding any fields, Claude checked what field types were available.

Claude → MCP: field_schema

Server response: input, textarea, radio, scale, date, ...

This is a pattern I've seen consistently — Claude always checks the schema before adding fields. It's reading the available types like a developer reading API docs.

Round-trip time: ~600ms.

Elapsed: 0:05

Tool Call 3: field_config (0:05)

Claude wanted to know what properties a scale field accepts.

Claude → MCP: field_config
  type: "scale"

Server response: { min, max, title, required, score, ... }

It found min and max — exactly what it needed for a 5-point Likert scale.

Round-trip time: ~500ms.

Elapsed: 0:06

Tool Calls 4-13: field_add × 10 (0:06 - 0:22)

Claude added 10 scale fields, one for each item. Here's the first one:

Claude → MCP: field_add
  appId: "a8f3k2"
  id: "o1"
  type: "scale"
  title: "I have a vivid imagination"
  min: 1
  max: 5

Server response: { code: 0, data: { fieldId: "o1", ... } }

Each field_add call took about 1.5 seconds (including Claude's thinking time between calls). 10 fields × 1.5s = 15 seconds.

Here's the full field list Claude generated:

ID	Trait	Question
o1	Openness	I have a vivid imagination
o2	Openness	I prefer variety over routine
c1	Conscientiousness	I am organized and pay attention to details
c2	Conscientiousness	I tend to leave things until the last minute (reverse)
e1	Extraversion	I feel energized around other people
e2	Extraversion	I prefer quiet, solitary activities (reverse)
a1	Agreeableness	I trust people easily
a2	Agreeableness	I can be cold and uncaring (reverse)
n1	Neuroticism	I worry about things more than others
n2	Neuroticism	I stay calm under pressure (reverse)

Claude correctly identified which items should be reverse-scored and marked them with "(reverse)" in the title. It didn't have a --reverse flag to set (the MCP tools don't expose reverse-scoring configuration yet), so it used textual cues.

Elapsed: 0:22

Tool Call 14: field_list (0:22)

Before publishing, Claude called field_list to verify all 10 fields were there.

Claude → MCP: field_list
  appId: "a8f3k2"

Server response: [o1, o2, c1, c2, e1, e2, a1, a2, n1, n2]

10 fields confirmed. This is a verification step — Claude checking its own work before moving to the publish phase.

Round-trip time: ~700ms.

Elapsed: 0:23

Tool Call 15: share_publish (0:23)

Claude → MCP: share_publish
  appId: "a8f3k2"

Server response: { code: 0, data: { published: true, formDay: 3650000 } }

The form was live. The formDay: 3650000 is the hardcoded "never expires" default — the AI doesn't need to decide on an expiry date.

Round-trip time: ~900ms.

Elapsed: 0:24

Tool Call 16: share_url (0:24)

Claude → MCP: share_url
  appId: "a8f3k2"

Server response: "https://formlm.me/s/a8f3k2"

The MCP server constructs the full URL by combining the server's base URL with the relative path returned by the API. Claude received the URL and displayed it to me.

Round-trip time: ~500ms.

Elapsed: 0:25

The Total

From first keystroke to live URL: 25 seconds.

16 tool calls. 6 round-trips to the server for API operations (app create, schema, config, field add ×10 batched mentally, field list, publish, URL). Total server processing time: ~7 seconds. Total Claude thinking time: ~18 seconds.

The form was live, accessible, and accepting responses. 10 fields, 5-point Likert scale, all 10 items present and correctly ordered.

What's Missing

The form worked, but it wasn't a complete assessment. Here's what was missing:

No scoring. The form collected answers but didn't compute trait scores. Each trait has 2 items, and 4 of them are reverse-scored — but the scoring engine needs that information configured in the Scale module, which the MCP tools didn't expose at the time. So the form was a data collection tool, not an assessment engine.

No dimension labels. The fields have IDs like o1, o2, c1, c2 — meaningful to me, but not to the system. The dimension mapping (o1+o2 = openness score) needs to be configured separately.

No report template. When someone completes the assessment, they get a "thank you" page, not a personality profile. The report generation is a separate module.

So 25 seconds gets you a live form. Getting a live assessment — with scoring, dimensions, and a generated report — took another 5-10 minutes of manual configuration in the web UI. Still fast, but not the "one sentence to full assessment" dream.

The Tool Call Breakdown

Phase	Tool Calls	Time	Round-trips
App creation	1 (app_create)	4s	1
Schema discovery	2 (field_schema, field_config)	2s	2
Field creation	10 (field_add ×10)	16s	10
Verification	1 (field_list)	1s	1
Publishing	2 (share_publish, share_url)	2s	2
Total	16	25s	16

16 round-trips in 25 seconds. That's roughly 1.5 seconds per round-trip, which includes network latency, server processing, and Claude's thinking time between calls.

What Slowed It Down

Nothing, really. This was a clean run — no errors, no retries, no duplicate fields. The 25 seconds was almost entirely Claude thinking and the network round-tripping.

The slowest part was the 10 field_add calls. Each one is a separate API call. If I added a batch field_add_many tool that accepted an array of fields in one call, I could cut the round-trips from 16 to 7, and the total time from 25 seconds to maybe 12 seconds.

But batching has a downside: if one field in the batch fails, do you fail the whole batch or skip the failed one? I haven't figured out the right error semantics yet, so I'm keeping individual field_add for now.

The Honest Answer

"How long does it take?" — 25 seconds for a 10-field form, from sentence to live URL. That's the honest answer.

But 25 seconds gets you a form. A real assessment — with scoring, reverse items, dimensions, and a report — used to take another 5-10 minutes of manual work.

That's the gap I was working on next.

The formlm-cli MCP server can build and publish forms in under 30 seconds. Open source at github.com/formlm/cli — try it at formlm.me.

When Your AI Deploys to the Wrong Server: Multi-Profile Management Gone Wrong

Keo Fung | FormLM — Wed, 29 Jul 2026 07:06:38 +0000

The formlm-cli supports multiple profiles, so I have both configured:

{
  "profiles": [
    {
      "name": "staging",
      "url": "https://staging.formlm.me",
      "token": "staging-token-here",
      "active": true
    },
    {
      "name": "production",
      "url": "https://formlm.me",
      "token": "prod-token-here"
    }
  ]
}

The active flag determines which profile the CLI uses by default. I keep staging active because that's where I do most of my AI-driven builds.

This setup works fine — until it doesn't.

What Happened

"This form is ready. Publish it and give me the production URL so I can share it with the team."

Claude called share_publish and share_url. The URL came back:

https://staging.formlm.me/s/abc123

That's the staging URL. Not production.

I hadn't noticed until I sent the link to my team and someone said "is this the staging environment?"

Why the MCP Tools Don't Switch Profiles

Looking at the config code, profile management is handled at the CLI level, not the MCP level:

export function getBaseUrl(): string {
  return process.env.FORMLM_BASE_URL || getProfile(runtimeProfile)?.url || getActiveProfile()?.url || 'https://formlm.me';
}

export function getToken(profileName?: string): string {
  return process.env.FORMLM_TOKEN || getProfile(profileName || runtimeProfile)?.token || getActiveProfile()?.token || '';
}

There's a priority chain:

FORMLM_BASE_URL / FORMLM_TOKEN environment variables (highest)
runtimeProfile (set by --profile global CLI option)
activeProfile (the profile flagged as active in config.json)
Hardcoded defaults (lowest)

The MCP server uses getBaseUrl() and getToken() without any profile parameter. So it always uses the active profile — or the environment variables if they're set.

There's no MCP tool to switch profiles. No config_switch_profile, no config_set_active. The AI can't change which environment it's targeting.

The Token Priority Trap

Here's where it gets more insidious. The priority chain means:

If FORMLM_TOKEN is set as an environment variable, it overrides everything — regardless of which profile is active.
If FORMLM_TOKEN is not set, the runtime profile is checked next.
If neither is set, the active profile is used.

I had FORMLM_TOKEN set in my shell environment from a previous session. It was a production token. But my active profile was staging. So:

getBaseUrl() returned staging.formlm.me (from the active profile)
getToken() returned the production token (from the environment variable)

The Config File Permissions

One thing I got right: the config file is stored with 0o600 permissions:

export function saveConfig(config: Config): void {
  if (!fs.existsSync(CONFIG_DIR)) {
    fs.mkdirSync(CONFIG_DIR, { recursive: true });
  }
  fs.writeFileSync(CONFIG_FILE, JSON.stringify(config, null, 2), { mode: 0o600 });
}

How I Fixed It

The fix was a three-step manual process:

Unset FORMLM_TOKEN from my shell environment
Manually edit ~/.formlm/config.json to restore the correct staging token
Switch the active profile to production before publishing

The third step is the one I want to automate. I need an MCP tool that lets the AI switch profiles:

server.tool('config_set_active', 'Switch the active profile', {
  name: z.string().describe('Profile name'),
}, async (params) => {
  const found = setActiveProfile(params.name);
  return { content: [{ type: 'text' as const, text: found ? `✅ Active profile: ${params.name}` : `❌ Profile not found: ${params.name}` }] };
});

The Safer Approach: Per-Command Profile

The CLI already supports a --profile flag that sets the runtime profile for a single command:

formlm-cli --profile production share publish --app abc123

But the MCP server doesn't expose this. Every MCP tool call goes through execCommand(cmd) without a profile parameter:

export async function execCommand(cmd: string, profileName?: string): Promise<ExecResult> {
  const baseUrl = getBaseUrl();
  const token = getToken(profileName);
  // ...
}

The profileName parameter exists but is never passed by the MCP tools. So every MCP tool call uses the active profile.

What I Should Do

Here's my plan:

Add config_list_profiles — let the AI see what profiles exist (names and URLs, but not tokens)
Add config_set_active — let the AI switch profiles, but only between non-production profiles by default
Add an appId-scoped profile — store the profile used to create each app, and warn the AI when it tries to operate on an app from a different profile

That's a cross-cutting concern — it requires the server to track which environment created each app, not just the CLI. But it would have prevented this entire incident.

The Lesson: Environment Leaking Is Silent

The fix isn't just adding tools to switch profiles. It's making the environment visible in every response. When Claude calls share_publish, the response should include the environment:

✅ Published on staging.formlm.me. URL: https://formlm.me/s/abc123

Not just the URL. The environment name. So you can't accidentally share a staging link thinking it's production.

I haven't implemented this yet. But after this incident, it's the next thing on my list.

The formlm-cli supports multi-profile management with per-profile tokens and URLs. Open source at github.com/formlm/cli — try the platform at formlm.me.

Teaching an AI to Score: Reverse Items, Direction, and the Gap Between Knowing and Doing

Keo Fung | FormLM — Mon, 27 Jul 2026 02:34:03 +0000

But knowing the concept and applying it correctly when building a form are different skills. Here's what happened when I asked Claude to build a PHQ-9 depression screening form with proper scoring.

The PHQ-9 Challenge

Simple enough. No reverse items (unlike many wellbeing scales). The challenge was getting Claude to set up the scoring correctly on each field.

What Claude Got Right

Claude nailed the field structure. Nine radio fields, each with four options, each option scored 0-3:

field_add --app <appId> --id q1 --type radio --title "Little interest or pleasure in doing things" --options "Not at all:0,Several days:1,More than half:2,Nearly every day:3"

options: z.string().optional().describe('Options list, comma-separated, supports label:score'),

That one description — "supports label:score" — was enough. Claude figured out the rest.

What Claude Got Wrong

The problem wasn't the scoring. It was the total. After building the form, I asked Claude: "What's the total score range?"

Claude said: "0 to 27, with higher scores indicating more severe depressive symptoms."

This was the gap I kept hitting: the AI could build the data collection layer, but not the scoring layer.

The Reverse Item Problem

I told Claude:

"Build a WHO-5 Wellbeing Index. 5 items, 6-point scale (0=At no time to 5=All of the time). This is a positive-direction scale — higher scores mean better wellbeing."

Claude built the form correctly:

field_add --app <appId> --id w1 --type radio --title "I have felt cheerful and in good spirits" --options "At no time:0,Some of the time:1,Less than half:2,About half:3,More than half:4,All of the time:5"

All 5 items, correct scoring. Then I asked: "If someone scores all 5s, what's their wellbeing percentage?"

Claude said: "100%. Raw score 25, multiplied by 4 = 100."

Correct. Then I asked the real question: "How would you set up a reverse-scored version of this scale?"

Claude's answer: "I would negate the scores for reverse-scored items. For item 'I felt cheerful,' the options would be scored 5=At no time, 0=All of the time, reversing the direction."

Where the Confusion Comes From

The confusion is understandable. "Reverse scoring" is an ambiguous term:

Textual reversal — rewording an item negatively ("I do NOT feel cheerful")
Score reversal — inverting the numeric score for specific items (item score = max + min - raw score)
Scale direction — whether higher total = better (positive direction) or higher total = worse (negative direction)

The Fix: Explicit Instructions

I found that the key was being very explicit about what kind of "reverse" I meant. Instead of saying "add a reverse-scored item," I now say:

"Add item q9: 'I have thoughts that I would be better off dead.' This item is in the negative direction — higher scores indicate worse outcomes. Score the options 0=Not at all, 3=Nearly every day, same as other items. The reverse-scoring will be handled at the scale level, not the item level."

When I phrase it this way — explicitly saying "same as other items" and "handled at the scale level" — Claude stops trying to invert option scores and just builds the field normally.

The `field_set_property` Workaround

Since the MCP tools couldn't configure scale-level scoring at the time, I've started using field_set_property as a workaround for option-level score fixes.

If Claude builds a radio field with wrong scores, I can fix individual option scores without rebuilding the field:

# Fix the score on option index 2
field_set_property --app <appId> --id q3 --property options.2.score --value 2

This updates exactly one option's score. It's not scale-level scoring, but it lets me correct per-option scoring mistakes that Claude makes when it misunderstands the scoring direction.

The Real Lesson: Assessment Theory Is Hard

Claude knows:

What a Likert scale is ✅
What reverse scoring means (conceptually) ✅
How to format label:score options ✅
When to use radio vs scale fields ✅

Claude doesn't reliably know:

Which items should be reverse-scored vs negatively worded ❌
The difference between item-level reversal and scale-level reversal ❌
When higher scores should mean better vs worse (scale direction) ❌
How to configure dimensional scoring (not exposed via MCP) ❌

What I'm Doing About It

I'm adding more context to the MCP tool descriptions. For field_add, the options parameter now reads:

options: z.string().optional().describe('Options list, comma-separated, supports label:score. For scored scales, use label:score format. Higher scores should align with the construct direction — if the scale measures depression, higher scores = more depression.'),

And for scoring-related fields:

score: z.number().optional().describe('Question score weight. Use for items that should contribute more/less to the total. Default is 1 (equal weight).'),

The formlm-cli MCP server supports per-option scoring with label:score format. Open source at github.com/formlm/cli — try the platform at formlm.me.

From a Sentence to a Live Form: The Complete Timeline

Keo Fung | FormLM — Mon, 27 Jul 2026 02:33:01 +0000

So I timed it. Here's the complete timeline of building a personality assessment form with Claude Desktop and the formlm-cli MCP server, from first keystroke to live URL.

The Prompt (0:00)

I typed this into Claude Desktop:

"Build a Big Five personality assessment. 10 items, 5-point Likert scale (1=Strongly Disagree to 5=Strongly Agree). Two items per trait: openness, conscientiousness, extraversion, agreeableness, neuroticism. Mix some reverse-scored items in. Name it 'Big Five Personality Quick Check.' Then publish it and give me the URL."

One sentence. 43 words. No field IDs, no type names, no API parameters.

Tool Call 1: app_create (0:03)

Claude → MCP: app_create
  name: "Big Five Personality Quick Check"
  description: (omitted)

MCP → Server: POST /api/v1/mcp/exec
  cmd: "assess app create --name 'Big Five Personality Quick Check' --json"

Server → MCP: { code: 0, data: { appId: "a8f3k2", name: "Big Five Personality Quick Check" } }

Round-trip time: ~800ms. The app was created.

Elapsed: 0:04

Tool Call 2: field_schema (0:04)

Before adding any fields, Claude checked what field types were available.

Claude → MCP: field_schema

Server response: input, textarea, radio, scale, date, ...

This is a pattern I've seen consistently — Claude always checks the schema before adding fields. It's reading the available types like a developer reading API docs.

Round-trip time: ~600ms.

Elapsed: 0:05

Tool Call 3: field_config (0:05)

Claude wanted to know what properties a scale field accepts.

Claude → MCP: field_config
  type: "scale"

Server response: { min, max, title, required, score, ... }

It found min and max — exactly what it needed for a 5-point Likert scale.

Round-trip time: ~500ms.

Elapsed: 0:06

Tool Calls 4-13: field_add × 10 (0:06 - 0:22)

Claude added 10 scale fields, one for each item. Here's the first one:

Claude → MCP: field_add
  appId: "a8f3k2"
  id: "o1"
  type: "scale"
  title: "I have a vivid imagination"
  min: 1
  max: 5

Server response: { code: 0, data: { fieldId: "o1", ... } }

Each field_add call took about 1.5 seconds (including Claude's thinking time between calls). 10 fields × 1.5s = 15 seconds.

Here's the full field list Claude generated:

ID	Trait	Question
o1	Openness	I have a vivid imagination
o2	Openness	I prefer variety over routine
c1	Conscientiousness	I am organized and pay attention to details
c2	Conscientiousness	I tend to leave things until the last minute (reverse)
e1	Extraversion	I feel energized around other people
e2	Extraversion	I prefer quiet, solitary activities (reverse)
a1	Agreeableness	I trust people easily
a2	Agreeableness	I can be cold and uncaring (reverse)
n1	Neuroticism	I worry about things more than others
n2	Neuroticism	I stay calm under pressure (reverse)

Elapsed: 0:22

Tool Call 14: field_list (0:22)

Before publishing, Claude called field_list to verify all 10 fields were there.

Claude → MCP: field_list
  appId: "a8f3k2"

Server response: [o1, o2, c1, c2, e1, e2, a1, a2, n1, n2]

10 fields confirmed. This is a verification step — Claude checking its own work before moving to the publish phase.

Round-trip time: ~700ms.

Elapsed: 0:23

Tool Call 15: share_publish (0:23)

Claude → MCP: share_publish
  appId: "a8f3k2"

Server response: { code: 0, data: { published: true, formDay: 3650000 } }

The form was live. The formDay: 3650000 is the hardcoded "never expires" default — the AI doesn't need to decide on an expiry date.

Round-trip time: ~900ms.

Elapsed: 0:24

Tool Call 16: share_url (0:24)

Claude → MCP: share_url
  appId: "a8f3k2"

Server response: "https://formlm.me/s/a8f3k2"

The MCP server constructs the full URL by combining the server's base URL with the relative path returned by the API. Claude received the URL and displayed it to me.

Round-trip time: ~500ms.

Elapsed: 0:25

The Total

From first keystroke to live URL: 25 seconds.

The form was live, accessible, and accepting responses. 10 fields, 5-point Likert scale, all 10 items present and correctly ordered.

What's Missing

The form worked, but it wasn't a complete assessment. Here's what was missing:

No dimension labels. The fields have IDs like o1, o2, c1, c2 — meaningful to me, but not to the system. The dimension mapping (o1+o2 = openness score) needs to be configured separately.

No report template. When someone completes the assessment, they get a "thank you" page, not a personality profile. The report generation is a separate module.

The Tool Call Breakdown

Phase	Tool Calls	Time	Round-trips
App creation	1 (app_create)	4s	1
Schema discovery	2 (field_schema, field_config)	2s	2
Field creation	10 (field_add ×10)	16s	10
Verification	1 (field_list)	1s	1
Publishing	2 (share_publish, share_url)	2s	2
Total	16	25s	16

16 round-trips in 25 seconds. That's roughly 1.5 seconds per round-trip, which includes network latency, server processing, and Claude's thinking time between calls.

What Slowed It Down

Nothing, really. This was a clean run — no errors, no retries, no duplicate fields. The 25 seconds was almost entirely Claude thinking and the network round-tripping.

The Honest Answer

"How long does it take?" — 25 seconds for a 10-field form, from sentence to live URL. That's the honest answer.

But 25 seconds gets you a form. A real assessment — with scoring, reverse items, dimensions, and a report — used to take another 5-10 minutes of manual work.

The formlm-cli MCP server can build and publish forms in under 30 seconds. Open source at github.com/formlm/cli — try it at formlm.me.

When Your AI Duplicates Every Field: A Story About Idempotency

Keo Fung | FormLM — Fri, 24 Jul 2026 08:55:00 +0000

I had a form with 12 fields. Then my Claude Desktop session crashed. I restarted it, asked Claude to "continue building the form," and ended up with 24 fields. Half of them were exact duplicates — same IDs, same titles, same types. The form rendered with two "Email" fields, two "Name" fields, and two of every radio button.

This is the story of how field_add isn't idempotent, why that matters when an AI is driving, and the pattern I adopted to fix it.

What Happened

Here's the sequence:

I asked Claude to build a customer satisfaction survey with 12 fields
Claude created the app, added 8 fields, and then the session timed out
I started a new session and said "continue building the customer satisfaction survey"
Claude called app_list, found the existing app, and started adding fields again
It re-added all 12 fields — including the 8 that were already there

The result was a form with 20 fields total (8 original + 12 duplicate). The duplicate field IDs were auto-suffixed by the server (email became email, email_1, email_2...). To the user filling out the form, it looked like the form had been designed by someone who really, really wanted your email address.

Why `field_add` Isn't Idempotent

Here's the thing about field_add — it's a create operation, not an upsert. If you call it twice with the same field ID, the server creates two fields. The field ID isn't a unique constraint; it's a label.

Looking at the MCP tool definition:

server.tool('field_add', 'Add a new field', {
  appId: z.string().describe('App ID'),
  id: z.string().describe('Field ID'),
  type: z.string().default('input').describe('Field type (default: input)'),
  title: z.string().optional().describe('Field title / question text'),
  // ...
}, async (params) => {
  let cmd = `assess form add --app ${params.appId} --id ${params.id} --type ${params.type}`;
  // ...
});

There's no check for "does this field ID already exist?" The tool just sends the add command and the server creates a new field. If a field with that ID already exists, the server either appends a suffix or creates a duplicate.

From a CLI perspective, this is fine. A human running formlm-cli field add knows whether they've already added a field. They don't need the CLI to check for them.

From an AI perspective, this is a problem. The AI doesn't remember what it did in a previous session. It doesn't check whether fields exist before adding them. It just executes the plan, and if the plan says "add 12 fields," it adds 12 fields — regardless of whether 8 of them are already there.

The Fix: Find Before Add

The pattern I now enforce is: always check before adding. Before calling field_add, call field_find to see if a field with that ID already exists:

# Step 1: Check if the field already exists
field_find --app <appId> --id email

# If the response is "field not found", proceed with add:
field_add --app <appId> --id email --type input --title "Email" --required

# If the field already exists, skip the add (or update it instead)

In practice, I instruct Claude to batch this: call field_list once to get all existing fields, then only add the ones that are missing.

# Get the current state
field_list --app <appId>
# Response: [name, email, phone, rating]

# Claude now knows which fields exist and only adds the missing ones
# It needs to add: comments, feedback_source
field_add --app <appId> --id comments --type textarea --title "Additional comments"
field_add --app <appId> --id feedback_source --type radio --options "Search:1,Social:2,Friend:3"

This is the read-before-write pattern again (same as find-then-set from the previous article), but applied to creation instead of updates.

Why Not Just Make `field_add` Idempotent?

I considered making field_add an upsert — if the field exists, update it; if not, create it. But that has its own problems:

Silent overwrites. If the AI accidentally re-adds a field with different options, the upsert would silently overwrite the existing field's options. No error, no warning — just changed data.
No explicit "create" semantics. Sometimes you want to fail if a field already exists — like when you're setting up a new form and a duplicate field ID indicates a bug in your plan.
The update path is different. Creating a field and updating a field go through different server-side validation. Merging them into one operation muddies the API.

So I kept field_add as a pure create operation and moved the idempotency check to the caller. The AI is responsible for checking before adding.

The Idempotency Wrapper

In my system prompt for Claude, I now include this instruction:

"Before adding any field, call field_list to get the current field set. Only add fields whose IDs don't already exist in the form. If a field already exists and needs changes, use field_update or field_set_property instead of field_add."

This is a soft constraint — it relies on Claude reading and following the instruction. In practice, it works about 90% of the time. The other 10%, Claude skips the field_list call and goes straight to field_add, usually when it's "in a hurry" (i.e., the context window is getting full and it's trying to save tokens).

The hard constraint would be making the server reject duplicate field IDs. But as I mentioned, field IDs aren't unique constraints — they're labels. Making them unique would break existing forms that intentionally use the same ID across different field types (don't ask).

The Deeper Lesson: AI Agents Need Idempotency Everywhere

This isn't just about field_add. Every create operation in an AI-accessible API needs an idempotency story:

app_create — what if the AI creates the same app twice? Currently it creates two apps with the same name. I should probably check for existing apps with the same name before creating.
share_publish — what if the AI publishes the same form twice? Currently it's idempotent (publishing an already-published form is a no-op). This one got it right.
auth_login — what if the AI logs in twice? Currently it overwrites the existing token. Also fine.

The pattern I now follow for any create operation:

Check-first: Provide a read operation that lets the AI verify whether the resource already exists
Document it: In the schema description, explicitly say "Check with [X] before calling this to avoid duplicates"
Soft idempotency: If the operation is called with the same parameters, make it a no-op rather than creating a duplicate, where possible

The Cleanup

Cleaning up the duplicated form was painful. I had to:

Call field_list to see all 20 fields
Identify the duplicates (fields with suffixed IDs like email_1, email_2)
Call field_remove for each duplicate
Re-order the remaining fields with field_move

Twelve field_remove calls. Each one was a round-trip to the server. It took longer to clean up than it took to create the mess.

After that experience, I wrote a cleanup script that detects and removes duplicate fields by comparing titles and types. It's not part of the CLI — it's a one-off shell script I keep in my snippets folder. If this happens often enough, I might add a field_dedup command to the CLI. But for now, the find-before-add pattern is enough.

The formlm-cli MCP server documents the find-before-add pattern in tool descriptions. Open source at github.com/formlm/cli — try the platform at formlm.me.

The Boolean Trap: When Your AI Literally Cannot Say "False"

Keo Fung | FormLM — Thu, 23 Jul 2026 09:31:48 +0000

This one took me two hours to debug. Two hours of staring at a form where every single field was marked "required" — even the ones I'd explicitly told Claude to make optional. The AI wasn't ignoring me. It physically could not express what I was asking for.

Here's how a TypeScript boolean type and a commander.js flag convention teamed up to make false an impossible value.

The Setup

I was building a customer feedback form. Most fields were required (name, email, rating), but a few were optional — a text field for "additional comments" and a radio for "how did you hear about us." Standard stuff.

I told Claude:

"Create a feedback form. Name and email are required. The comments field is optional. The 'how did you hear about us' field is optional."

Claude went to work. It created the app, added the name and email fields with required: true, and then... this happened.

What Claude Did

For the required fields, Claude called field_add with required: true. Clean, correct:

{
  "appId": "app_abc123",
  "id": "email",
  "type": "input",
  "title": "Email address",
  "required": true
}

For the optional fields, Claude called field_add and simply... didn't include the required parameter:

{
  "appId": "app_abc123",
  "id": "comments",
  "type": "textarea",
  "title": "Additional comments"
  // no "required" field at all
}

On the surface, this looks fine. The field was created. The form rendered. But every single field — including the "optional" ones — showed up with a red asterisk. Every field was required.

The Root Cause

Here's the thing. In the MCP schema, required is defined as:

required: z.boolean().optional().describe('Mark as required'),

z.boolean().optional() means the parameter accepts true, false, or undefined (not provided). When Claude wants to make a field optional, it has two options:

Pass required: false — explicitly set it to false
Don't pass required at all — leave it undefined

Claude chose option 2. And here's where the problem starts.

When required is undefined, the CLI builds the command string like this:

const required = normBool(opts.required);
if (required !== undefined) cmd += ` --required=${required}`;

If required is undefined, the --required flag is never added to the command. The server receives the assess form add command without any --required parameter.

And the server's default behavior? When no --required flag is present, it defaults to true.

So "I didn't say required" became "required by default." Every optional field silently became mandatory.

Why Claude Didn't Pass `false`

I tested this. I explicitly told Claude: "Make the comments field NOT required. Set required to false."

Claude tried. It passed required: false in the MCP parameters. But here's what happened on the CLI side:

The normBool function in field.ts handles three cases:

function normBool(v: unknown): boolean | undefined {
  if (v === undefined) return undefined;
  if (typeof v === 'boolean') return v;
  return v === 'true' || v === '1';
}

When the MCP server receives required: false (a proper boolean), it passes it through and builds:

assess form add --app <appId> --id comments --required=false --json

This should work. And in my local testing, it did work. But in Claude Desktop, something weirder happened — Claude kept omitting the parameter instead of passing false. When I checked the conversation log, Claude's reasoning was:

"The field is optional, so I don't need to set required."

Claude was treating "optional" as "don't set the flag" rather than "set the flag to false." From a human perspective, that makes sense — if something is optional, you don't mention it. But from a programmatic perspective, "not setting it" and "setting it to false" are different operations when the default is true.

The Real Problem: Defaults That Bite

The deeper issue is the server-side default. When field_add is called without --required, the server treats it as required=true. That's a reasonable default for form fields — most fields in most forms are required.

But it creates a trap: the only way to make a field optional is to explicitly pass required=false. And AI agents are bad at explicitly passing false for boolean parameters because:

They've been trained on tons of code where "optional" means "don't pass it"
The schema says .optional(), which reinforces the "just don't include it" pattern
Passing false feels redundant when the field isn't required — why state the obvious?

The result is a silent data corruption: fields that should be optional become required, and nobody notices until a user complains that they can't submit the form without filling in the "optional" comments field.

The Fix

I tried three approaches before settling on one.

Attempt 1: Change the server default. Make required default to false when not specified. This fixed the AI problem but broke every existing form build — now Claude had to explicitly pass required: true for every required field, and it kept forgetting. Same problem, opposite direction.

Attempt 2: Remove the default entirely. Force the parameter to be required. This made the schema non-optional, which meant Claude had to always pass required: true or required: false. It worked, but it made every field_add call noisier. And for fields where required-ness doesn't matter (like a description field), it was confusing.

Attempt 3: Fix the schema description. This is what I went with. I changed the .describe() string to be explicit about the default:

required: z.boolean().optional().describe('Mark as required. Defaults to true if omitted. To make a field optional, explicitly set this to false.'),

That one sentence — "To make a field optional, explicitly set this to false" — was the instruction Claude needed. After this change, Claude started passing required: false for optional fields instead of omitting the parameter.

It's a band-aid, not a real fix. The real fix is making the server's default behavior match the common-sense expectation: if you don't specify required, the field should be optional, not required. But changing a default that existing forms depend on is a migration I'm not ready to do.

The `normBool` Safety Net

The normBool function itself is fine — it correctly handles true, false, the string "false", and undefined. The problem was never in the function. It was in the gap between:

What the schema says (.optional() → "you can skip this")
What the server does (defaults to true when skipped)
What the AI assumes (skipping = not required)

Three layers, each reasonable in isolation, combine to produce a bug where "optional" silently means "required."

The Pattern I Now Follow

For any boolean parameter where false is a meaningful value, I:

Make the description explicit about the default. If the default is true, say so. If omitting it means true, say that explicitly.
Test the false path. Every boolean parameter gets tested with true, false, and omitted. The omitted case is the one that bites you.
Don't rely on "omit means false." In a world where an AI is reading your schema and making decisions, the safest approach is to make every state explicit. If false is a valid and important value, make the parameter required, not optional.

The boolean trap isn't unique to my CLI. It's a general problem with AI-accessible APIs: optional booleans with non-null defaults are a footgun. The AI's instinct is to omit rather than to explicitly negate, and if your default doesn't match "omitted = the safe option," you'll get silent bugs.

The formlm-cli MCP server has explicit descriptions for all boolean parameters. Open source at github.com/formlm/cli — try the platform at formlm.me.

MCP Tools, One Weekend: Stress-Testing an AI-Accessible Form Builder

Keo Fung | FormLM — Wed, 22 Jul 2026 09:02:27 +0000

I have 20 MCP tools in formlm-cli. On paper they cover the full lifecycle: auth, app management, field CRUD, and sharing. But "covers the lifecycle" and "actually works when you throw a complex real-world task at an AI agent" are different things.

So I spent a weekend trying to build a genuinely complex form — an enterprise onboarding assessment with multiple field types, conditional logic, and scoring — using only Claude Desktop and the MCP tools. Here's what worked, what didn't, and what surprised me.

The Task

Build an "Enterprise Onboarding Readiness Assessment" with:

5 text/input fields (employee name, department, role, manager, start date)
2 radio fields with scoring (technical proficiency, communication level)
3 scale fields (1-10) for confidence dimensions
1 textarea for open feedback
Field ordering (specific sequence)
Published with a shareable URL

Not the most complex form in the world, but complex enough to exercise most of the toolset. I wanted to hit at least 15 of the 20 tools in a single session.

Saturday: Building the Form

Tools used: app_create, field_schema, field_config, field_add (×11)

I started with a prompt describing the full form. Claude's first move was app_create — clean, no issues.

Then it did something I've come to expect: it called field_schema before adding any fields. It wanted to see the available types. The response listed input, textarea, radio, scale, date, and others. Claude picked date for the start date field instead of input, which was smart — it means the form will show a date picker instead of a text box.

The field_config --type radio call came next. Claude wanted to know what properties a radio field accepts. The response showed options (comma-separated with label:score format) and required. Claude then added the radio fields with properly scored options:

field_add --app <appId> --id tech --type radio --options "Beginner:1,Intermediate:2,Advanced:3,Expert:4" --required
field_add --app <appId> --id comm --type radio --options "Needs improvement:1,Meets expectations:2,Exceeds expectations:3" --required

The scale fields came next:

field_add --app <appId> --id conf_tools --type scale --min 1 --max 10 --title "How confident are you with the tools?"
field_add --app <appId> --id conf_team --type scale --min 1 --max 10 --title "How confident are you with your team?"
field_add --app <appId> --id conf_process --type scale --min 1 --max 10 --title "How confident are you with the processes?"

Eleven field_add calls. All succeeded. No issues.

Tools used: field_list, field_move

Here's where I hit my first snag. The fields were added in the right order, but I wanted the textarea (open feedback) to appear last, after the scale fields. Claude had added it in the middle.

Claude called field_list to see the current ordering:

field_list --app <appId>
# Response: [name, department, role, manager, start_date, tech, comm, conf_tools, conf_team, conf_process, feedback]

Then field_move:

field_move --app <appId> --id feedback --pos 11

Worked. The feedback field moved to the end.

This was a good test of field_move — it's a tool that seems simple but has edge cases. What if the position is out of range? What if the field doesn't exist? I tested both scenarios manually afterward. Out-of-range position silently clamps to the end. Non-existent field ID returns an error. Reasonable behavior.

Tools used: field_find

I asked Claude to check the scoring on the tech field. It called field_find:

field_find --app <appId> --id tech
# Response: { id: "tech", type: "radio", options: [{label: "Beginner", score: 1}, ...] }

The find response was clean and structured. Claude could read the scores and confirm they were correct.

One thing I noticed: the field_find response includes a lot of metadata — field type, validation rules, placeholder text, etc. For a human reading the JSON, it's fine. For an AI parsing it, the extra fields add tokens but don't cause confusion. Claude had no trouble extracting what it needed.

Sunday: Publishing and Cleanup

Tools used: share_publish, share_url, share_query

Publishing was the easiest part. One share_publish call, one share_url call. Done.

I tested share_query to check the publish status:

share_query --app <appId>
# Response: { published: true, formType: "all", formPerm: 1, formDay: 3650000 }

The formDay: 3650000 is the "never expires" default I hardcoded in the MCP tool. That's intentional — the AI doesn't need to decide expiry dates for public forms. Safe defaults matter.

Tools used: share_unpublish, app_remove

After testing, I unpublished and deleted the app to clean up. Both worked without issue. The app_remove is permanent — no confirmation prompt, no undo. That's slightly terrifying for an AI-accessible tool, but the whitelist on the backend would block a clear command (which deletes ALL apps). Individual deletes are safe as long as the AI targets the right app ID.

What I Didn't Use

Out of 20 tools, I used 11 in this session:

auth_login, auth_status — skipped, I was already logged in
app_list, app_get, app_update — didn't need to list or update apps
field_update, field_remove, field_set_property — didn't need to modify or remove fields

That's fine. Not every form build needs all 20 tools. The test was whether the tools I did need worked correctly when chained together by an AI agent. They did.

What Broke

Two things went wrong over the weekend:

1. Claude forgot the app ID mid-session. After adding about eight fields, Claude lost track of the app ID it had created at the start. It tried to use field_add without the --app parameter. The error message was clear ("Missing required option: --app"), and Claude recovered by calling app_list to find the app. But it cost three extra API calls and broke the flow.

Fix idea: The MCP server could maintain a "current app" state, similar to how git remembers your current branch. app_get could set the active app, and subsequent field commands could default to it.

2. The --filter parameter on field_find returned too many results. When Claude searched for "confidence" (trying to find the scale fields), the filter matched on field titles and returned all three confidence fields plus the textarea (which mentioned "confident" in its placeholder text). Not wrong, but noisier than expected. Claude handled it by reading the response and picking the right fields, but the filter could be more precise — maybe matching on field ID or key instead of title text.

The Scorecard

Tool Category	Tools	Used This Weekend	Worked?
Auth	2	0	N/A (already logged in)
App	5	2 (create, delete)	✅
Field	9	9 (all of them, across the session)	✅ with minor issues
Share	4	4 (all of them)	✅
Total	20	15	15/15 passed

Every tool I used worked. The issues were with Claude's context management (forgetting the app ID) and search precision (filter returning too many results), not with the tools themselves.

The takeaway: 20 tools was enough coverage for a complex form build. In v0.2.0, the architecture was streamlined to 6 layered tools — formlm_generate handles the entire build in one call, while formlm_exec provides direct access for fine-grained control. The gaps aren't in the toolset — they're in the workflow ergonomics (state management, search precision). Those are fixable without adding new tools.

All MCP tools are available in formlm-cli. Open source at github.com/formlm/cli — try the platform at formlm.me.

The find-then-set Pattern: How I Stopped My AI Agent From Breaking Fields

Keo Fung | FormLM — Tue, 21 Jul 2026 10:38:08 +0000

It started with a radio field that lost its scores. Claude was trying to update one option's label on a question, and somehow every score on that field vanished. The form still worked — it just stopped scoring anything. Nobody noticed until a respondent completed the assessment and got a zero.

Here's how it happened, and the pattern I adopted to make sure it never happens again.

The Incident

I had a radio field with three options:

field: q3
type: radio
options:
  - "Satisfied:3"
  - "Neutral:2"
  - "Unsatisfied:1"

I asked Claude: "Change the 'Neutral' label to 'Okay' on field q3."

Claude called field_update:

field_update --app <appId> --id q3 --options "Satisfied:3,Okay:2,Unsatisfied:1"

Looks fine, right? It passed all three options, keeping the scores intact. The label changed. The scores stayed. Problem solved.

Except that's not what happened every time. In a different session, Claude decided to be "efficient" and only pass the option it wanted to change:

field_update --app <appId> --id q3 --options "Okay:2"

The --options parameter in field_update replaces the entire options list. It doesn't merge. So that command wiped out "Satisfied:3" and "Unsatisfied:1" and left the field with a single option: "Okay:2."

The form now had one radio button instead of three. The other two options were gone. And since the remaining option had a score of 2, every respondent who clicked it got 2 points — for a question that used to have a range of 1 to 3.

The respondent who triggered the discovery answered "Okay" on this question and nothing else — because there was nothing else to answer. They got a 2 where they should've gotten context-dependent scoring.

Why This Happened

The field_update command is documented as "Update a field." From an AI's perspective, that's a reasonable tool to use when you want to change something about a field. The schema says:

options: z.string().optional().describe('Options list — replaces existing options')

The description says "replaces existing options." That's accurate. But AI agents don't always read carefully when they're in the middle of a task. They see "update field" and "options" and they pass the option they want to change. The "replaces" part gets lost in the context window.

This isn't Claude's fault. It's a UX problem with the tool design. field_update is a blunt instrument — it replaces whatever you pass. If you only pass --options, you get a new options list. If you only pass --title, you get a new title and everything else stays. But the options parameter is particularly dangerous because it's a list — partial replacement of a list silently destroys data.

The Fix: find-then-set

I added field_set_property specifically for this scenario. Instead of replacing the whole field, it updates a single property at a precise path:

server.tool('field_set_property', 'Fine-grained field property update', {
  appId: z.string().describe('App ID'),
  id: z.string().optional().describe('Field ID'),
  key: z.string().optional().describe('Field Key'),
  property: z.string().describe('Property path (e.g. options.1.score)'),
  value: z.string().describe('Property value'),
}, async (params) => {
  let cmd = `assess form set-property --app ${params.appId}`;
  if (params.id) cmd += ` --id ${params.id}`;
  if (params.key) cmd += ` --key ${params.key}`;
  cmd += ` --property ${params.property} --value "${params.value}"`;
  const r = await execCommand(cmd);
  // ...
});

Now, to change the "Neutral" label without touching scores, the AI uses:

field_set_property --app <appId> --id q3 --property options.1.label --value "Okay"

This updates exactly one property — the label of option index 1. Nothing else changes. The scores, the other options, the field type — all untouched.

But there's a catch. The AI needs to know that option index 1 is "Neutral" before it can update it. That's where field_find comes in.

The Full Pattern

The pattern I now enforce in my prompts is: find first, then set property.

Call field_find to get the current field state
Identify the exact property path that needs changing
Call field_set_property with that precise path

# Step 1: Find the field
field_find --app <appId> --id q3

# Response includes:
# options: [
#   { label: "Satisfied", score: 3 },
#   { label: "Neutral", score: 2 },
#   { label: "Unsatisfied", score: 1 }
# ]

# Step 2: I want to change options[1].label

# Step 3: Set the property
field_set_property --app <appId> --id q3 --property options.1.label --value "Okay"

This is more API calls (find + set vs. a single update). But it's safe. The AI can't accidentally destroy data because it's only touching one property at a time, and it saw the current state before making the change.

When to Use update vs. set-property

I still use field_update for legitimate full replacements — when I actually want to replace the entire options list, or change the field type, or update multiple properties at once. The rule I follow:

Use field_update when you're replacing the entire structure of a field (new type, new options list, new title — a deliberate full replacement)
Use field_set_property when you're tweaking one thing (one option's label, one option's score, one property)

The AI needs to be told this explicitly. In my system prompt, I now include: "When changing a single property of a field, always use field_set_property with a property path. Use field_update only for full field replacement."

That instruction, combined with the find-first pattern, has eliminated the "disappearing options" bug entirely. The cost is one extra API call per update. The benefit is not silently destroying form data.

The Bigger Lesson

When you design tools for AI agents, you have to think about what happens when the AI takes you literally but incompletely. "Update the label" doesn't mean "replace the entire options list with just the new label." But if the only tool available is a blunt "update" command, that's what can happen.

Fine-grained tools (set-property) combined with a read-before-write pattern (find before set) make AI interactions safer. Not because the AI is malicious — but because partial updates on complex objects are inherently dangerous when the tool doesn't distinguish between "replace this one thing" and "replace everything."

The find-then-set pattern isn't new. It's how most database transactions work. But in the MCP world, where tools are the API and the AI is the client, it's easy to forget that your tools need the same safety patterns your database does.

The formlm-cli MCP server includes field_set_property (accessible via formlm_exec) specifically for this use case. Open source at github.com/formlm/cli — try it at formlm.me.

DEV Community: Keo Fung | FormLM

50 Forms in 30 Days: What I Learned Building Assessment Forms With an AI Agent

The Numbers

The 7 Bugs

1. The field_update Options Wipe (Critical)

2. The Staging/Production Profile Leak (High)

3. The assess form clear Attempt (High)

4. The Duplicate Fields (Medium)

5. The Boolean Trap (Medium)

6. The Reverse Scoring Confusion (Low)

7. The Forgotten App ID (Low)

What Worked Well

Schema-Driven Discovery

Natural Language to Field Structure

Safe Defaults

The Whitelist

What Didn't Work

State Management Across Sessions

Complex Scoring Configuration

Boolean Parameters

Environment Awareness

Patterns That Emerged

What I'd Do Differently

The Meta-Lesson

What's Next

The Whitelist That Saved My Production Data: A Security Boundary Story

The Setup

What Claude Tried

The Whitelist

Why Subcommand-Level, Not Module-Level

What Would Have Happened Without the Whitelist

The Extract Function

Why This Works for AI Agents

What I Learned

When Your AI Deploys to the Wrong Server: Multi-Profile Management Gone Wrong

What Happened

Why the MCP Tools Don't Switch Profiles

The Token Priority Trap

The Config File Permissions

How I Fixed It

The Safer Approach: Per-Command Profile

What I Should Do

The Lesson: Environment Leaking Is Silent

Teaching an AI to Score: Reverse Items, Direction, and the Gap Between Knowing and Doing

The PHQ-9 Challenge

What Claude Got Right

What Claude Got Wrong

The Reverse Item Problem

Where the Confusion Comes From

The Fix: Explicit Instructions

The field_set_property Workaround

The Real Lesson: Assessment Theory Is Hard

What I'm Doing About It

From a Sentence to a Live Form: The Complete Timeline

The Prompt (0:00)

Tool Call 1: app_create (0:03)

Tool Call 2: field_schema (0:04)

Tool Call 3: field_config (0:05)

Tool Calls 4-13: field_add × 10 (0:06 - 0:22)

Tool Call 14: field_list (0:22)

Tool Call 15: share_publish (0:23)

Tool Call 16: share_url (0:24)

The Total

What's Missing

The Tool Call Breakdown

What Slowed It Down

The Honest Answer

When Your AI Deploys to the Wrong Server: Multi-Profile Management Gone Wrong

What Happened

Why the MCP Tools Don't Switch Profiles

The Token Priority Trap

The Config File Permissions

How I Fixed It

The Safer Approach: Per-Command Profile

What I Should Do

The Lesson: Environment Leaking Is Silent

Teaching an AI to Score: Reverse Items, Direction, and the Gap Between Knowing and Doing

The PHQ-9 Challenge

What Claude Got Right

What Claude Got Wrong

1. The `field_update` Options Wipe (Critical)

3. The `assess form clear` Attempt (High)

The `field_set_property` Workaround

The `field_set_property` Workaround

Why `field_add` Isn't Idempotent

Why Not Just Make `field_add` Idempotent?

Why Claude Didn't Pass `false`

The `normBool` Safety Net