How should a developer replace an incomplete agent-registration card with a conversation that establishes what the agent should actually do? Shelly’s mid-July–early August 2026 work combined conversational slot-filling, LLM-led follow-up questions, and deterministic revalidation of proposed values. [M] Where did the original flow fail, how did Tier 3 change registration, and which limits still governed confirmation and execution?
Where did registration lose the user’s intent?
In the v7.5.0 / v7.5.5 period, the relevant conditions included missing registration fields, an unclear task, and recurring requests with a delayed start. [M] For @agent natural-language requests missing schedule, notificationTrigger, or outputPath, the 2026-07-14 change introduced one follow-up question at a time in chat instead of an AgentConfirmCard containing blank fields for manual completion. [M1]
The distinction between an incomplete task and an unclear task mattered in the on-device case recorded on 2026-07-24. [M1] The failure and correction followed this chain:
| Before the fix | After the fix |
|---|---|
| “Please prepare for tomorrow” described an outcome without a concrete task, but registration immediately asked when to execute it. [M1] | The existing isLowConfidenceAgentDraft → extractAgentFieldsWithLlm path gained taskClear and clarifyingQuestion so extraction could establish what to do before asking when. [M1] |
That correction reused the first-parse fallback already connected in hooks/use-ai-pane-dispatch.ts rather than adding a new trigger. [M1] Validation applied a strict boolean check to taskClear and a length cap to the question string, following the existing fail-closed treatment of extracted fields. [M1]
A separate on-device request, translated as “check the news every morning starting around next week,” exposed a scheduling problem: registration discarded the delayed-start phrase, scheduled execution from tomorrow, and left the phrase in the derived agent name or prompt. [M1] The scheduler had lacked a deferred-start-date concept. [M1]
Agent.startNotBefore added that boundary as epoch ms, and the next-fire calculation moved its anchor to max(now, startNotBefore). [M1] This preserved the existing calculation of the soonest cron occurrence at or after the anchor, with the deferred-start effect ending once its date passed. [M1]
How did Tier 3 change the design?
The 2026-08-02 design divided registration into three paths: Tier 1 resolved every field with high confidence through parseAgentNL, Tier 2 asked a fixed-template question when one field was missing, and Tier 3 let the LLM lead an ambiguous registration conversation in its own words. [M1] Existing parser, slot-filling, and narrow-extraction code remained authoritative for Tier 1/2 and supplied hints and fallback paths for Tier 3. [M1]
For conversation state, the design extended PendingAgentSession with phase: 'llm-conversation' alongside 'slot-fill' and 'await-confirm'. [M1] It reused the existing ChatMessage[] history and planned extraction equivalent to toOpenAIHistory(), following the normal chat approach of resending the full history. [M1]
The proposed exit was equally important: mergeConversationalExtractionIntoDraft() converted the final LLM proposal into ParsedAgentDraft before joining presentDraftForConfirmation(). [M1] The design kept AgentConfirmCard, AgentChatConfirm, confirmAgentDraft, and the capability broker unchanged, preserving the human’s final Confirm tap and runtime dispatch boundary. [M1]
The queue’s UX direction called for plain-text confirmation in chat rather than structured confirmation cards. [M] By contrast, the dated 2026-08-02 design retained the existing confirmation UI, making conversational collection and final confirmation distinct parts of the design. [M1]
The validation boundary also changed: the LLM could propose broadly, while connector and secret references required checks against existing entries. [M1] The queue paired that approach with cron reparsing and substring matching against original input to prevent hallucinated proposals. [M]
For high-risk strings, the design proposed requireVerbatimSubstringMatch() so a dangerous value had to exist in the conversation’s raw text before acceptance. [M1] The later implementation narrowed that check specifically to an exact, case-sensitive substring of text the user actually typed during the session. [M1]
What changed in the dated implementation?
The dated implementation milestones established different defaults and action boundaries:
- 2026-08-01 — 7e33f4c0e connected Tier 3 Phase 0/1 as an opt-in with agentConversationalRegistrationEnabled default false; LLM-authorable actions were limited to draft/notify. [M1]
- 2026-08-03 — fdc185e76 added webhook/cli authoring behind agentConversationalHighRiskActionsEnabled, default false, with both registration and high-risk flags required. [M1]
- 2026-08-03 — ab8b00909 changed agentConversationalRegistrationEnabled to default true and connected conversational steps[] to orchestrationSteps and detectApiCallSteps(). [M1]
The orchestration merge preserved existing content on rejection and used truncate-not-drop behavior on overflow, while granting no new action types or connector access. [M1] In the high-risk implementation, app-act remained outside scope, and AgentAction.appActRecipeId was schema-only with no dispatch logic reading it. [M1]
The queue specified the provider sequence as Cerebras → Groq → local → fixed-question fallback. [M] The earlier design described cloud-first calls through fast APIs such as Groq/Cerebras, with local inference last and immediate return to Tier 2 if all providers failed. [M1]
Official structured-output behavior concerned a separate boundary: Cerebras accepted response_format with json_schema, and strict: true enabled token-level constrained decoding for its supported JSON Schema subset. [1] API version 2 enforced an object root and additionalProperties: false on every object, with strict-mode limits of 10 levels of nesting and 500 object properties. [1]
Those API constraints governed response structure, while Shelly’s transcript matching and scheduling checks governed acceptance of the proposed registration values. [M1]
Other approaches
Cerebras strict structured output addressed malformed or schema-invalid responses by constraining generation, whereas strict: false or omission treated the schema as a hint and could yield missing fields, additional fields, or incorrect types. [1] Its supported subset excluded string pattern and string format validation, including formats such as email, date-time, and uuid. [1]
Groq supported strict structured outputs for openai/gpt-oss-120b, while strict: false was its default best-effort behavior. [2] Strict mode required every field to be required and every object to set additionalProperties: false, but structured outputs provided schema compliance rather than semantic accuracy. [2] Streaming and tool use were outside Groq’s supported Structured Outputs behavior. [2]
What to check
- If a conversation exhausts its turns or produces an unusable response, check the fallback transition: the 2026-08-01 implementation imposed a 5-turn cap and returned to existing slot-filling for LLM unavailability or an unparseable response. [M1]
- If registration depends on cloud availability, check the unreachable-provider case: the 2026-08-03 default-on implementation failed closed to Tier 2 when no LLM was reachable, with the existing benign fallback notice as the user-visible effect. [M1]
- If an initial Tier 3 proposal attempts to promote an action or supply infrastructure references, check the accepted fields: Phase 0 excluded webhook/cli/social-post promotion, connector ids, hosts, and secrets from model exposure and acceptance. [M1]
- If Groq requests fail while configuring openai/gpt-oss-20b or openai/gpt-oss-120b, check reasoning_effort: supported values were low, medium, and high, with medium the default and unsupported values rejected with a 400 response. [3]
The stated implementation limit remained action-specific: the high-risk exception covered webhook and cli under their independent gate. [M1]
Sources
- [1] Structured Outputs
- [2] Structured Outputs
- [3] API Reference
- [M1]
dev/ep26.md(author's own material)
AI disclosure: this post was researched, written and reviewed by AI tools.
Publishing: a post that passes automated checks may be published automatically, before the author reads it.
Top comments (0)