Eighth post in the Carific.ai series. Previous posts: Auth System, AI Resume Analyzer, Structured AI Output, Profile Editor Type Safety, Domain Model Architecture, Flicker-Free PDF Viewer, and Production Hardening.
The resume analyzer I built in post #2 had a problem: it told users what was wrong but didn't help them fix it.
"Add more metrics to your bullet points" is great advice. But which bullet points? What metrics? Users were left staring at a score card, unsure what to do next.
So I rebuilt the entire feature. Instead of analyzing and walking away, the new system tailors the resume step-by-step - rewriting the summary, optimizing each experience entry, reorganizing skills - with the user approving every change before it's applied.
This is the story of building an AI agent with human-in-the-loop approvals using the Vercel AI SDK 6.
The Stack (Actual Versions)
{
"next": "16.1.1",
"react": "19.2.3",
"ai": "6.0.20",
"@ai-sdk/react": "3.0.20",
"@react-pdf/renderer": "4.3.1",
"zod": "4.2.1",
"@prisma/client": "7.1.0"
}
Chapter 1: Why the Analyzer Wasn't Enough
The original resume analyzer used generateObject to return structured feedback:
// ❌ The old approach - analyze and done
const { object } = await generateObject({
model: RESUME_ANALYZER_MODEL,
schema: ResumeAnalysisOutputSchema,
prompt: `Analyze this resume against the job description...`,
});
return object; // Score, missing keywords, suggestions... now what?
Users got a score card with bullet fixes like:
Original: "Responsible for managing team projects"
Improved: "Led cross-functional team of 8 engineers to deliver React dashboard"
But they couldn't actually apply these changes. They had to manually copy suggestions, find the right section in their resume, and paste. Most didn't bother.
The feedback was good. The workflow was broken.
I also needed a way to manage multiple resumes - one tailored for each job application. The old analyzer had no concept of "resumes" at all. It just analyzed whatever PDF you uploaded and forgot about it.
Chapter 2: The Agent Architecture
The fix wasn't better analysis - it was guided execution. Instead of dumping suggestions, the AI should:
- Create a plan based on the user's resume
- Collect job details
- Tailor each section one at a time
- Wait for user approval before moving on
- Apply approved changes to the actual resume
This is a multi-step agent with human-in-the-loop control.
ToolLoopAgent from AI SDK 6
The Vercel AI SDK 6 introduced ToolLoopAgent - an agent that executes tools in a loop until a stop condition is met. Perfect for this use case.
// ai/agent/resume-tailor.ts
import { ToolLoopAgent, stepCountIs } from "ai";
export const createResumeTailorAgent = (chatId: string) =>
new ToolLoopAgent({
model: RESUME_CHAT_MODEL,
instructions: RESUME_TAILOR_SYSTEM_PROMPT,
tools: {
createTailoringPlan: createTailoringPlanTool(chatId),
collectJobDetails: collectJobDetailsTool,
tailorSummary: tailorSummaryTool,
approveSummary: approveSummaryTool,
tailorExperienceEntry: tailorExperienceEntryTool,
approveExperienceEntry: approveExperienceEntryTool,
tailorSkills: tailorSkillsTool,
approveSkills: approveSkillsTool,
},
stopWhen: stepCountIs(30),
});
The agent has 8 tools, each with a specific job:
| Tool | Purpose |
|---|---|
createTailoringPlan |
Generate a step-by-step plan based on user's profile |
collectJobDetails |
Show a form to collect job title and description |
tailorSummary |
AI generates a tailored professional summary |
approveSummary |
User reviews and approves/edits the summary |
tailorExperienceEntry |
AI optimizes bullets for one work experience |
approveExperienceEntry |
User approves the optimized bullets |
tailorSkills |
AI reorganizes skills by relevance |
approveSkills |
User approves the skill changes |
The key insight: tailor tools generate suggestions, approval tools wait for user input. The agent can't proceed until the user responds.
Each tailoring tool also returns a match score (0-100) with analysis. For summaries, it's how well the summary matches the job. For experiences, it's a relevance score - how related that role is to the target job:
// ai/tool/resume-tailor/schemas.ts
export const TailoredExperienceOutputSchema = z.object({
experienceId: z.string(),
relevanceScore: z
.number()
.min(0)
.max(100)
.describe(
"How relevant this role is to the target job. 90+: Directly related, 70-89: Transferable skills, 50-69: Some relevance, <50: Limited"
),
suggestedBullets: z.array(z.string()).min(2).max(6),
improvements: z.array(z.string()).describe("What was improved"),
// ...
});
This helps users understand which experiences matter most for the job they're applying to.
Chapter 3: Human-in-the-Loop with Approval Tools
The magic happens in how approval tools work. They have no execute function - they just define a schema and wait:
// ai/tool/resume-tailor/tailor-summary.ts
// This tool DOES work - generates the suggestion
export const tailorSummaryTool = tool({
description: "Generate a tailored professional summary",
inputSchema: z.object({
jobTitle: z.string(),
jobDescription: z.string(),
}),
outputSchema: TailoredSummaryOutputSchema,
execute: async ({ jobTitle, jobDescription }) => {
// AI generates the tailored summary
const { output } = await generateText({
model: RESUME_ANALYZER_MODEL,
output: Output.object({ schema: TailoredSummaryOutputSchema }),
system: TAILOR_SUMMARY_PROMPT,
prompt: `...`,
});
return output;
},
});
// This tool has NO execute - it waits for user input
export const approveSummaryTool = tool({
description: "Show the approval form for the tailored summary",
inputSchema: z.object({}),
outputSchema: SummaryApprovalSchema,
// No execute! The frontend provides the output
});
When approveSummary is called, the AI SDK marks it as approval-requested. The frontend renders an approval card. When the user clicks "Approve" or "Edit", we send the result back:
// Frontend: components/dashboard/resume-tailor/resume-tailor-page.tsx
const handleSummaryApproval = useCallback(
async (toolCallId: string, data: SummaryApproval) => {
addToolOutput({
tool: "approveSummary",
toolCallId,
output: data, // { approved: true, customText: null }
});
},
[addToolOutput]
);
The agent receives the approval and continues to the next step. No polling, no webhooks - just streaming.
Chapter 4: The Plan System
Every tailoring session starts with a plan. The plan is dynamic - it depends on what's in the user's profile:
// ai/tool/resume-tailor/create-plan.ts
export const createTailoringPlanTool = (chatId: string) =>
tool({
description: "Create a tailoring plan for the user's resume",
inputSchema: z.object({}),
outputSchema: TailoringPlanSchema,
execute: async () => {
const profile = await getFullProfile(session.user.id);
const steps: PlanStep[] = [
{ id: "collect_jd", type: "collect_jd", label: "Collect Job Details" },
{
id: "tailor_summary",
type: "tailor_summary",
label: "Tailor Summary",
},
{
id: "approve_summary",
type: "approve_summary",
label: "Review Summary",
},
];
// Add steps for EACH work experience
profile.workExperiences.forEach((exp) => {
steps.push({
id: `tailor_exp_${exp.id}`,
type: "tailor_experience",
label: `Tailor: ${exp.position} @ ${exp.company}`,
context: { experienceId: exp.id },
});
steps.push({
id: `approve_exp_${exp.id}`,
type: "approve_experience",
label: `Review: ${exp.position}`,
context: { experienceId: exp.id },
});
});
// Skills if present
if (profile.skills?.length > 0) {
steps.push({
id: "tailor_skills",
type: "tailor_skills",
label: "Tailor Skills",
});
steps.push({
id: "approve_skills",
type: "approve_skills",
label: "Review Skills",
});
}
// Persist to database
await savePlanSteps(chatId, steps);
return { steps, targetJob: { title: null, description: null } };
},
});
A user with 5 work experiences gets 15 steps. A user with no experience and no skills gets 3 steps. The plan adapts.
DB-Backed Plan State
Initially, I derived plan progress from chat messages. Bad idea. If the stream disconnected mid-step, the UI showed the wrong state.
The fix: store plan steps in the database with explicit status:
// lib/db/tailoring-chat.ts
export async function savePlanSteps(chatId: string, steps: PlanStep[]) {
await prisma.tailoringPlanStep.createMany({
data: steps.map((step, index) => ({
chatId,
stepId: step.id,
stepType: step.type,
label: step.label,
description: step.description,
order: index,
status: "pending", // pending | in_progress | completed | skipped
context: step.context,
})),
});
}
export async function completeStep(chatId: string, stepId: string) {
await prisma.tailoringPlanStep.update({
where: { chatId_stepId: { chatId, stepId } },
data: { status: "completed" },
});
}
Now the UI always shows accurate progress, even after page refresh.
Chapter 5: Message Persistence & Type Mapping
Here's a problem I didn't anticipate: the AI SDK's UI message format doesn't match what you'd want to store in a database.
UI messages have deeply nested parts with different shapes per tool. Storing them as raw JSON works, but querying becomes painful. I needed a mapping layer.
// lib/utils/tailoring-message-mapping.ts
export function mapUIMessagePartsToDBParts(
parts: ResumeTailorAgentUIMessage["parts"],
messageId: string
): DBPartInsert[] {
const result: DBPartInsert[] = [];
parts.forEach((part, index) => {
switch (part.type) {
case "step-start":
result.push({ messageId, order: index, type: "step-start" });
break;
case "text":
result.push({ messageId, order: index, type: "text", text: part.text });
break;
// Tool parts use generic JSONB columns
default:
if (part.type.startsWith("tool-")) {
result.push({
messageId,
order: index,
type: "tool-invocation",
toolCallId: part.toolCallId,
toolName: part.type.replace("tool-", ""),
toolState: part.state,
toolInput: part.input,
toolOutput:
part.state === "output-available" ? part.output : undefined,
});
}
}
});
return result;
}
The reverse mapping (mapDBPartToUIMessagePart) reconstructs the typed parts when loading chat history. This keeps the database schema clean while preserving full type safety in the UI.
Chapter 6: Streaming with createUIMessageStream
The agent streams its output to the frontend. But there's a trick: we need to persist messages AND update plan status as tools complete.
// app/api/resume-tailor/route.ts
const stream = createUIMessageStream({
execute: async ({ writer }) => {
if (message.role === "user") {
writer.write({ type: "start", messageId: generateId() });
}
const agent = createResumeTailorAgent(chatId);
const result = await agent.stream({
messages: await convertToModelMessages(messages),
});
result.consumeStream();
writer.merge(result.toUIMessageStream({ sendStart: false }));
},
originalMessages: messages,
onFinish: async ({ responseMessage }) => {
// Persist the response using our mapping layer
await upsertMessage({
chatId,
id: responseMessage.id,
message: responseMessage,
});
// Update step status based on completed tools
for (const part of responseMessage.parts) {
if (!("state" in part) || part.state !== "output-available") continue;
if (part.type === "tool-tailorSummary") {
await completeStep(chatId, "tailor_summary");
} else if (part.type === "tool-approveExperienceEntry") {
await completeStep(chatId, `approve_exp_${part.output.experienceId}`);
}
// ... other tools
}
// Apply approved changes to the actual resume
await applyApprovedChanges(chat.resumeId, responseMessage, allMessages);
},
});
The onFinish callback is critical. It runs after the stream completes, persisting everything to the database.
Chapter 7: Live PDF Preview
As users approve changes, they see their resume update in real-time. The preview panel shows a live PDF:
// components/dashboard/resume-tailor/resume-tailor-page.tsx
const previewData = useMemo((): ResumeData => {
const preview = { ...initialProfile };
// Apply approved summary
if (approvedChanges.summary?.approved && approvedChanges.summary.text) {
preview.bio = approvedChanges.summary.text;
}
// Apply approved experience changes
if (approvedChanges.experiences) {
preview.workExperiences = preview.workExperiences.map((exp) => {
const change = approvedChanges.experiences[exp.id];
if (change?.approved && change.bullets) {
return { ...exp, bullets: change.bullets };
}
return exp;
});
}
return preview;
}, [initialProfile, approvedChanges]);
// Render with @react-pdf/renderer
<PDFPreview data={previewData} />
Every approval triggers a re-render of the PDF. Users see exactly what their resume will look like before committing.
Chapter 8: The step-start Bug That Broke Everything
This one took hours to debug.
Users would click "Start Tailoring" and... nothing. The chat showed their message, but no response. The UI looked completely broken.
The problem: when a user sends a message, the API immediately writes a step-start event to signal "processing has begun":
// app/api/resume-tailor/route.ts
if (message.role === "user") {
writer.write({ type: "start", messageId: generateId() });
}
This creates an assistant message with a single step-start part. If the stream disconnects before the agent responds, you're left with:
- User message: "Start tailoring my resume"
- Assistant message:
{ parts: [{ type: "step-start" }] }- empty!
My initial stuck detection only checked if the last message was from the user. It completely missed this case.
The Fix
Check for assistant messages that have no meaningful content:
// components/dashboard/resume-tailor/resume-tailor-page.tsx
const stuckState = useMemo(() => {
if (messages.length === 0) return null;
if (status === "streaming" || status === "submitted") return null;
const lastMessage = messages[messages.length - 1];
// Case 1: Last message is from user - no response at all
if (lastMessage.role === "user") {
return { messageId: lastMessage.id, messageText: extractText(lastMessage) };
}
// Case 2: Assistant message with no meaningful content (just step-start)
if (lastMessage.role === "assistant") {
const hasMeaningfulContent = lastMessage.parts.some(
(p) =>
p.type === "text" ||
p.type === "reasoning" ||
p.type.startsWith("tool-")
);
if (!hasMeaningfulContent && messages.length >= 2) {
const userMessage = messages[messages.length - 2];
return {
messageId: userMessage.id,
messageText: extractText(userMessage),
};
}
}
return null;
}, [messages, status]);
The RetryCard Component
When stuck, we show a friendly card with animated icons:
// components/dashboard/resume-tailor/retry-card.tsx
export function RetryCard({
failedMessage,
onRetry,
isRetrying,
}: RetryCardProps) {
const [isHovered, setIsHovered] = useState(false);
return (
<Card
className="border-amber-500/30 bg-linear-to-br from-amber-500/5 to-orange-500/10"
onMouseEnter={() => setIsHovered(true)}
onMouseLeave={() => setIsHovered(false)}
>
<CardContent className="pt-6">
<div className="flex items-start gap-4">
<div className="rounded-full bg-amber-500/10 p-3 relative">
<WifiOffIcon
className={`h-6 w-6 text-amber-600 transition-opacity ${isHovered ? "opacity-0" : "opacity-100"}`}
/>
<RefreshCwIcon
className={`h-6 w-6 text-amber-600 absolute inset-0 m-auto transition-all ${isHovered ? "opacity-100 rotate-180" : "opacity-0"} ${isRetrying ? "animate-spin" : ""}`}
/>
</div>
<div className="flex-1 space-y-4">
<h3 className="font-semibold text-amber-800">
Connection Interrupted
</h3>
<p className="text-sm text-muted-foreground">
The connection was lost before I could respond.
</p>
{failedMessage && (
<div className="p-3 bg-muted/50 rounded-lg">
<p className="text-xs text-muted-foreground">Your message:</p>
<p className="text-sm italic">
"{failedMessage.slice(0, 100)}..."
</p>
</div>
)}
<Button onClick={onRetry} disabled={isRetrying}>
{isRetrying ? "Retrying..." : "Try Again"}
</Button>
<span className="text-xs">Don't worry, your progress is saved</span>
</div>
</div>
</CardContent>
</Card>
);
}
The retry handler re-sends the last user message:
const handleRetry = useCallback(() => {
if (!stuckState) return;
// Find the original user message and re-send it
const userMessage = messages.find((m) => m.id === stuckState.messageId);
if (userMessage) {
handleSubmit({ preventDefault: () => {} } as React.FormEvent);
}
}, [stuckState, messages, handleSubmit]);
Simple recovery, no Redis required. Users forgive network issues if recovery is easy.
The Final Architecture
┌─────────────────────────────────────────────────────────────────────┐
│ Resume Tailor Flow │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ User clicks "Start Tailoring" │
│ ↓ │
│ Agent calls createTailoringPlan → Plan saved to DB │
│ ↓ │
│ Agent calls collectJobDetails → Form shown to user │
│ ↓ │
│ User submits job details → Agent receives input │
│ ↓ │
│ Agent calls tailorSummary → AI generates suggestion │
│ ↓ │
│ Agent calls approveSummary → Approval card shown │
│ ↓ │
│ User approves → Agent continues to next experience │
│ ↓ │
│ [Loop for each experience] │
│ ↓ │
│ Agent calls tailorSkills → AI reorganizes skills │
│ ↓ │
│ User approves → Changes applied to resume in DB │
│ ↓ │
│ PDF preview updates in real-time │
│ │
└─────────────────────────────────────────────────────────────────────┘
TL;DR
| Problem | Solution |
|---|---|
| Analyzer gave advice but didn't help apply it | Agent tailors step-by-step with user approval |
| No control over AI changes | Human-in-the-loop with approval tools |
| Plan progress lost on refresh | DB-backed plan state with explicit status |
| Stream disconnects left UI broken | Stuck state detection with retry mechanism |
| Users couldn't preview changes | Live PDF preview updates on each approval |
| Generic bullet suggestions | AI tailors each experience entry individually |
| No way to manage multiple resumes | Resume dashboard with per-job tailoring |
| UI/DB message type mismatch | Mapping layer for clean persistence |
step-start bug left chat empty |
Check for meaningful content, not just role |
Key Lessons
1. Agents > Analyzers for actionable features.
Analysis tells users what's wrong. Agents help them fix it. If your AI feature ends with "here's what you should do," you've only done half the job.
2. Human-in-the-loop is easier than you think.
Approval tools with no execute function + addToolOutput on the frontend = full user control. The AI SDK handles all the state management.
3. Persist plan state to the database.
Deriving state from chat messages is fragile. Explicit status columns (pending, in_progress, completed, skipped) make the UI predictable.
4. Stream disconnects happen. Plan for them.
Detect stuck states (user message with no response, or empty assistant message) and offer a retry button. Users forgive network issues if recovery is easy.
5. Live preview builds trust.
Users are hesitant to let AI modify their resume. Showing changes in real-time before committing makes them confident to approve.
6. The step-start trap is real.
If your streaming API writes a "processing started" event, you can end up with empty assistant messages on disconnect. Always check for meaningful content, not just message role.
What's Next
- Resume templates - Multiple PDF layouts to choose from
- Cover letter generation - Use the same agent pattern for cover letters
- Job application tracking - Track which resumes were sent to which jobs
- ATS compatibility scoring - Analyze PDF structure, not just content
Why Open Source?
Building an AI agent with human-in-the-loop approvals took weeks of iteration. The streaming, the state management, the edge cases - it's all in the repo.
If you're building something similar, you don't have to start from scratch. Fork it, learn from it, improve on it.
The repo: github.com/ImAbdullahJan/carific.ai
⭐ If you find this useful, consider starring the repo - it helps others discover the project!
Key files from this post:
-
ai/agent/resume-tailor.ts- The ToolLoopAgent definition -
ai/tool/resume-tailor/- All 8 tools with schemas -
app/api/resume-tailor/route.ts- Streaming API with persistence -
components/dashboard/resume-tailor/- UI components for approvals -
lib/db/tailoring-chat.ts- Plan state management -
lib/utils/tailoring-message-mapping.ts- UI ↔ DB message conversion
Your Turn
I'd love feedback:
- On the code: See something that could be better? Open an issue or PR.
- On the post: Too long? Missing something? Tell me.
- On AI agents: How do you handle human-in-the-loop in your projects?
Building in public only works if there's a public to build with.
If this post helped you, drop a ❤️. It means more than you know.
Let's connect:
- 🐙 GitHub: @ImAbdullahJan - ⭐ Star the repo if you found this helpful!
- 🐦 Twitter/X: @abdullahjan - Follow for more dev content
- 👥 LinkedIn: abdullahjan - Let's connect
Eighth post of many. See you in the next one.
Top comments (0)