DEV Community

Cover image for Lessons from Embedding an LLM Inside a Drag-and-Drop Editor
Innocent Oyebode
Innocent Oyebode

Posted on • Originally published at webdigitize.com

Lessons from Embedding an LLM Inside a Drag-and-Drop Editor

#nextjs #fastapi #promptengineering #startup

When you integrate an AI generation step into a drag-and-drop editor, you face a problem that is rarely discussed in LLM tutorials: the model's output must be structurally valid, not just semantically good. A hallucinated CSS class name is a style regression. A hallucinated prop key in an editor schema is a broken component.

This is a writeup of what I learned building WebDigitize, an AI-powered website generator and editor for Nigerian small businesses, using Puck as the editor layer.

What Puck Expects

Puck stores page state as a JSON document:

{
  "root": { "props": {} },
  "content": [
    {
      "type": "HeroSection",
      "props": {
        "headline": "Fresh Bread, Delivered Daily in Lagos"
      }
    }
  ],
  "zones": {}
}
Enter fullscreen mode Exit fullscreen mode

Each entry in content maps to a React component registered in Puck's config:

const config: Config = {
  components: {
    HeroSection: {
      fields: {
        headline: { type: "text" },
        subheadline: { type: "text" },
        ctaLabel: { type: "text" },
        ctaHref: { type: "text" },
      },
      render: ({ headline, subheadline, ctaLabel, ctaHref }) => (
        <section>
          <h1>{headline}</h1>
          <p>{subheadline}</p>
          <a href={ctaHref}>{ctaLabel}</a>
        </section>
      ),
    },
  },
};
Enter fullscreen mode Exit fullscreen mode

The critical insight: Puck only renders props defined in fields. If the AI emits headlineText instead of headline, the component renders silently with undefined. No error. No warning. An invisible field.

The Schema Representation Problem

Your LLM knows nothing about your Puck config. You need to tell it what is valid. The naive approach is to describe your components in prose:

"The HeroSection component takes a headline, a subheadline, a CTA label, and a CTA href."

This works for simple cases and fails for complex ones. If you have 15 components, each with 4–8 fields, prose descriptions become ambiguous, and the model drifts toward invented field names that sound plausible.

The approach that worked: represent your component schemas as a compact formal notation in the system prompt.

=== PUCK BLOCK SCHEMAS (STRICT) ===
Only use the exact prop keys listed. No extra keys. No variations.

HeroSection:
  headline: string           (max 60 chars — punchy, specific to this business)
  subheadline: string        (max 120 chars)
  ctaLabel: string           (max 30 chars)
  ctaHref: string            (WhatsApp link preferred; format: https://wa.me/{number})

ServicesSection:
  title: string
  services: Array<{
    icon: "star" | "check" | "bolt" | "heart" | "shield" | "phone"
    title: string            (max 40 chars)
    description: string      (max 100 chars)
  }>                         (2–4 items, no more)

ContactSection:
  address: string
  phone: string              (must match brief — do not invent)
  hours: string              (format: "Mon–Sat, 8am–6pm")
  mapEmbedQuery: string      (Google Maps query string, e.g. "Chicken Republic Ikeja Lagos")
Enter fullscreen mode Exit fullscreen mode

Key decisions in this notation:

  • Pipe-separated enums ("star" | "check" | ...) are extremely effective — the model treats them as a closed set.
  • Inline constraints (max 60 chars) prevent verbose fields that break layouts.
  • Explicit notes (must match brief — do not invent) address the most common semantic failures.

System Prompt vs User Message

I spent some time debugging why the critic pass kept ignoring my schema constraints. The answer was placement.

Schema in the user message:

[System]: You are a web design assistant. Output only valid JSON.
[User]:   Generate a home page for Mama Titi's Kitchen.
          Here are the allowed block schemas: ...
          Business details: ...
Enter fullscreen mode Exit fullscreen mode

In multi-turn conversations (generation → critic → refinement), the schema in the user message is treated as part of the task context that can be "overridden" by subsequent turns. The critic saw new information in its turn and occasionally invented prop names that were not in the original schema.

Schema in the system prompt:

[System]: You are a web design assistant. Output only valid JSON.
          === PUCK BLOCK SCHEMAS (STRICT) ===
          HeroSection: ...
          [all schemas]
[User]:   Generate a home page for Mama Titi's Kitchen.
          Business details: ...
Enter fullscreen mode Exit fullscreen mode

The system prompt is treated as a persistent context across all turns. Schema violations dropped to nearly zero. The model seems to correctly understand that system prompt constraints are inviolable while user message constraints can flex.

Validating Output at the Boundary

Even with strict prompting, I added a validation step before storing any generated content:

ALLOWED_BLOCKS = {
    "HeroSection": {"headline", "subheadline", "ctaLabel", "ctaHref"},
    "ServicesSection": {"title", "services"},
    "ContactSection": {"address", "phone", "hours", "mapEmbedQuery"},
    # ...
}

def validate_puck_content(content: dict) -> list[str]:
    errors = []
    for block in content.get("content", []):
        block_type = block.get("type")
        if block_type not in ALLOWED_BLOCKS:
            errors.append(f"Unknown block type: {block_type}")
            continue
        allowed_keys = ALLOWED_BLOCKS[block_type]
        for key in block.get("props", {}):
            if key not in allowed_keys:
                errors.append(f"{block_type}: unknown prop '{key}'")
    return errors
Enter fullscreen mode Exit fullscreen mode

When validation fails, I log the raw output and retry generation with an amended prompt that includes the specific error:

if errors:
    additional_constraint = f"CRITICAL: Previous attempt used invalid props: {errors}. Fix these."
    return await generate_with_retry(brief, page_type, extra_constraint=additional_constraint)
Enter fullscreen mode Exit fullscreen mode

In practice, the retry only triggers for about 2% of generations.

The Array Prop Problem

Arrays are the most reliably broken output format. Consider services: Array<{icon, title, description}>. The model produces:

"services": [
  {"icon": "star", "title": "Fresh Bread", "description": "Baked daily"},
  {"icon": "check", "title": "Fast Delivery", "description": "Within 2 hours"}
]
Enter fullscreen mode Exit fullscreen mode

This is valid. But then on a retry or critic pass, I have seen:

"services": {
  "items": [...],
  "count": 2
}
Enter fullscreen mode Exit fullscreen mode

Or:

"services": [
  {"icon_name": "star", "service_title": "Fresh Bread", "text": "Baked daily"}
]
Enter fullscreen mode Exit fullscreen mode

The second failure (renamed keys inside array items) is invisible without deep validation. I extended the validator to recurse into array items:

SERVICES_ITEM_KEYS = {"icon", "title", "description"}

def validate_services(items):
    for i, item in enumerate(items):
        for key in item:
            if key not in SERVICES_ITEM_KEYS:
                return [f"services[{i}]: unknown key '{key}'"]
    return []
Enter fullscreen mode Exit fullscreen mode

Rich Text: The Unexpected Footgun

One of the blocks — a ContentBlock for long-form text — accepted a body field that was supposed to be a Puck rich text value (a Lexical editor serialised state, not a plain string).

I made the mistake of exposing this to the AI. The model produced beautiful-looking text, but it was a plain string. The Lexical editor expected an object. The result was a crash on the editor load that only appeared when the user tried to edit the block.

The fix: never ask an LLM to produce rich text editor state. Instead, the ContentBlock accepts a bodyText: string prop. A useEffect in the component converts it to a Lexical initial state on first render. The AI never touches the editor's internal format.

Structuring the Puck Config for AI-Friendliness

After building this, I restructured the Puck config with AI consumers in mind:

  1. Use consistent naming conventions. All text fields are string types named with a noun (headline, subheadline, address). All boolean toggles start with show (showCta, showImage). The model learned these conventions quickly and stopped inventing alternatives.

  2. Keep array items flat. Nested object arrays are reliable. Arrays of arrays are not. If you need complex nested structures, flatten them at the schema layer.

  3. Provide default prop values in the component. If ctaHref is missing, the component should gracefully render without a CTA link rather than throwing. This makes partial AI output survivable.

  4. Separate "AI-generated" fields from "user-configured" fields. Fields like backgroundImage and colorScheme are set by the user through the visual editor, never by the AI. Keeping these out of the generation schema prevents the model from guessing colours or image URLs, which it consistently does poorly.

The Rendering Architecture

Puck renders in two modes in WebDigitize:

  • Editor mode (<Puck config={config} data={data} />) — used in the dashboard, full interactive editing
  • Render mode (<Render config={config} data={data} />) — used on public-facing site pages, no editing chrome

Both modes use the same config and data. When the AI generates content and stores it, it is immediately renderable on the public site. When the user opens the editor, they see the AI output in the Puck canvas, ready to drag, drop, and edit.

This single-schema-serves-both-modes design is the most important architectural decision. It means the AI, the editor, and the public renderer are all working with the same contract.

Takeaways

  1. Formal schema notation in the system prompt outperforms prose descriptions for structured output tasks.
  2. System prompt beats user message for persistent constraints across multi-turn generation pipelines.
  3. Validate at the boundary, not at render time. Silent failures are the enemy.
  4. Never send rich text editor state formats to an LLM. Accept plain strings and convert at the component layer.
  5. Design your block schemas with AI consumers in mind from the start — consistent conventions, flat structures, clear defaults.

Have you run into schema validation problems with LLM output in your own projects? I'd love to hear how you solved it.

Top comments (0)