Teaching AI Agents by Example: The End of brittle selectors

#ai #automation #showdev #webdev

The Brittle Selector Problem

If you've ever built browser automation, you know the pain. You spend hours crafting the perfect XPath or CSS selector, test it thoroughly, deploy it to production... and then the target website updates their CSS framework and everything breaks.

// Yesterday this worked
const submitBtn = await page.$('#submit-button');

// Today it's broken because the ID changed to 'btn-submit-primary'

This isn't just annoying—it's a fundamental limitation in how we teach AI agents to interact with the web.

Why Demonstration Beats Specification

Humans don't learn by reading CSS selectors. We learn by watching, doing, and repeating. A child learns to tie their shoes by watching their parent, not by reading a DOM specification.

AI agents should learn the same way.

When you record your screen performing a task, you're capturing:

Intent - What you're trying to accomplish
Context - The surrounding UI elements that help identify targets
Flow - The sequence of actions and decision points
Recovery - How you handle errors or unexpected states

This is infinitely more robust than a list of selectors.

The SKILL.md Approach

SKILL.md files capture the essence of a task in a format that's both human-readable and machine-executable. Instead of:

# Fragile approach
click: "#submit-btn"
fill: "#email-field"

You get:

# Book a Demo Workflow

## Goal
Schedule a product demo through the website booking form

## Workflow
1. Navigate to /book-demo
2. Identify the booking calendar widget
3. Select first available time slot
4. Fill contact information
5. Confirm booking

## Context Signals
- Look for calendar UI with time slots
- Form should have email, name, company fields
- Success indicator: confirmation message or email

## Error Handling
- If no slots available, try next day
- If form validation fails, check required fields

The difference? This skill description is resilient. It describes what to look for, not where to find it.

Real-World Impact

I've seen teams spend weeks maintaining brittle automation scripts. With demonstration-based skill creation:

5 minutes to record a new workflow
Zero ongoing maintenance when the UI updates
Reusable skills across different agent frameworks
Shareable knowledge within teams

The Bigger Picture

We're moving from an era of "telling computers what to do" to "showing them what we want." This shift democratizes automation—you don't need to be a developer to teach an AI agent. You just need to know how to do the task yourself.

For developers, this means focusing on high-level orchestration rather than low-level DOM manipulation. For businesses, it means capturing institutional knowledge in executable form.