The Future of AI Agent Training
We've been teaching AI agents wrong. Instead of showing them what we want, we've been writing complex code with brittle selectors that break at the slightest UI change. There's a better way.
The Problem with Traditional Automation
Browser automation has always been fragile. You write scripts that depend on specific DOM elements, CSS classes, and page structures. Then the website updates its design, and everything breaks.
// Yesterday this worked
await page.click('#submit-button');
// Today it's broken because the ID changed
This approach requires constant maintenance, deep technical knowledge, and endless debugging. It's not scalable.
A New Paradigm: Learning by Demonstration
What if we could teach AI agents the same way we teach humans? By showing, not telling.
When you record your screen performing a task, you're capturing:
- Intent - What you're trying to accomplish
- Context - The surrounding UI elements that matter
- Flow - The sequence of actions and decisions
- Recovery - How you handle errors or unexpected states
This is infinitely more robust than a list of selectors.
Enter SKILL.md
SKILL.md is a structured format that captures agent skills in a way that's both human-readable and machine-executable. Instead of brittle selectors, it describes intent:
# Book a Meeting Workflow
## Goal
Schedule a product demo through the website booking form
## Workflow
1. Navigate to /book-demo
2. Identify the calendar widget
3. Select first available time slot
4. Fill contact information
5. Confirm booking
## Context Signals
- Look for calendar UI with time slots
- Form should have email, name, company fields
- Success indicator: confirmation message or email
## Error Handling
- If no slots available, try next day
- If form validation fails, check required fields
The beauty? This skill works across different scheduling platforms and survives UI updates.
From Recording to Skill
The pipeline is elegantly simple:
- Record - Capture your screen performing the task
- Extract - AI analyzes to identify actions and context
- Structure - Convert to SKILL.md format
- Execute - Any compatible agent can perform the skill
Why This Changes Everything
For Developers:
- No more maintaining brittle selectors
- Skills are portable across frameworks
- Version control friendly
For Domain Experts:
- Create automation without coding
- Capture institutional knowledge
- Share skills with team members
For Organizations:
- Build reusable skill libraries
- Reduce maintenance costs
- Democratize tool creation
Try It Yourself
Want to see demonstration-based skill creation in action?
🚀 Check out SkillForge — record your screen, get a SKILL.md file
🔥 Support us on Product Hunt
What workflows would you teach an AI agent if you could just show it once?
Top comments (0)