DEV Community: Pablo Orozco

Optimizing Agentic Task Predictability

Pablo Orozco — Wed, 13 Aug 2025 16:24:04 +0000

One of the most important shifts in how we use AI is happening quietly, and it’s easy to miss. It’s not about making the model smarter. It’s about making its behavior predictable.

Few days ago, OpenAI released the GPT-5 Prompting Guide in the Cookbook:

GPT-5 Prompting Guide

On the surface, it looks like another collection of tips. It’s not. It’s a blueprint for transforming the model from a reactive answer machine into a reliable, collaborative agent.

The most valuable idea inside it is a pattern I think every builder should adopt: Optimizing Agentic Task Predictability.

The pattern is deceptively simple

You are a methodical AI assistant with expertise in Next.js, TypeScript, TailwindCSS, and shadcn/ui. Here’s the development task: [insert task]. First, outline a detailed, step-by-step technical plan covering file structure, components, and integration details. Wait for my approval before writing any code. After approval, implement only the first step. Continue step-by-step after each confirmation.

You are a methodical AI assistant with expertise in agile product management and Jira workflows. Here’s the product planning task: [insert task]. First, present a clear, step-by-step breakdown into user stories, acceptance criteria, and priorities. Wait for my approval before creating any Jira issues. After approval, create only the first item. Continue step-by-step after each confirmation.

Why this matters

Control: Multi-step tasks no longer run away from you. You decide what’s next.
Transparency: The plan is explicit before execution begins.
Iterative collaboration: You can adjust the approach before resources are spent.

A real example

You are a senior software architect with expertise in Vercel AI SDK v5 and Next.js App Router. Outline the detailed steps to build a streaming chat endpoint that validates incoming user messages before processing, using API Routes and Vercel AI SDK helpers. Include setup, route structure, and integration points in your plan. Wait for my approval before coding. After approval, implement only the validation logic for the first message type. Continue step-by-step after each confirmation.

Three things are happening here:

Role definition — The model takes on relevant expertise.
Precise task framing — You define exactly what needs to happen.
Phase separation — The plan and execution are split, giving you checkpoints.

Compared to older prompting styles, this is a mental shift. The old approach was: "Give me the answer", the new approach is: "Let’s work this out together, step by step".

Agentic workflows are not just a feature of GPT-5, they’re the future of how we’ll work with AI. Predictability is what makes them viable at scale. Without it, your "agent" is just a stochastic suggestion engine. With it, you have something closer to a dependable teammate.

If you want to get more out of GPT-5, stop chasing clever prompts and start designing for predictable collaboration.

🧠 Lead Agents with Prompts

Pablo Orozco — Mon, 28 Apr 2025 12:08:31 +0000

Reviewing Andrej Karpathy’s definition of "vibe coding", I think he captured a new "hands-free" way of building with LLMs.

AI Coding Tools like Cursor and GitHub Copilot’s Agent Mode are bringing that vision to life—turning the IDE into a collaborative, autonomous workflow engine powered by protocols like MCP.

In this post, we’ll explore how features like MCP, NES, and customizable agents are driving a new wave of intelligent, agentic development across modern AI Coding Tools such as:

We’ve been playing with AI completing snippets or suggesting one-liners. Agent Mode goes further—it’s closer to working with a smart developer who can write, run, and even fix things. Whether inside the IDE or through CLI tools like Claude Code or OpenAI Codex, it’s the closest we’ve gotten to truly collaborative AI in the development workflow.

So what does Agent Mode really do?

Writes code — Think full files, not just lines. You feed it intent, it returns structure.
Iterates — It reviews its own output and revises.
Executes commands — From installing dependencies to scaffolding projects.
Self-corrects — When it fails (e.g., bad lint or broken build), it often recovers on its own.

It can plan, execute, and react. That’s not a glorified copilot—that’s autonomous task flow.

🧠 The Real Engine: Model Context Protocol (MCP)

Let’s talk architecture. Agent Mode’s capabilities are powered by MCP, and if you’re a builder, this is the rabbit hole you want to go down.

What Makes MCP Essential for You

MCP Servers

Value: You can build custom, domain-specific agents that interact with your company’s internal tools or data. The community already supports 1,000+ MCP servers: modelcontextprotocol.io

🔗 What is MCP?

MCP is an open protocol for feeding LLMs with structured, real-world dev context.

Think of it like what LSP (Language Server Protocol) did for syntax and intellisense—MCP does for AI agents.

It standardizes how tools like VS Code provide context to AI models.
It connects your AI not just to code, but to:
- Figma designs.
- Internal wikis.
- GitHub repos.
- Databases.
- APIs.

🌱 Why Should You Care?

Because with MCP, you can build custom agents that:

Understand your product domain.
Use internal tools.
Access backend APIs or datasets.
Help teammates onboard faster.

And with it being just HTTP or std I/O, developers are shipping servers for everything: modelcontextprotocol.io

🛠️ How to Plug MCP Into Your Stack

Getting Started

Add an MCP server (e.g. GitHub) to your workspace using .vscode/mcp.json or via VS Code's Command Palette.
Install methods range from Docker to Python CLI wrappers.

Security Improvements

MCP started with JSON env var keys (yes, the usual papercut). But we’re moving toward:

Secure input variables inside VS Code.
OAuth-enabled remote MCP servers.
Less friction, more security.

Example: GitHub MCP Server

It allows Agent Mode to:

Create repos.
Open pull requests.
Search issues.
Manage branches.

Everything works via natural language. You can literally type:

"open a PR with changes from main to staging"

…and it’ll handle it.

Building Your Own Server?

It’s easier than you think. SDKs exist for:

Python (my go-to)
Node
Go
C#

Wrap your internal logic or APIs, write solid descriptions, and boom—you’ve got a tool your Agent can invoke. Test locally with tools like mcp-inspector.

💬 Agent Mode Features Worth Using Daily

Unified Chat Modes

One chat window, three gears:

Ask Mode: Like Stack Overflow meets your context.
Edit Mode: Tightly scoped changes to known files.
Agent Mode: All tools, full workspace, terminal, and file edits unlocked.

Switch freely mid-convo:

Plan in Ask.
Execute in Agent.

Powerful Context Tags

#codebase: Grants search across files.
#file: Targets specific files.
#fetch: Pull in web content.

You can also drag and drop code, images, or datasets into the chat window.

Prompt Files

Save reusable .prompt.md files inside .github/prompts/ to standardize workflows, like scaffolding components or writing tests with mocks.

Interact with Changes

Keep — Approve generated diffs.
Undo/Redo — Navigate iterations.
History — All sessions saved.
Compact Chat Window — Keep the convo in your flow, not your face.

Tool Permissions

Agent Mode asks before running tools. Grant:

Per tool.
Per session.
Per workspace.
Or globally (not recommended for the careless among us).

📓 Jupyter Notebook Integration

Jupyter Notebooks

Agent Mode + Jupyter = 🔥

Auto-setup environments with uv.
Fix import errors via built-in terminal actions.
Explain outputs or improve charts.
Generate entire notebooks from scratch via prompt.

Try: “Create a notebook to analyze Titanic dataset” and watch the structure roll in.

🧠 Next Edit Suggestions (NES)

Next Edit Suggestions

This is inline magic. NES predicts the next logical change as you edit.

Examples:

Rename Point to Point3D? NES adds the z-axis logic.
Add get_max after get_min? NES updates your comparator.
Misspell const? NES fixes it.

Press Tab to accept, Esc to skip. It’s contextual, real-time, and fast.

🔧 Model Customization & BYOK

You’re not locked into GPT-4.

Agent Mode supports:

OpenAI (03, 04 Mini, 4.1).
Claude.
Gemini.
Ollama (local LLMs!).
Anthropic, OpenRouter, Azure, and more...

Plug in your own API keys (BYOK) and experiment. Some models don’t support tool calling—so test before going all in.

Bonus: `.github/copilot-instructions.md`

Copilot Customization

Standardize your LLM behavior:

"Always write unit tests"
"Use React hooks"
"Don’t use for-loops"

codeGeneration.instructions.file

Also, auto-include certain files (like schema.sql) using codeGeneration.instructions.files in your workspace.

🔭 Looking Ahead: Project Padawan

This is the asynchronous agent future—running directly on GitHub.com.

How It Works:

Assign an issue to Copilot.
It works in the background via GitHub Actions.
Opens a PR with a proposed fix.
You review + comment → it iterates.

Imagine automating your backlog, onboarding smart tasks, or killing tech debt without opening VS Code.

🧩 My Dev Recommendations

Context matters: Better input = better output.
Break down work: Guide the agent like you would a smart dev.
Use NES: Inline intelligence that complements completion.
Explore MCP: Build agents tailored to your stack.
Experiment with models: BYOK is a playground.
Always commit: Let Git be your safety net.
Stay curious: This space is moving fast—ride the wave.

Agent Mode is the real deal. It’s not just about AI writing lines of code—it’s about building with you. And with MCP, NES, and customizable tooling, it’s evolving into something that can actually improve how we build, collaborate, and ship.

I’ve already made it part of my dev workflow—and honestly, I don’t want to go back.

Let’s Chat

What tool would you wrap into an MCP server?
What’s your favorite Agent Mode use case?
Tried Padawan yet?

Agentic AI Assistants

Pablo Orozco — Tue, 25 Feb 2025 14:03:25 +0000

Software development is undergoing a massive transformation with AI, yet the popular narrative of "AI replacing developers" misses the most important point: AI's greatest value comes from augmenting human capabilities, not replacing them.

GitHub Repository: https://github.com/josoroma/imaginex-pr-reviewer

Agentic AI Assistants are revolutionizing how developers work by augmenting tools and automating repetitive tasks—whether in the browser with external websites or within native system applications inside an organization.

The Case of the AI Code Reviewer

Let me show you a concrete example I've been working with. I've built an automated code review system that integrates AI to help development teams maintain code quality without slowing down velocity.

The system works by:

🔹 Connecting to GitHub's API to fetch pull request data.

🔹 Running code diffs through advanced LLMs (like GPT-4o or Ollama models).

🔹 Generating detailed markdown reports that analyze code changes.

🔹 Providing explanations, issue identification, and actionable improvements.

Why This Matters for Developers

This approach exemplifies how AI shines as a developer tool, not a replacement:

🔺 Enhanced Capabilities, Not Substitution

The AI reviewer doesn't replace human reviewers - it provides an initial analysis that helps developers focus their attention. Human reviewers still provide context-aware judgment and make the final decisions.

🔺 Addressing Real Developer Pain Points

Code reviews are often bottlenecks in development workflows. By automating the initial review, developers can:

🔹 Get immediate feedback without waiting for human reviewers.

🔹 Identify obvious issues before humans spend time on them.

🔹 Focus human review time on more complex architectural concerns.

Customizability for Team Needs

The system allows teams to use different LLMs based on their requirements:

# Configuration supports both cloud and local models
OLLAMA_MODEL_NAME = os.getenv("OLLAMA_MODEL_NAME", "phi4")
LLM_MODEL_NAME = os.getenv("LLM_MODEL_NAME", "gpt-4o")

Teams can choose privacy-preserving local models for sensitive code or more powerful cloud models when needed.

Beyond Code Review: The AI Developer Toolkit

This pattern extends to numerous development tools that augment rather than replace:

🔹 Pair programming assistants that suggest implementations in real-time.

🔹 Automated test generators that create test cases based on implementation.

🔹 Documentation generators that produce readable explanations from code.

🔹 Code migration tools that help port between frameworks or languages.

🔹 Performance optimization analyzers that suggest improvements.

Why AI Won't Replace Developers

While these tools are powerful, they highlight precisely why AI won't replace developers:

🔺 Context understanding: Developers know the business requirements, organizational constraints, and customer needs that AI lacks.

🔺 Tool orchestration: Developers determine which AI tools to apply and when.

🔺 Judgment calls: Technical decisions often involve tradeoffs that require human judgment.

🔺 Architectural thinking: Systems design remains a deeply creative human activity.

🔺 Human collaboration: Software development is a team sport requiring human communication.

The Future: AI-Enhanced Development

Rather than fearing replacement, developers should embrace AI as the next evolution of our toolset. Just as we moved from assembly to high-level languages, from manual memory management to garbage collection, and from physical servers to cloud infrastructure, AI represents the next abstraction layer.

It is not a secret that the most successful developers won't be those who compete with AI but those who become experts at integrating AI to solve increasingly complex problems.

Getting Started

Want to explore AI-augmented development? The code for this review system is built with:

🔺 CrewAI for orchestrating AI agents.

🔺 LangChain for LLM integration.

🔺 Support for both OpenAI and local Ollama models.

🔺 GitHub API integration.

This pattern of AI augmentation - not replacement - represents the future of software development. The goal isn't to build AI that codes, but to build AI that helps humans code better.

Why did I choose CrewAI for this agentic assistant?

When building AI-driven workflows, I wanted a framework that was both developer-friendly and efficient—and CrewAI checked all the boxes. Here’s what makes it stand out:

🔹 Seamless Collaboration – CrewAI enables AI agents to work together like a well-coordinated team, making automation feel more natural.

🔹 Simple & Intuitive – With YAML-based configuration, setting up and deploying workflows is quick and painless.

🔹 Rapid Prototyping – It lets me focus on building logic instead of worrying about orchestration complexity.

🔹 Highly Scalable – Whether it’s a simple automation or a complex AI-powered pipeline, CrewAI scales effortlessly.

🔹 Thriving Open-Source Community – Backed by a growing community, CrewAI benefits from continuous improvements and innovation.

For teams looking to seamlessly integrate AI into their workflows and boost productivity, this framework is a game-changer.

What’s in Your AI Toolkit?

What AI developer tools are you using in your projects?

Let’s swap ideas—drop a comment below! 👇

DEV Community: Pablo Orozco

Optimizing Agentic Task Predictability

The pattern is deceptively simple

Why this matters

A real example

🧠 Lead Agents with Prompts

So what does Agent Mode really do?

🧠 The Real Engine: Model Context Protocol (MCP)

What Makes MCP Essential for You

🔗 What is MCP?

🌱 Why Should You Care?

🛠️ How to Plug MCP Into Your Stack

Getting Started

Security Improvements

Example: GitHub MCP Server

Building Your Own Server?

💬 Agent Mode Features Worth Using Daily

Unified Chat Modes

Powerful Context Tags

Prompt Files

Interact with Changes

Tool Permissions

📓 Jupyter Notebook Integration

🧠 Next Edit Suggestions (NES)

Examples:

🔧 Model Customization & BYOK

Bonus: .github/copilot-instructions.md

codeGeneration.instructions.file

🔭 Looking Ahead: Project Padawan

How It Works:

🧩 My Dev Recommendations

Let’s Chat

Agentic AI Assistants

The Case of the AI Code Reviewer

The system works by:

Why This Matters for Developers

Customizability for Team Needs

Beyond Code Review: The AI Developer Toolkit

Why AI Won't Replace Developers

The Future: AI-Enhanced Development

Getting Started

Why did I choose CrewAI for this agentic assistant?

What’s in Your AI Toolkit?

Bonus: `.github/copilot-instructions.md`