The latest articles on DEV Community by Vinicius Pacheco (@vinipace). https://dev.to/vinipace https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4150717%2Ffa9d5072-2823-41d0-8b88-44967d781695.jpg https://dev.to/vinipace en Vinicius Pacheco Wed, 30 Sep 2026 03:25:23 +0000 https://dev.to/vinipace/improve-your-agent-from-a-bad-answer-to-a-report-showing-how-much-it-improved-and-why-all-via-mcp-2ph1 https://dev.to/vinipace/improve-your-agent-from-a-bad-answer-to-a-report-showing-how-much-it-improved-and-why-all-via-mcp-2ph1 <p>I’m launching <strong>Datool</strong>, an open-source alternative to Langfuse for tracing and evaluating AI applications.</p> <p>The workflow I want to make easier is the one after you find a bad answer: keep the example, turn it into a test, try a change, and check whether the result improved.</p> <h2> From a Bad Answer to Report showing how much it was improved </h2> <p>Imagine a support agent telling a customer that returns are allowed for 60 days when the supplied policy says 30. A trace helps you see the input, output, and steps that produced that answer. The next step is keeping that case so the same mistake can be tested again.</p> <p>In Datool, you can:</p> <ol> <li>Inspect a trace and its nested spans.</li> <li>Save real examples as dataset cases with expected outputs.</li> <li>Define scorers in JavaScript or Python and test them against recorded outputs.</li> <li>Run evaluations and compare a candidate with a baseline.</li> <li>Inspect the individual cases behind the scores.</li> </ol> <p>ALL VIA MCP.</p> <p>Prompt versions keep the change identifiable. Dataset snapshots help keep the cases consistent between runs. The CLI and MCP interface let these workflows fit into development and agent tooling.</p> <h2> What’s included </h2> <ul> <li>Traces and spans for AI workflows</li> <li>Versioned prompts</li> <li>Datasets and evaluation runs</li> <li>Custom scorers and saved result views</li> <li>Self-hosting under Apache 2.0</li> </ul> <p>The launch video uses an illustrative support-agent example to show how these pieces connect.</p> <p><a href="https://www.trydatool.com/launch.mp4" rel="noopener noreferrer">Watch the 46-second launch video</a>.</p> <h2> Try it or inspect the source </h2> <ul> <li><a href="https://www.trydatool.com/" rel="noopener noreferrer">Datool</a></li> <li><a href="https://github.com/vinipace/datool" rel="noopener noreferrer">Source on GitHub</a></li> <li><a href="https://www.trydatool.com/docs" rel="noopener noreferrer">Documentation</a></li> </ul> <p>I’d like feedback from people building AI apps: how do you turn production failures into regression tests today, and where does that process get stuck?</p> opensource ai testing showdev