Dakota Ma - DEV Community

Dakota Ma

Sep 20

A Perfect Pass Rate Is a Blind Harness

#ai #testing #python #promptengineering

9 min read

Dakota Ma

Sep 19

Keep Schema Failures Out of Your Quality Score

#ai #python #testing #llm

7 min read

Dakota Ma

Sep 18

Tuning-Set Pass Rate Is Not an Eval

#ai #testing #python #devops

7 min read

Dakota Ma

Sep 17

The Completion Matched. The Envelope Did Not.

#ai #testing #python #programming

1

8 min read

Dakota Ma

Sep 16

Treat the Grader as Code, Not a Hidden Prompt

#ai #python #testing #opensource

6 min read

Dakota Ma

Sep 14

Calibrate the Noise Floor of Your Eval Harness

#ai #testing #python #llm

6 min read

Dakota Ma

Sep 13

Fail the Eval When Samples Disagree

#ai #testing #python #programming

6 min read

Dakota Ma

Sep 12

Diff Last-Good Completions, Not Just Pass Rate

#testing #ai #python #llm

7 min read

Dakota Ma

Sep 11

Mutate Golden Cases Before You Trust the Eval

#python #ai #testing #programming

6 min read

Dakota Ma

Sep 9

When Context Is Incomplete, Completion Is the Bug

#testing #ai #python #agents

6 min read

Dakota Ma

Sep 7

Pin Behaviors Across Model Swaps

#ai #python #testing #agents

8 min read

Dakota Ma

Sep 7

Version Golden Cases Like Schema Migrations

#ai #testing #python #llm

5 min read

Dakota Ma

Sep 7

Put Forbidden Inferences in the Golden File

#ai #testing #python #agents

6 min read

Dakota Ma

Sep 7

Score the Trace, Not the Final Payload

#ai #testing #python #agents

7 min read

Dakota Ma

Sep 6

Score the Trace, Not the Last Message

#ai #testing #python #agents

7 min read

Dakota Ma

Sep 4

Catch Tool Calls That Invent Missing Arguments

#python #testing #ai #llm

1

7 min read

Dakota Ma

Sep 4

Schema-First Validation: Catch Silent Model Drift

#jsonschema #llm #python #contracttesting

7 min read

Dakota Ma

Sep 4

Free Tiers Are Just Queues: A Capacity Plan for Token-Limited Pipelines

#devops #python #architecture #ai

5 min read

Dakota Ma

Sep 3

Treat LLM Output Like an API Contract

#python #ai #testing #llm

5 min read

Dakota Ma

Sep 2

Your Paid Model Needs a Cheap Skeptic

#ai #llm #python #devops

3 min read

Dakota Ma

Sep 1

Your Grader Is Drifting Too: A Self-Auditing Prompt Eval Harness

#ai #testing #llm #opensource

5 min read

Dakota Ma

Aug 31

A Cache-First LLM Gateway You Can Run on a Free Server

#ai #llm #python #webdev

5 min read

Dakota Ma

Aug 30

Golden Sets Are the Unit Tests Your LLM Feature Never Had

#ai #testing #llm #python

4 min read

Dakota Ma

Aug 29

Prompt Drift Is a Quiet Deployment Bug: A Nightly Check That Costs Nothing

#ai #testing #python #productivity

1

4 min read

Dakota Ma

Aug 28

Silent Regressions Have No Stack Trace: A Minimal Prompt Eval Harness

#ai #testing #python #llm

2

5 min read

Dakota Ma

Aug 26

The Prompt Changed. Nothing Broke. That's the Problem.

#ai #llm #testing #programming

5 min read

Dakota Ma

Aug 25

A Quota-Aware Proxy for Free AI Endpoints: Treating Limits as a Budget, Not a Wall

#ai #python #webdev #architecture

3 min read

Dakota Ma

Aug 25

A Free-Tier AI PR Reviewer: A GitHub Actions Workflow That Actually Works

#ai #github #devops #python

1

4 min read

Dakota Ma

Aug 23

The Model Was Fine. My Token Assumptions Weren't.

#debugging #ai #llm #python

5 min read

Dakota Ma

Aug 22

The Crash That Only Happened in Production

#go #debugging #concurrency #ai

3 min read

Dakota Ma

Aug 14

I Almost Posted a Hot Take About a Cheap New Model. Then I Built a Free Triage Harness.

#ai #python #testing #webdev

3 min read

Dakota Ma

Aug 10

Shadow-Test a New Coding Model in One Week: A Correction-Log Method

#ai #productivity #testing #programming

4 min read

Dakota Ma

Aug 10

Your Agent Reads Untrusted Text All Day. Here's How I Grade What It Does With It.

#security #ai #agents #testing

6 min read

Writing Debut