DEV Community: Marvin Ma

AI Made Development Faster. Testing Needs to Stop Living in Spreadsheets.

Marvin Ma — Wed, 17 Jun 2026 03:29:24 +0000

AI agents are making software development faster.

That is great.

But there is a problem I do not think we are talking about enough:

testing is not speeding up in the same way.

In many teams, testing is still held together by spreadsheets, meeting notes, screenshots, chat messages, and the memory of a few experienced QA engineers.

That worked when delivery was slower.

It becomes fragile when one developer can use multiple agents to change code across several modules in a single afternoon.

The bottleneck is no longer "can we write more test cases?"

The bottleneck is:

Can the team prove what was tested, why it was tested, what failed, what was fixed, and whether the release is safe?

That is the problem I built testboat for.

The Most Dangerous Sentence Before A Release

The sentence I worry about most is not:

We did not test this.

At least that is honest.

The dangerous sentence is:

I think we tested this.

That sentence usually means the team has test artifacts, but they are disconnected:

requirements live in a doc
test cases live in a spreadsheet
automation scripts live somewhere in the repo
execution results live in CI logs or chat
bugs live in an issue tracker
release reports are written manually before sign-off

Each piece may be useful on its own.

But when a Tech Lead asks, "Which requirements are not covered?" or a founder asks, "Can we release today?", the team has to reconstruct the answer manually.

That is not a testing process.

That is institutional memory under pressure.

AI Makes This Gap Worse

AI agents are very good at increasing throughput.

They can:

implement a feature faster
refactor code faster
generate UI faster
write automation faster
fix bugs faster

But faster change creates more testing uncertainty.

If an agent changes the authentication module, what should be rerun?

If a test fails, is it a product bug, a flaky automation script, or an environment issue?

If a developer says "fixed", has the failed test actually been rerun?

If a release report says "main flows passed", where is the evidence?

Without a structured system, QA becomes the human buffer. Tech Leads become risk translators. Founders buy uncertainty with every release.

That is not sustainable.

Testing Needs To Become An Engineering System

testboat treats test artifacts like code.

It creates a .testboat/ directory in your project:

.testboat/
  .active
  draft/
    strategy.yaml
    tags.yaml
    cases/
      TC-001.yaml
    bugs/
      BUG-001.yaml
    executions/
      plans/
      results/
      execution-matrix.yaml
      automate/
    reports/

The important part is not "YAML is nice."

The important part is connection.

A requirement connects to a test case through req_id.

A test case connects to an execution plan.

An execution plan connects to an automation script.

A result connects back to the test case.

A bug can connect to both the test case and the failing result.

The latest execution state is summarized in an execution matrix.

Reports are generated from the same artifacts, not written from memory.

That changes the conversation.

Instead of asking:

Did we test login?

You can ask:

Show me every auth test case, its latest result, open bugs, and whether the release exit criteria passed.

What QA Gets

QA should not have to be the team's memory database.

With testboat, a test case is a structured file:

id: TC-001
title: Login with wrong password returns 401
status: ready
priority: P1
automation: to-automate
tags:
  sprint: v1.0
  type: functional
  module: auth
req_id: STORY-001
steps:
  - action: Enter wrong password
    expected: API returns 401
expected_result: User sees a clear error message

It is diffable.

It is reviewable.

It has a state:

draft -> ready -> pass / fail / blocked / skipped

That means QA can maintain testing facts instead of constantly answering questions from memory.

What Tech Leads Get

Tech Leads need quality gates, not just good intentions.

testboat validate runs pre-report checks:

format validation
requirements coverage
execution completeness
exit criteria compliance

That last part matters.

Your strategy.yaml can define severity rules and exit criteria. For example, P0 and P1 bugs must be zero before release.

So the report is not just a nice HTML page.

It is generated after the system checks whether the release evidence is healthy enough.

This is the kind of thing that can eventually belong in CI.

What Founders And Managers Get

Founders do not need to read every test case.

But they do need release confidence.

"Main flows passed" is not enough.

The useful questions are:

how many test cases exist?
how many passed?
how many were not executed?
which requirements are uncovered?
how many open P0/P1 bugs remain?
did the release satisfy exit criteria?
can we trace failures back to bugs and fixes?

testboat generates strategy, sprint, and closure reports from the actual test artifacts.

That gives leadership evidence instead of vibes.

Where AI Agents Fit

The goal is not to replace QA.

The goal is to give AI agents a testing workflow they can follow.

testboat enable creates agent-specific instructions for tools like Claude, Copilot, Cursor, Kiro, and others.

An agent can then follow a repeatable SOP:

check the active test version
read the strategy
inspect registered tags
create or update test cases
validate test cases
create execution plans
run automation or guide manual testing
record results
file bugs on failures
rerun affected tests after fixes
validate before reporting

That is the difference between "AI wrote some tests" and "AI participated in the testing lifecycle."

A Small Example

If the auth module changed, you should not ask:

Can someone test login?

You should be able to do this:

testboat case list --module auth
testboat matrix show

Then rerun the affected tests and record results:

testboat result record TC-001 pass --type automated --by "AI"

If a failure appears:

testboat bug add \
  --title "Wrong password returns 500 instead of 401" \
  --tc TC-001 \
  --severity major \
  --priority P1

And after the fix, the bug should not jump straight to "closed."

It should move through retest:

fixed -> pending-retest -> verified -> closed

That is the loop teams need when development is moving faster.

Why I Think This Matters Now

AI is making code cheaper to produce.

That does not automatically make releases safer.

If anything, it makes weak testing systems more visible.

The next layer of AI engineering is not just faster code generation.

It is turning the surrounding engineering practices into systems that agents can participate in.

Testing is one of those practices.

That is why I built testboat.

Not to generate more test cases.

To make testing traceable, reviewable, versioned, validated, and reportable.

Try It

pip install testboat
testboat init
testboat enable cursor
testboat strategy create

Project:

https://github.com/lijma/testboat

Docs:

https://lijma.github.io/testboat/

Question

How does your team know a release is actually ready?

Is that answer stored in a system, or mostly in people's heads?

What's causing you more rework with AI right now?

Marvin Ma — Tue, 10 Mar 2026 08:44:57 +0000

What's causing you more rework with AI right now?

Session resets, wrong requirements, unreadable docs, or team context drift?

My take: wrong requirements cost the most, but session resets hide the problem until late.

What hurts more in your workflow?

My AI Agent Forgot Yesterday's Debugging. So I Changed the Workflow.

Marvin Ma — Mon, 09 Mar 2026 09:44:23 +0000

AI coding sessions do not usually fail because the model is weak. They fail because yesterday's debugging conclusions disappear when the chat ends. Yesterday I spent 2 hours debugging an auth refresh bug, and this morning my AI agent still remembered the mutex, the wrong storage key, and the exact files we changed.

That changed how I think about AI workflows. I used to think the fix was a better prompt or a bigger context window. It wasn't.

What fixed it was much simpler: move the useful context out of the chat and into the repo.

The Failure Moment

If you use AI coding agents regularly, you've probably had this happen:

Day 1:
You: Help me debug this token refresh loop.
AI: Finds the issue after a long back-and-forth.

Day 2:
You: What edge cases should we test for that fix?
AI: Can you share the relevant code and explain the bug first?

That second answer is the real tax of AI-assisted coding.

The model may be smart. The session may be long. But if yesterday's conclusions are trapped inside yesterday's conversation, today still starts from zero.

The Bug Wasn't Trivial

The concrete example here was a token refresh loop in a React + Node.js app.

After a long debugging session, the useful conclusions were:

the refresh endpoint returned a new access token, but the client stored it under the wrong key
two concurrent API calls could trigger refresh at the same time
the fix needed a mutex, a consistent access_token key, and a debounce on refresh

Those are exactly the kinds of details that matter the next day.

Not the whole codebase. Not the whole architecture. Just the things we already paid to learn.

What Changed

Instead of letting that knowledge die in the session, I persisted it to the project itself.

With fcontext, the agent writes important conclusions to .fcontext/_topics/ as plain Markdown.

So instead of this:

Yesterday's debugging -> lost in chat history

I get this:

Yesterday's debugging -> .fcontext/_topics/auth-token-refresh-bug.md

That one change turns a temporary conversation into reusable project knowledge.

The Smallest Workflow That Made It Work

This is the shortest version:

pip install fcontext
cd your-project
fcontext init
fcontext enable copilot

That creates a .fcontext/ directory and gives the agent instructions to read it at the start of each session.

The important part is not the setup itself. The important part is the workflow:

work with the agent normally
when the session reaches a useful conclusion, save it as project context
let the next session read that context before you start asking follow-up questions

In practice, the useful file looks more like this:

# Auth Token Refresh Bug — Root Cause Analysis

## Problem
1. Client stores refreshed token in `sessionStorage.token`
   but reads from `sessionStorage.access_token`
2. Concurrent API calls trigger duplicate refresh attempts

## Fix Applied
- Added refresh mutex in `src/auth/tokenManager.ts`
- Fixed storage key to `access_token`
- Added debounce in `src/api/client.ts`

Now the next session does not need to rediscover the bug. It can build on it.

Why This Worked Better Than a Better Prompt

I think this is the part most people underestimate.

Prompting helps you describe what you want right now.

It does not automatically preserve what you learned yesterday.

A bigger context window helps you fit more tokens into one conversation.

It does not automatically convert yesterday's hard-won debugging conclusions into durable project state.

That is why I no longer think of AI context as "the chat so far." I think of it as something closer to project memory.

If the insight matters tomorrow, it should exist somewhere more stable than a conversation tab.

The Real Payoff

The next morning, the follow-up question becomes much more useful:

Is there anything else related to the token refresh fix that we should test?

And instead of asking me to restate the entire bug, the agent can respond with things like:

test expired refresh tokens during an active request
test concurrent refresh attempts after the mutex change
test whether debounce hides a legitimate retry path
test the exact files changed in the previous fix

That is the workflow improvement I actually care about.

Not "AI remembers everything magically."

Just: the next session starts from the last useful conclusion, not from polite amnesia.

Honest Limitation

This is not magic memory.

If the saved topic is vague, low quality, or missing the real conclusion, the next session will inherit weak context.

So this approach does not remove the need for good engineering judgment. It makes good engineering judgment reusable.

That distinction matters.

Why I Think This Matters More Than It Sounds

Most AI coding discussions still focus on prompts, models, and context windows.

Those matter.

But in day-to-day delivery, I hit a more basic problem first: useful project knowledge keeps evaporating.

Once I treated context as repo state instead of chat residue, the quality of follow-up sessions improved immediately.

That feels like a more durable workflow change than endlessly rewriting the perfect prompt.

Try It Yourself

pip install fcontext
cd your-project
fcontext init
fcontext enable copilot

Then use your agent on a real bug, save the conclusion, and come back the next day with a follow-up question.

That is the fastest way to tell whether your AI workflow has memory or just momentum.

Question

How are you handling cross-session context today?

Are you still re-explaining important debugging sessions to each new agent, or are you treating project context as something worth persisting?

How to Give Your AI Coding Agent Persistent Memory in 30 Seconds

Marvin Ma — Wed, 25 Feb 2026 16:54:01 +0000

Your AI coding agent doesn't remember yesterday.

You spent an hour debugging a tricky race condition, the AI understood every nuance — and this morning it asks you to "explain the project architecture." Again.

By the end of this post, you'll have persistent memory working across sessions in under 30 seconds. I'll show you the exact terminal output at every step so you can follow along.

Prerequisites

Python 3.9+
Any AI coding agent (Copilot, Claude Code, Cursor, Trae)
A project you're actively working on

Step 1: Install and Initialize (15 seconds)

pip install fcontext

Successfully installed fcontext-1.0.0

Navigate to your project and initialize:

cd your-project
fcontext init

✓ Created .fcontext/
✓ Generated _README.md
✓ Generated _workspace.map

That's it. You now have a .fcontext/ directory:

.fcontext/
├── _README.md          # Project summary — AI reads this first
├── _workspace.map      # Auto-generated project structure
├── _topics/            # Where AI saves session knowledge
├── _requirements/      # Optional: track stories/tasks/bugs
└── _cache/             # Optional: converted binary docs

Everything is plain Markdown. No database, no cloud, no API keys.

Step 2: Enable Your Agent (15 seconds)

fcontext enable copilot

✓ Generated .github/instructions/fcontext.instructions.md
✓ Copilot will now read .fcontext/ on every session

Using a different agent? Swap the name:

fcontext enable claude    # → .claude/rules/fcontext.md
fcontext enable cursor    # → .cursor/rules/fcontext.md
fcontext enable trae      # → .trae/rules/fcontext.md

Each agent gets instructions in its native config format. No plugins, no extensions — fcontext generates the standard config files that your agent already knows how to read.

Want multiple agents? Enable them all:

fcontext enable copilot && fcontext enable claude && fcontext enable cursor

They all read the same .fcontext/ data. Switch agents freely — no context loss.

Step 3: Verify It Works

Check what you've set up:

fcontext status

fcontext status
  Initialized: yes
  Agents: copilot, claude
  Topics: 0
  Requirements: 0
  Cached files: 0

Now open your AI agent and start coding normally. Here's what happens behind the scenes:

Session starts → Agent reads _README.md and _topics/
You work → Agent builds understanding of your project
Important discovery → Agent saves it to _topics/your-topic.md
Session ends → Knowledge persisted on disk
Next session → Agent reads everything from step 1 — full continuity

Real Example: Before and After

Before fcontext

# Monday session
You: "Help me debug the auth token refresh"
AI: [2 hours of deep debugging, finds race condition, adds mutex]

# Tuesday session
You: "What about edge cases in the auth fix?"
AI: "Could you provide more context about what you're working on?"

# 😤 starts re-explaining from scratch

After fcontext

# Monday session — same debugging work
# AI saves to .fcontext/_topics/auth-token-debugging.md:
#   - Race condition: concurrent API calls trigger duplicate refresh
#   - Fix: added mutex + storage key dedup + debounce
#   - TODO: test edge case with expired refresh token

# Tuesday session
You: "What about edge cases in the auth fix?"
AI: "Based on yesterday's debugging, the mutex handles concurrent
     refreshes. The edge case to test is an expired refresh token
     during an active request. Want me to write that test?"

No re-explaining. The AI picks up exactly where you left off.

Bonus: Index Your Documents

Got PDFs, DOCX, or XLSX files in your project? Your AI can't read binary files — but fcontext can convert them:

fcontext index docs/

✓ Indexed docs/product-requirements.pdf → _cache/docs/product-requirements.pdf.md
✓ Indexed docs/api-spec.docx → _cache/docs/api-spec.docx.md
2 files indexed

Now your AI can reference those documents directly. No more copy-pasting from PDFs.

Supported formats: PDF, DOCX, XLSX, PPTX, Keynote, EPUB.

Bonus: Track Requirements

If your project has user stories or tasks scattered across Slack and docs:

fcontext req add "OAuth login flow" -t story
fcontext req add "Support Google provider" -t task --parent STORY-001
fcontext req set TASK-001 status in-progress

fcontext req board

📋 Board

TODO          IN-PROGRESS       DONE
─────────     ─────────         ────
              TASK-001
              Support Google
              provider

STORY-001
OAuth login
flow

Your AI reads _requirements/ and builds against tracked specs — not guesses.

Common Gotchas

"My AI didn't read .fcontext/ on first session"

After fcontext enable, tell your AI: "Read .fcontext/_README.md and update it with the project info." It needs one nudge, then it maintains the file automatically.

"Can I git-commit .fcontext/?"

Yes — and you should. Your teammates pull the repo and get the same context. Their AI instantly knows the project.

git add .fcontext/
git commit -m "add project context"

"What if I want to start fresh?"

fcontext reset

Gone. Clean slate.

The Numbers

I tracked my context re-explanation time for two weeks. Week 1 without fcontext, week 2 with:

Metric	Without	With fcontext
Daily context setup time	~12 min	~0 min
Agent switching overhead	~10 min	0 min
Weekly total waste	~60 min	~3 min

The time saving is nice. But the real win is answer quality — an AI with accumulated project context gives better, more consistent responses than one starting from zero every morning.

TL;DR

pip install fcontext
fcontext init
fcontext enable copilot   # or: claude, cursor, trae

30 seconds. Your AI now remembers.

GitHub: github.com/lijma/agent-skill-fcontext

What's your current workaround for AI context loss? Curious how others are dealing with this — drop a comment 👇

I Built a Tool to Stop AI Coding Agents from Forgetting Everything Between Sessions

Marvin Ma — Sun, 15 Feb 2026 09:15:15 +0000

The Problem: Every Session Starts from Zero

If you use AI coding agents, you've experienced this:

Session 1 (Copilot): "Build the auth module with OAuth support"
  → Great work. Deep debugging. Important decisions made.

Session 2 (Claude): "Can you continue the auth work?"
  → "What auth module? I don't see any context about OAuth."

Session 3 (Cursor): "We need to refactor the auth flow"
  → "Can you explain the project architecture first?"

Every new session starts from zero. You re-explain the project. You repeat decisions. You waste tokens on context that should already be there.

It gets worse when you work in a team:

Your teammate's AI has zero knowledge of what you've built
Your product specs are in PDF/DOCX — AI can't read binary files
Requirements are scattered across Slack, docs, and meetings

I got tired of this and built fcontext.

What is fcontext?

fcontext is an open-source CLI tool that maintains a .fcontext/ directory in your project. Think of it as a shared brain that every AI agent reads at the start of each session.

pip install fcontext

One command to set up:

cd your-project
fcontext init
fcontext enable copilot   # or: claude, cursor, trae

That's it. Your AI agent now reads project context automatically.

How It Works

Here's what lives inside .fcontext/:

your-project/
  .fcontext/
    _README.md          # AI-maintained project summary
    _workspace.map      # Auto-generated project structure
    _cache/             # Binary docs converted to Markdown
    _topics/            # Session knowledge & conclusions
    _requirements/      # Stories, tasks, bugs
    _experiences/       # Imported team knowledge (read-only)

Each AI agent gets instructions in its native config format:

Agent	Config Location
GitHub Copilot	`.github/instructions/*.instructions.md`
Claude Code	`.claude/rules/*.md`
Cursor	`.cursor/rules/*.md`
Trae	`.trae/rules/*.md`

The instructions teach the agent to:

Read _README.md first to understand the project
Check _topics/ for prior session conclusions
Check _cache/ before asking you about binary documents
Use fcontext req commands for requirements
Save important conclusions to _topics/ before the session ends

Feature 1: Cross-Session Memory

The biggest win. Your AI agent saves session knowledge to _topics/, and the next session picks it up:

# End of today's session — AI saves conclusions
# .fcontext/_topics/auth-debugging.md gets created automatically

# Tomorrow, new session:
# AI reads _topics/ → knows exactly what happened yesterday

# You can also check manually:
fcontext topic list
fcontext topic show auth-debugging

Before fcontext: "Can you explain the project?"
After fcontext: "Yesterday we fixed the OAuth redirect. Ready to implement GitHub provider?"

Feature 2: Cross-Agent Portability

Switch between agents freely. They all read the same .fcontext/ data:

fcontext enable copilot
fcontext enable claude
fcontext enable cursor

# All three agents now share the same context
# Use Cursor for frontend, Claude for backend — no context loss

No vendor lock-in. The context belongs to your project, not to any agent.

Feature 3: Binary Document Indexing

Your product specs are in PDF. Your contracts are in DOCX. Your data is in XLSX. AI agents can't read any of them.

fcontext index specs/product-requirements.pdf
fcontext index contracts/
fcontext index data/quarterly-report.xlsx

fcontext converts them to Markdown and stores them in _cache/. Now any agent can read your documents.

Supported formats: PDF, DOCX, XLSX, PPTX, Keynote, EPUB.

Feature 4: Requirements Tracking

Stop losing requirements across Slack threads and meeting notes:

# Add requirements
fcontext req add "User authentication via OAuth" -t story
fcontext req add "Support Google provider" -t task --parent STORY-001

# Track progress
fcontext req set TASK-001 status in-progress
fcontext req set TASK-001 status done

# Visualize
fcontext req board    # Kanban view
fcontext req tree     # Hierarchy view

Your AI reads _requirements/ and builds against tracked specs — not hallucinated assumptions.

Feature 5: Experience Packs — Team Knowledge Sharing

This is where it gets powerful for teams.

# Team lead: export accumulated project knowledge
fcontext export team-knowledge.zip

# New team member: import it
fcontext experience import team-knowledge.zip

# Their AI instantly knows:
# - Project architecture
# - Domain concepts
# - Coding conventions
# - Known pitfalls
fcontext experience list

You can also share via git:

# Export to a git repo
fcontext export git@github.com:team/domain-knowledge.git

# Import in another project
fcontext experience import git@github.com:team/domain-knowledge.git

# Keep it updated
fcontext experience update

Real-World Scenario

Here's a typical day with fcontext:

Morning — Open a new Claude session. Claude reads .fcontext/_README.md and _topics/. It knows what you did yesterday, what decisions were made, and what's next.

Midday — Product manager sends updated specs (PDF). You run fcontext index specs/v2.pdf. Claude immediately references the new requirements.

Afternoon — Switch to Cursor for frontend work. Cursor reads the same .fcontext/ — no re-explaining needed.

End of day — Claude saves today's conclusions to _topics/debugging-payment-flow.md. Tomorrow's session starts with full context.

Next week — New developer joins. They run fcontext experience import team-pack.zip. Their AI is instantly productive.

Technical Details

Python 3.9+, single pip install
213 tests, CI on Python 3.9/3.12/3.13
Zero cloud dependency — all data in local .fcontext/ directory
Apache 2.0 license
Uses markitdown for document conversion
Plain Markdown and CSV files — no proprietary formats

Get Started

# Install
pip install fcontext

# Set up in your project
cd your-project
fcontext init
fcontext enable copilot   # or: claude, cursor, trae, opencode, openclaw

# Index your documents
fcontext index docs/

# Start coding — your AI now has persistent context

GitHub: github.com/lijma/agent-skill-fcontext
PyPI: pypi.org/project/fcontext
Docs: lijma.github.io/agent-skill-fcontext

I'd love to hear your feedback. Have you experienced the "context amnesia" problem? How do you currently handle context across AI sessions?

Drop a comment or open an issue — I'm actively developing this and every perspective helps.