DEV Community

Cover image for My tests were green. The window was on the wrong monitor.
Raymond Chang
Raymond Chang

Posted on

My tests were green. The window was on the wrong monitor.

Every key on my options trading desk was written with an AI coding assistant. The desk runs from a Stream Deck: one key picks the strategy, another moves the strikes, CHECK runs my rules, and SEND places the order. Everything you see here runs in a sandbox account, and nothing in this post is a trade signal.

People keep asking me the same question. If the AI writes the code, how do I trust what it sends? This is the workflow I use, step by step, with the two misses that taught me the most.

Step 1: a spec with numbers, before any code

The idea starts as one line in my notes. For the payoff chart it was "I want to see the trade before I send it."

I have the AI turn that line into a spec. It drafts the scope and the math, and it lists the decisions it is not allowed to make for me. Where does the chart sit on screen? Which formula gives the probability of profit? I make those calls.

Then come the expected numbers. A put spread, 740 over 735, for a 1.50 credit. Max profit 150. Max loss 350. Breakeven 738.50. I worked those out by hand before a single line of code existed.

Step 2: a test that fails

The first test asks for exactly those numbers. The code doesn't exist yet, so the test fails. If a test passes before the code exists, it isn't testing anything.

Step 3: the AI writes until the test passes

This part is fast. The AI writes code faster than I can read it, which is exactly why the tests get the final word on when it's done.

Step 4: break the code on purpose

This is mutation testing. I flip a greater-than or delete a line, then run the tests again. If they still pass, the tests are too weak.

On the payoff chart I tried eighteen broken versions. Two survived the first round: a trade that pays exactly zero, and expiry day. Two more tests later, all eighteen got caught.

Step 5: the whole suite, on every change

The desk has over 5,000 tests. They run on my own machine on every change, before anything gets committed. The suite caught one problem right away: the first chart version drew itself inside the order screen's update loop, so a chart crash would have marked my order screen as not updating.

Step 6: look at the real thing, on the real machine

This is the step I almost skipped.

Every test was green. I started the desk, and the chart window opened on my vertical monitor, squeezed to about two thirds of its width, sitting on top of the order screen.

My tests checked the position I asked Windows for. Nothing checked where Windows actually put the window. I caught it on a screenshot.

So I measured it on that machine before writing the fix. Setting the position before the window is shown doesn't stick. Setting it, showing the window, and setting it again does. That sequence is the fix, and a test now checks the real position after the window is on screen.

The miss that was on the AI

The same week, the AI ran the full suite and told me everything was clean. It had searched the output for failures and found none.

One test file had crashed before it printed a result at all. No result meant no failure line to find. My own run on Windows showed it.

My test runner lists every file that never reported a result. I read that list now, along with the failure lines.

Step 7: I commit it myself

The AI never commits. I run the suite myself, look at the result on screen, and then commit. My name goes on it, so my hands go on it.

The check you can copy

When your code puts something somewhere, a window or an order, test where it actually ended up, on the machine it runs on. Asking for the right thing is only half the test.

I walk through all seven steps on video, including the wrong-monitor screenshot:

I'm Raymond Chang. I run Codebat, a post-quantum cryptography hardware company, and I build my trading tools on the side with an AI coding assistant. The channel is Dr_Delta on YouTube.

Top comments (0)