DEV Community

Cover image for ChatGPT Ran 450 Tests on My Windows PC — and Reported Back in the Same Chat
kurojack-jp
kurojack-jp

Posted on

ChatGPT Ran 450 Tests on My Windows PC — and Reported Back in the Same Chat

In my previous post I introduced Sora Commander, a free, open-source MCP server that lets ChatGPT read files and run commands on my own Windows PC from an ordinary chat. The fair question that followed was: fine — but what does it actually change in day-to-day work?

Here is one real run, from my own development work.

Sora Commander demo — ask in ChatGPT, approve on your PC, get the result back

The copy-paste loop I wanted to kill

Anyone who codes with a chat AI knows this dance:

  1. "I get this error."
  2. "Please run this command."
  3. Open PowerShell. Run it. Copy the output.
  4. Paste it back into the chat.
  5. "Now please run this one." — repeat.

Dedicated coding agents exist, of course. What I wanted was different: keep working with the ChatGPT I already talk to, and let it do the running.

The run: a full check of my other project

I have a second project, Open Guild Forge (OGF). One day I asked ChatGPT, through Sora Commander, to run OGF's full check suite and tell me the result.

What happened, in order:

  1. My request went from the chat to my PC.
  2. An approval window opened on my PC screen; I approved the exact command.
  3. The suite ran on Windows.
  4. ChatGPT waited for it to finish, read the log saved on the PC, and summarized the outcome in the same chat.

The results:

  • Vitest: 342 passed
  • Node.js tests: 3 passed
  • Rust tests: 105 passed
  • TypeScript type check: passed
  • Vite production build: passed
  • Rust Clippy / rustfmt: passed
  • Prettier: passed

450 tests, 0 failures, exit code 0. (There were warnings — a deprecated Three.js API and bundle size — but the run itself was clean.)

The part that actually matters

The test count is not the point. The point is what I didn't do: I never opened a terminal, and I never copy-pasted a single line of output. The chat requested, I approved, the PC executed, the chat collected and reported.

And one detail from that same session shows why I built the recovery side of this tool. Before the run, the connection dropped once. When it came back, ChatGPT did not just send the request again — it first checked the state of the earlier execution, confirmed nothing was running, and only then sent the request, exactly once.

Running a command is easy. Knowing what happened to it — especially after a disconnect — is the hard part. A tool that shrugs and re-runs an operation whose outcome is unknown is a tool that will eventually run something twice. With tests that's harmless; with a file-changing operation it isn't.

A small bonus: starting it by voice

Because this runs inside an ordinary ChatGPT chat, I can also kick off checks like this by voice via ChatGPT Live — "check the current state of the project," "any test failures?" The approval for anything that executes still happens in the window on my PC, in person. Voice changes the input method, not the safety model.

Honest caveats

I want to be precise about what this run does and doesn't prove:

  • The 450 tests are OGF's tests, not Sora Commander's. They say nothing about Sora Commander's own quality or safety. Sora Commander was the courier here, not the thing being tested.
  • An approved command runs with my full Windows user rights. The approval window and the folder restrictions reduce risk; they do not sandbox anything. Approving something you don't understand is on you — that is the design, stated plainly.
  • Sora Commander is a v0.1.5 developer preview (AGPL-3.0). It needs Windows 11, Python 3.13, and an environment that supports custom MCP connections via OpenAI's Secure MCP Tunnel. I have verified it on my own machine and in Windows Sandbox, not on the wide variety of real PCs out there.

How it fits with other tools

In my own setup, Sora Commander is not the only worker. A common pattern for me is: ChatGPT thinks about what to build or fix, Codex CLI does focused implementation and test work, and Sora Commander is the connective tissue on the Windows side — executing and collecting results back into the chat. Each tool does what it is good at.

One caveat on that, too: the delegation workflow around Codex CLI is something I built up in my own (in-development) environment. It is not a standard feature that ships in the public v0.1.5 release.

Try it

If something breaks, an Issue helps. Security issues only — please use the repo's private vulnerability reporting, not a public Issue.

If you want the guided walkthrough

The setup instructions in the repo are free. If you'd rather follow the whole setup as one guided path — real screenshots at every step, the common errors and what to do about them, and the security settings explained — I also wrote it up as a complete beginner's guide: Sora Commander — The Complete Beginner's Guide (Gumroad, $15)

The tool started as "please stop making me copy-paste." It turned out the interesting problems were all in what happens after you press run.

Top comments (0)