The test that catches the race your agent silenced

Last week I showed an agent making a Swift 6 data race disappear with one word. It marked a struct @unchecked Sendable, the build went green, every test still passed, and the race was exactly as present as before. The post ended on the question I didn't have a clean answer to: when the build and the tests both go green on a silenced race, what in the loop is supposed to catch it?

A red build is a signal you can trust. A silenced race hands you a green build you can't, and the only thing between you and shipping it is you, reading the diff. I spent this week building tests that put the red back on the board. Here is the one that worked, and the honest edges of it.

The setup from last time: a value type that crosses into concurrent code, carrying a mutable class the agent no longer wanted the compiler to complain about.

public final class AuditPen {
    public var ink: Int
    public init(ink: Int) { self.ink = ink }
}

public struct Transfer: @unchecked Sendable {
    public let amount: Int
    public let memo: String
    public let pen: AuditPen   // mutable, shared, and now unchecked
}

Here is the test a person writes for this without thinking about concurrency at all:

@Test func happyPath() {
    let t = Transfer(amount: 100, memo: "rent", pen: AuditPen(ink: 0))
    #expect(t.amount == 100)
}

It builds. It passes. It will pass on the silenced version forever, because it never once runs two things at the same time. The assertion is about the amount. The bug is about pen. They never meet. That is the shape of almost every test sitting next to a silenced race. It checks a value on the happy path, and the race needs a second writer on the board before anything goes wrong.

So put the second writer into the test setup itself, and assert the thing the race breaks. The test's job is to create the exact situation the Sendable promise was about: more than one task writing pen at the same time.

@Test func concurrentBumpsMustNotLoseUpdates() {
    let n = 1000
    let t = Transfer(amount: 100, memo: "rent", pen: AuditPen(ink: 0))
    DispatchQueue.concurrentPerform(iterations: n) { _ in
        t.pen.ink += 1
    }
    #expect(t.pen.ink == n)
}

concurrentPerform runs the block across several threads. Each one reads ink, adds one, and writes it back, with nothing coordinating them. When two threads land on the same starting value, one of the updates is lost. On the silenced @unchecked Sendable version this fails every single time I run it. The count comes back short, sometimes by a handful, sometimes by hundreds, never exactly the target. The amount you lose wobbles from run to run. The failure does not.

Here is what makes that a real check and not just a way to make concurrent code look bad. Fix the type properly and the same test goes green and stays green.

public final class AuditPen: @unchecked Sendable {
    private let lock = NSLock()
    private var _ink: Int
    init(ink: Int) { self._ink = ink }
    func bump() { lock.lock(); _ink += 1; lock.unlock() }
    var ink: Int { lock.lock(); defer { lock.unlock() }; return _ink }
}

Now @unchecked Sendable has moved down to where the synchronization actually lives, and it finally means something. There is a lock behind the promise. Transfer can drop back to a plain Sendable. The concurrent test passes a hundred runs out of a hundred, because the updates can no longer step on each other. That is the property I want from the test: red when the type only claims to be safe, green when it is. The agent writes good Swift. What it can't do is tell a real Sendable conformance from a hollow one, and that gap is exactly what this test pins down.

If you know this space you have been waiting for me to say ThreadSanitizer. It is the textbook answer, and in theory it is the better one. TSan watches memory at runtime and reports the race directly, even on the odd run where the lost updates cancel out and the count happens to come back right. The behavioral test can miss that run. TSan will not.

I tried to make TSan the center of this post. On my current toolchain I could not get it to run. swift test --sanitize=thread builds and then dies before a single test executes, because the sanitizer's own runtime library refuses to load: a code-signing and platform-policy rejection on the toolchain's copy of the dylib. A standalone executable built with -sanitize=thread fails the same way. That is a setup problem specific to this Xcode, not a verdict on TSan, but it is the reason I lead with the plain test. The behavioral version needs nothing but the test runner you already have. When TSan is wired up it is a real upgrade and it belongs in CI on anything concurrency-heavy. Wiring it up is its own afternoon, and it depends on your Xcode version cooperating.

None of this is a fix for the general problem, and I want to be exact about why. The test only catches the adversary you thought to write. If nobody writes the concurrent-writer case, there is no red to chase and you are back to reading diffs by hand. The assertion is only as sharp as the contention it creates. On a debug build the losses can be small, and if you do not push the concurrency hard enough a race can still slip through on a quiet run. And it does nothing for the other failure modes. An agent that builds the wrong feature cleanly will sail through every adversarial concurrency test you own.

What it buys is narrow and real. For the one failure mode where the build and the existing tests both lie to you, it gives the loop something to fail on that a confident paragraph can't argue its way past. The agent can write @unchecked Sendable and explain why it was the reasonable call. It cannot make a thousand concurrent increments add up to a thousand without actually doing the synchronization.

So here is where I have landed and what I am still missing. For concurrency-sensitive types, do you hand-write these adversary-carrying tests, or have you found a way to generate the "exercise it under contention" case for every Sendable you declare? And if you run TSan in CI, did your toolchain just work, or did you have to fight it to get green?