My Pest test passed because I added a helpful failure message

I write marketing copy about my own products and I have a bad habit of leaving claims in it that stop being true. Prices change, a feature moves tier, and the blog post from March keeps happily saying the old thing. So I wrote a test that reads the published posts and asserts the dead claims are gone.

expect($post)->not->toContain('$29/mo', 'pricing changed in June, this post is stale');

Green. Every one of them green. The string $29/mo was still sitting in the post.

What that second argument actually does

Here is the signature, straight out of vendor/pestphp/pest/src/Mixins/Expectation.php:

public function toContain(mixed ...$needles): self
{
    foreach ($needles as $needle) {
        if (is_string($this->value)) {
            Assert::assertStringContainsString((string) $needle, $this->value);
        } else {
            if (! is_iterable($this->value)) {
                InvalidExpectationValue::expected('iterable');
            }
            Assert::assertContains($needle, $this->value);
        }
    }

    return $this;
}

It is variadic. My careful explanatory message was never a message. It was needle number two.

The positive expectation is therefore a conjunction: contains A and contains B. Which means the negation is "missing at least one of them", and my sentence about June pricing was guaranteed to be missing.

How not gets there

Pest does not implement a separate negative assertion. OppositeExpectation::__call runs the positive one and catches the failure:

public function __call(string $name, array $arguments): Expectation
{
    try {
        $this->original->{$name}(...$arguments);
    } catch (ExpectationFailedException|AssertionFailedError) {
        return $this->original;      // positive failed, so `not` succeeds
    }

    $this->throwExpectationFailedException($name, $arguments);
}

So follow my call through. The loop checks $29/mo, finds it, passes. Then it checks pricing changed in June, this post is stale, does not find it, throws. That throw is caught, and not reports success.

The first needle was checked. The result was thrown away.

Why this one is so easy to walk into

I counted the public methods on that mixin. There are 68 expectations. 66 of them take a real message parameter.

public function toBe(mixed $expected, string $message = ''): self
public function toBeFalse(string $message = ''): self
public function toStartWith(string $expected, string $message = ''): self

Two do not: toContain and toContainEqual.

toStartWith is defined forty lines below toContain in the same file, and it takes a message. So the habit is correct 66 times out of 68 and you are not misremembering the library, you are remembering the other 97% of it. At the call site a variadic parameter and an optional trailing string are the same keystrokes. Nothing about ('needle', 'my message') looks different from ('needle', 'other needle').

The four cases, run

Pest 4.4.1 on PHP 8.4.13, value is 'hello world':

assertion	result
not->toContain('zzz', 'qqq')	passes, correctly
not->toContain('hello', 'qqq')	passes, and 'hello' is right there
not->toContain('qqq', 'hello')	passes
not->toContain('hello')	fails, correctly

Only the single-needle form does what you read it as doing. Add anything after it and the assertion turns into a coin flip you always win.

What I do now

For the check I actually wanted:

expect(str_contains($post, '$29/mo'))->toBeFalse('pricing changed in June, this post is stale');

toBeFalse has a real $message, so the sentence lands where I meant it to and the assertion is about one thing.

The wider habit I picked up from this: if an assertion is load-bearing, make it fail once on purpose before you trust it. I had written a test specifically to catch a stale claim, the stale claim was present, and the test was green. Deleting the second argument turned it red immediately. That took ten seconds and I had not done it, because the test was passing and passing tests do not feel like they need investigating.

That is the actual failure. Not the variadic signature, which is documented and reasonable. It is that a green test reads as evidence, and a test that cannot fail is green in exactly the same shade as a test that just did its job.