The most expensive ad I ever ran cost about nine hundred pounds to find out it was bad.
Not nine hundred in production. Nine hundred in media, spent over five days, learning something that in hindsight three people in the room could have told me on the Friday before launch. The hook was weak. Everyone sort of knew. Nobody could say so in a way that survived contact with "well, let us just test it and see".
That phrase has cost our industry an enormous amount of money.
Testing is not free, it just feels free
The orthodoxy is that you cannot know until you test, so test everything and let the data decide. I believed this for years and I still think it is mostly right. But "mostly right" has been doing a lot of load-bearing work.
Every test has a floor. You need enough spend for significance, enough days to clear the learning phase, enough impressions for the platform to find anyone. For most accounts that floor is somewhere between a few hundred and a few thousand pounds per variant before the result means anything.
So when someone says let us test it, what they are proposing is to pay the floor price to answer a question. That is a perfectly reasonable thing to do when the question is genuinely open. It is a waste when the answer was available in advance and nobody had a vocabulary for saying it.
Opinions do not survive meetings, scores do
The real problem with pre-launch judgment is not that it is unavailable. It is that it is unarguable.
When I said "I think the hook is weak", I was offering a feeling, and feelings lose to "let us test it and see" every time, because the person proposing the test sounds rigorous and the person with the reservation sounds precious. I have been on both sides of that exchange and the test always wins.
What changes the dynamic is making the reservation specific and attached to evidence. Not "the hook is weak" but "the opening three seconds score poorly against the eleven creatives that worked for us this year, and here is the element they share that this one does not have".
That is a sentence you can disagree with on the merits. It is also a sentence that survives a meeting.
Scoring the parts, not the ad
This is the shift that mattered for me: scoring at the element level rather than the asset level.
An overall score for an ad is close to useless. It tells you the thing might be bad without telling you which decision inside it is the problem, which means the only available action is to make another whole ad and hope.
In Hawky's command center each creative gets scored per component, headline, visual, call to action and overall composition, against the patterns from your own winners. What comes back is not a verdict but a location: this visual is consistent with what has worked, this call to action is not, here is the element from your winning set that would usually be doing that job.
The value is not the number. It is that the conversation moves from whether the ad is good to which part of it is carrying risk, and that is a conversation a creative team can actually act on without feeling it is being marked.
What a score cannot tell you
It is measuring consistency with what has worked before, which means by definition it will be unenthusiastic about anything genuinely new. The best performing ad we ran last year scored poorly, because it did the opposite of our house style on purpose. If I had treated the score as a gate rather than a prompt, I would have killed it.
That is the real risk here, and it is not a technical one. A scoring system in a team with a nervous culture becomes a reason to say no, and you end up optimising towards a local maximum of things that look like last quarter.
It also cannot see the offer, the landing page, the season or what your competitors launched on Tuesday. It is scoring the creative in isolation, and creative does not perform in isolation.
And a low score is not a prediction. It is a flag that this one departs from a pattern. Sometimes departing is the entire point.
Closing Thoughts
I am not arguing against testing. I am arguing against using testing as a substitute for having a view, because that is what "let us test it and see" usually is.
The cheapest question is the one you answer before the media spend starts. Not every question can be answered there, and the ones that cannot are exactly what testing is for. But a meaningful share of what we send to market carries a flaw someone in the room could have named, and the reason it goes out anyway is that naming it was socially expensive and the evidence was not to hand.
Make it cheap to say "this part looks risky, and here is why". You will still run the test. You will just run fewer of the ones you already knew the answer to.
Top comments (0)