Your gut feeling is costing you $60 million.
The secret behind Barack Obama’s victory wasn’t just charismatic speeches.
It was a simple A/B test run by a Google alum.
Before Dan Siroker co-founded Optimizely, his job as Director of Analytics for the campaign was to get people to click a button on a website.
The campaign’s high-paid political consultants were absolutely convinced they knew what the people wanted.
They insisted on using an emotional video of Obama speaking, paired with a giant button that said "Sign Up Now".
Because of course they did—"experts" love relying on vibes over data.
Siroker ignored them and set up a multivariate test instead.
He pitted 4 different buttons against 6 different media formats (3 photos and 3 videos).
The results were a complete embarrassment for the consultants.
Every single video they fought for performed significantly worse than the static images.
The ultimate winner?
A low-friction button that said "Learn More" paired with a warm, black-and-white photo of the Obama family.
📊 The ROI of One Simple Test
By listening to the data instead of the highest-paid people in the room, the campaign saw a 40.6% increase in conversions.
Here is how that split-test changed history:
📈 Sign-Up Rate: Jumped from 8.3% to 11.6% (+40.6% Lift)
👥 Total Sign-ups: Gained an extra 2.9 million email addresses
🥾 Volunteers: Recruited an extra 288,000 boots on the ground
💰 Donations Raised: Generated an extra $60,000,000 in small donations
That is an extra $60 million in small-dollar donations, all because they didn’t let a room full of suits dictate user experience.
Statistics teaches you the ultimate lesson in humility.
Your opinion on what your customers want is usually wrong.
My opinion on what my customers want is usually wrong.
The only opinion that actually matters is the statistical significance of a live user interaction.
Stop arguing in meeting rooms about which landing page headline sounds "punchier."
Build both, run the traffic, and let your users tell you who is right.
Unless you have $60 million to burn on being wrong, of course.
Top comments (0)