I analyzed 10,242 website A/B tests, here is what the data says

Casey Hill on Marketing by Design, ranking real homepages live and explaining why the findings that surprised him most were the ones everybody already does.

Show
Marketing by Design
Host
MMG Studio
Guest
Casey Hill, DoWhatWorks
Runtime
50:00

What is covered

01 The two findings that genuinely surprised him Logo bars, which sit on nearly every SaaS homepage and lose at a very high rate. And video in the hero, which kept losing to static imagery across hundreds of tests.
02 The expectation to reality gap Autoplaying GIF-style video does better than a video you have to click, because people do not know what they are about to get. The same gap explains why Learn more underperforms.
03 Why job function beats industry A VP of product marketing and a CFO work at the same company and arrive with completely different priorities. Role is closer to how people think about themselves.
04 Strike-through pricing lands at D tier The bigger the discount you advertise, the more it signals the original price was never real. Trust takes the hit, in both B2B and consumer.
05 Cost signalling as a test for social proof From a biologist who studies signalling in animals: the harder something is to fake, the more credible it is. Elk antlers are hard to fake. A G2 badge from 2023 is not.

Most website debates inside marketing teams come down to one person’s opinion against another’s, dressed up with enough confidence to sound like expertise.

This episode goes through what the database says instead, including a live tier ranking of real homepages, graded on air.

Most website decisions are made with more confidence than the evidence warrants.

What to take from it

  • Audit your hero for video that requires a click. Static imagery or an autoplaying loop usually beats it.
  • Segment by role before you segment by industry, and put the selector high on the page.
  • Ask how easily each piece of your social proof could be faked. That is the whole test.
  • Drop strike-through pricing unless the original price was genuinely being charged.