LLMs give terrible website advice

The Validation Economy

It’s never been easier to build things.

AI tools can spin up a landing page, generate copy variants or scaffold an entire product in minutes.

A new class of “builder” has emerged: product managers, marketers, and founders who can ship without writing a line of code.

GitHub predicts there will be 1 billion new builders in the next few years.

But here’s the problem: In a world where building is easy, the hard part becomes knowing what matters and building the right thing

The build-measure-learn loop is fundamentally broken, and AI makes it worse.

AI has turbocharged the build phase, which creates a new bottleneck, and most companies haven’t caught up.

We’re entering what we call the Validation Economy: an era where the ability to quickly confirm what actually works is the most valuable resource.

Take healthcare as an example.

While you can ask ChatGPT for medical advice, it doesn’t have access to the full body of medical literature.

That’s where a platform like OpenEvidence comes in, trained on proprietary medical journals to support the kind of high-stakes decisions where accuracy matters most.

The Missing Piece: “Evals”

The way you make AI build the right thing is with an “eval”.

You define what good looks like, decide how you’ll measure success, then systematically evaluate how close the output aligns with your standard. Engineers run evals to validate their code.

UX needs evals too. Every landing page, every onboarding flow, every pricing layout is a bet on what will move a customer to act.

LLMs generate output, but you need to validate whether that move will actually help or make things worse. Without that, you’re shipping blind.

But here’s the problem… An eval is only as good as its source of truth.

LLMs do not learn from real results. They are trained by blog posts, reddit threads and opinions.

In other words, they are just guessing.

The real source of truth is not what an LLM says, it is what actually wins in real tests online.

That signal is locked inside the A/B tests companies run privately.

This is where DoWhatWorks comes in.

Where the industry is headed

The world needs a source of truth for UX.

You should be able to validate ideas, copy, and designs as fast as you can build them.

We’re about to launch a new product that makes that possible.

We have the data on what actually works in practice.

We built the world’s largest database of A/B website test results over the last six years.

Today, we have captured and tracked over a million data points from experiments run by the world’s most innovative companies.

Our AI is trained on what works, what doesn’t, and why.

We are about to launch a new agent that:

1. Runs instant split tests to validate your ideas, designs, and copy
2. Generates options based on what is proven to work (and iterates on them with you)
3. Tests your experience versus competitors

We want to help you learn from what others got wrong, so you can get to better answers faster.

If you want to help us shape the future of testing, get on the early access list.

‍