Tested Into Oblivion.

There's a particular kind of email that arrives in a founder's inbox at 11pm. Subject line: “Quick CRO win — 30% uplift, 14 days, no contract.” Sent from someone who has never visited the site they're proposing to optimise. It is, by some margin, the most-sent email in e-commerce.

The pitch is familiar. Run a few A/B tests. Move the button. Recolour the call-to-action. Add a countdown timer, an exit-intent pop-up, a frequently-bought-together carousel. Watch the conversion rate climb. Pay us a percentage of the lift. Everyone wins.

The maths only works under one condition: you've got the traffic. Real, statistically-meaningful traffic. A test that needs 8,000 sessions per variant to read clean. Most independent brands we work with don't get 8,000 sessions in a month, let alone per variant. They've got 2,000, maybe 3,500 on a good week, and they've been told their job is to “optimise” them.

This is the part of the CRO conversation nobody seems to want to have.

The big agencies built their playbook on Adidas, ASOS, Net-a-Porter — sites where 100,000 sessions a week is a quiet Tuesday. At that volume, a 2% conversion rate that moves to 2.3% is real money and a defensible test. Run the same test on a 10k-MAU site and you'll need six months to know whether the lift was real or whether someone's TikTok went viral on a Wednesday. By the time you have an answer, the design that won has been overtaken by the next round of changes, and the maths has been quietly forgotten.

We've seen this happen enough times to stop pretending it's the rare case. It's the usual case.

Here's the thing we've come back to, again and again: when conversion rates climb on the sites we work on, it's almost never because we ran a test. It's because we did the upstream work.

We rewrote the product page so it explained the thing the brand had been failing to say out loud for two years. We replaced the agency photography with the founder's own iPhone shots taken in a working studio, and the conversion rate didn't lift — it doubled, because the new photos passed a sniff test the old ones couldn't. We shaved a second off the LCP because the hero video was eight megabytes of nothing. We rewrote the size guide. We made the returns page honest. We took the carousel off the homepage.

None of these were tests. They were decisions, made with judgement, defensible from first principles. The conversion rate moved because the product was finally being represented honestly to the people who wanted to buy it. We didn't need a statistical readout to know that.

The CRO industry will tell you this is unscientific. They are correct. They will also tell you that, without testing, you can't know whether the change you made was the right change. They are also correct. What they don't lead with, of course, is that for a small brand the unscientific decision made well usually outperforms the scientific decision made on insufficient data. Because the scientific one isn't actually scientific. It's expensive cover for changes that would have made themselves.

There is a moment in this conversation, usually about two coffees in, when a founder asks the only question that matters: “If you don't run tests, how do you know what's working?”

The honest answer is: you watch. You read the support inbox. You sit on a call with three customers and ask them, in plain English, what nearly stopped them buying. You watch returns rates by product. You look at the search bar inside the site and see what people are typing that they shouldn't have to. You read the abandoned-cart emails out loud and ask yourself whether you'd open one.

These are the signals. They are slower than a Hotjar heatmap and infinitely more useful. They tell you what is actually broken. The tests, when they come, exist to confirm — not to discover.

There is also a category of work that resists attribution entirely. Sentiment, for one. The way a customer feels about a brand — whether they trust it enough to come back, whether they'd recommend it to a friend, whether they're slightly more relaxed when their card is in their hand — does not arrive in the analytics dashboard as a discrete event. Sentiment is just that — ephemeral. It builds, it shifts, it accrues. It does not raise its hand to be measured.

The CRO industry, for understandable reasons, has built itself around the things that can be measured. It does not have a column in its dashboard for the photography is finally honest, or the tone of voice has stopped trying so hard, or the founder has started signing the newsletter again. These moves cannot be A/B tested. They can only be made. And when they're made well, the conversion rate moves, and no one can tell you exactly why.

We've come to the view, after enough years inside enough merchant accounts, that the conversion-rate industry is a dead horse. Not because the underlying discipline is unsound — A/B testing as a tool is fine, and at sufficient scale it is genuinely useful — but because the field has been intellectually static for a decade. The same five tests. The same heatmap rituals. The same urgency timers borrowed wholesale from a 2014 deck and pasted onto a 2026 brand. There has been no new idea in the field for as long as we have been in it. Only new vendors.

We hear this from clients more often than any other request: more analytics. More dashboards. More events. More attribution. A new tool. A second tool. A meeting about whether the tools agree. The implicit theory is that the answer is in the data somewhere — that with one more cohort report, one more session recording, one more funnel visualisation, the truth will assemble itself and the next move will become obvious.

Sometimes that's right. Most of the time, it isn't. Most of the time the answer is staring at the founder from the other side of the desk: the market is no longer what it was. The brand has lost ground that wasn't quite mapped on the dashboard. The product is no longer as sharp as it was three years ago, against competitors who have closed the gap. The pricing has drifted out of step with the value the customer feels. None of these things show up in a heatmap. All of them show up in revenue. Analytics cannot tell you any of that. It can describe what is happening on the site. It cannot tell you what is happening in your market.

The data does not lie, but it can be terribly polite. It will not tell a founder, in plain English, that the brand they built has been overtaken. Looking at the data instead of looking at the market is, often, a way of avoiding the conversation that needs to happen first.

What we would do instead is start with the whole picture. Holistic, used properly. That means looking beyond the funnel, beyond the product page, beyond the things the dashboard is willing to admit. It means looking at your category — at where competitors have closed ground, where they've lost it, where new entrants have changed the price the customer expects to pay. It means looking at your brand's promise against what it now delivers. And it means, most uncomfortably, looking at the things you would rather not see. The product that is no longer as sharp as it was. The story that has stopped being true. The pricing that no longer holds.

The exercise we keep coming back to is the fresh-eyes one. If you were an upstart entering this market today — with none of your sunk costs, none of your inherited assumptions, none of your stock to clear — would you build the business the way it is currently built? Would you sell what you sell? Price it the way you price it? Tell the story you tell? Most founders, asked honestly, find the answer is no. And the gap between what they would build today and what they have actually inherited from their own past is the gap where entropy has been doing its work. Did entropy kill you? In our experience, more often than the market did.

Meanwhile the actual work — the work that has moved merchant revenue, every time, on every site we've touched — has barely been touched: making the product page tell the truth, making the photography honest, making the site fast, making the returns policy something you'd be willing to read out loud at the family lunch. None of this is CRO. All of it is conversion.

Strong design — in the sense Dieter Rams meant: considered, restrained, honest about what it is — is the part of the equation the CRO industry has never been able to price into its model. Good design earns trust. Authority follows. Both convert. The micro-moments matter: button colour, link weight, the rhythm of a paragraph, the spacing of a price next to a delivery promise. But they matter only inside a frame where the whole thing coheres.

Tested in isolation, a button colour is noise. Placed inside design that has been considered from the first principle outward, the same button colour is a small confirmation that the brand has its house in order. And the confidence of a brand that has its house in order is what the customer is buying alongside the product.

Conversion follows clarity. It is, on every site we've worked on, mostly a matter of telling the truth — about the product, the price, the timeline, the return policy, the people behind it — and then making that truth fast and easy to find.

The rest is theatre.

The data does not lie, but it can be terribly polite.