TLDR; Experimentation has its place. Intuition has its place. There is inherent bias with validating the new be that a pre-conditioned way of thinking, fear or timing which is what I want to explore…

“Don’t experiment when you should think; don’t think when you should experiment” Jon Bentley

This is going to sound counter-intuitive coming from someone who has built his career up evangelising experimentation. And as someone who considers creativity one of his core values. But I’m not sure the two work that well hand in hand.

I genuinely feel that experimenting can restrict creativity and innovation just as…


TLDR; Start to re-frame the conversation about experimentation. Talk about expected loss as much as expected gain, use OKRs to focus and align, talk in ranges, use degradation, highlight your program quality metrics.

This is part of a 4-article series on whether, and how, you can attribute revenue to experimentation:

  1. Why do we assume experimentation is about financial gain?
  2. No, you can’t accurately attribute, nor forecast, revenue to experimentation. Here’s why.
  3. Yes. We need to attribute revenue to experimentation. Here’s how: Part 1
  4. Yes. We need to attribute revenue to experimentation. Here’s how: Part 2

In the first part, we…


TLDR: Communication is key. Align and define on your purpose of experimentation for the business. Set your standards and therefore expectations. Define what accuracy is acceptable. And tell stories.

Woah, this is a long one. Sorry. So much so it’s a two-parter. I actually think the second part is better than the first part, too; this is Joss Whedon — all about communication — and next Friday is Zach Synder — the Batman.

This is part of a 4-article series on whether, and how, you can attribute revenue to experimentation:

  1. Why do we assume experimentation is about financial gain?
  2. No…

TLDR; I don’t believe you should attribute revenue to individual experiments, but I get the fact that we have to; it’s fuzzy. I don’t believe you can accurately forecast that revenue attribution; it gets even fuzzier. And, when you collate experiments together to determine overall revenue impact? I certainly don’t believe you can attribute that type of revenue; that’s super fuzzy. Experiment results tell you which way you’re headed, not how far you’re going.

This is part of a 4-article series on whether, and how, you can attribute revenue to experimentation:

  1. Why do we assume experimentation is about financial gain?

The over-simplification and virality of click-bait headlines for case studies designed for commercial gain and a wider audience has led to associating financial gain to an output. I blame Optimizely, the $300m button and WhichTestWon.

This is part of a 4-article series on whether, and how, you can attribute revenue to experimentation:

  1. Why do we assume experimentation is about financial gain?
  2. No, you can’t accurately attribute, nor forecast, revenue to experimentation. Here’s why.
  3. Yes. We need to attribute revenue to experimentation. Here’s how: Part 1
  4. Yes. We need to attribute revenue to experimentation. Here’s how: Part 2

A beautiful juxtaposition…


TLDR; I am not a believer that you should test everything and, instead, it should have a swim-lane of prioritisation. Testing is a (brilliant) methodology for decision making. Factors that affect this inc. purpose, traffic, type of test and time.

Reminder; There’s no such thing as right or wrong, just contextualised experiences. I’ll be sure to give my opinion, with others, on questions people have asked at my Slido. Please comment, like, share, debate, drink when you’re done. I’d love us learn more and share our experiences together to help our CRO community. …


TLDR; You can’t do it alone. It must be democratised in the form of structure and mindset. If you educate, encourage failure, gamify and demonstrate humility, I think you’ll take a good swing at it.

Every week, I answer the top voted question in my Slido. There are currently 37 questions on there. Id love for you to upvote or contribute to ask a question about my experience or if I can crowd-source other opinions.

Please also leave a comment on this article and give me your opinion. …


TLDR; Prioritise down, not up. It’s not about prioritising experiments. It’s about prioritising everything that comes before the experiment.

Previously I spoke about why prioritisation is so damn difficult. I thought it was important to highlight that first so we can start to frame the thinking, and think about what we need in our prioritisation models for experimentation. Those things being alignment, focus and objectivity.

I’m going to save you a tonne of reading and bullet point how, I think, prioritisation should be done. I’m hoping you’re going to say “David, those bullet points are platitudinal. Who are you Boris…


TLDR; Prioritisation is bloody hard. That’s because we either over-simplify it, or try to append arbitrary and binary frameworks to the problem, or focus on outcomes rather than value, or don’t take into account iteration, or don’t take into account alignment with the wider business objectives, or isolate experiments into a pot ignoring the wider context of the ‘value add’. See. It’s bloody hard, isn’t it?

Most of my conversations go like this on a day-to-day basis.

“How are you”

“Busy. What about you?”

”Busy, too”

We’re always busy. There’s never enough “time” in the day. And as a father…


I wrote, read, and re-wrote this a number of times. Not because I’m unclear on the why; I’m resolute and content. Dare I say excited. But my journey of building User Conversion has been an eye-opening, identity-finding, life re-prioritising one. Full of demons, hurdles and hard knocks, but also white knights, new friends and opportunity.

I recently sold User Conversion to, what I would consider to be, the most reputable performance marketing agency in the UK (and beyond).

It’s fantastic. Joining forces with a company who built its reputation — and even brand logo — on experimentation. Furthering it’s reputation…

David Mannheim

Stories and advice within the world of conversion rate optimisation. Founder @ User Conversion. Global VP of CRO @ Brainlabs. Experimenting with 2 x children

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store