How do you prioritize experiments?

  1. Prioritisation models should be contextualised, evaluated and iterated. Don’t forget.
  2. The objective of prioritisation should be alignment, focus and objectivity
  3. The execution is different than the hypothesis. It is my hypothesis where most people get bogged down in prioritisation — because they are too narrow in their execution.
  4. Prioritising experiments is about prioritising everything before the experiment

Alignment.

Purpose of an experiment.

Outcomes vs Outputs.

  1. The experiment is so bedded into in the execution, not the concept. Meaning if the experiment ‘fails’ as it were, any analysis or lack thereof may not highlight that it was the execution that failed, not the concept. And you get a lot of hopping from one to the next…
  2. An over-structured process that lacks fluidity. Often as a result of having a ‘testing to win’ attitude rather than a ‘testing to learn’ attitude.
  • the experiment output is less about the revenue attribution, and more about the leanings learning toward the objective (given the muddy waters with experiment financial attribution, which is completely a different post). It was a win-win-win.
  • OKRs gives focus. A series of experiments focussed on improving a specific feature, experience or behaviour is 100x more impactful than a series of experiments arbitrarily ‘prioritised’ within a spreadsheet.
  • As well as alignment. Both client and agency know what we are working towards and how each experiment fits into that piece of the puzzle.
  • It breaks down “conversion rate optimisation” into manageable chunks. There’s always been debate of the “conversion rate” within that phrase, and so, removing the ambiguity and leaving just “optimisation” helps when looking at micro and macro metrics.
  1. What did we learn
  2. What does that mean
  3. What next
  1. Prioritisation models should be contextualised, evaluated and iterated. Don’t forget.
  2. The objective of prioritisation should be alignment, focus and objectivity
  3. The execution is different than the hypothesis. It is my hypothesis where most people get bogged down in prioritisation — because they are too narrow in their execution.
  4. Prioritising experiments is about prioritising everything before the experiment
  1. Please read, please comment, please share. I learn more from the thread of the your reactions than reading 20 different articles on a single topic.
  2. Ask questions and vote https://app.sli.do/event/orj9eqab/live/questions. I answer these on a weekly basis by taking the top voted topic and writing about it with my stories and experience.
  3. Subscribe to my newsletter at https://optimisation.substack.com/

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
David Mannheim

David Mannheim

Stories and advice within the world of conversion rate optimisation. Founder @ User Conversion. Global VP of CRO @ Brainlabs. Experimenting with 2 x children