(2025-10-20) Gilad The Total Impact Matrix Beyond Blind Bets

Itamar Gilad: The Total Impact Matrix - Beyond Blind Bets. Impact is a central concept in product development, but it’s often unerdused or misused. In this article I want to suggest a thinking tool, The Total Impact Matrix, to help you use it better.

The impact of an idea is the sum of value it creates for our target audience — customers/users/partners — and for the business. Unlike outcomes that look at short term changes lower in the metrics tree, impact looks at the broader effect an idea may have.

Impact = Value to customers + Value to Business

Let’s assume we’re able to assess the value an idea delivers to the customers and to the business. The classic consultant move is to lay those on an X-Y matrix:

Now let’s fill out the space with...

We created an impact map of sorts with these main areas:

Um, Something’s Not Right

Product ideas face a sad, rarely discussed, reality: most don’t work.

Here are the success rates — % of ideas that created positive change — as measure by various companies and product units:

• Microsoft: 35%
• Slack.com monetizations: 30%
• Netflix: 35%
• VWO and Convert.com (experimentation platform) 14% of conversation rate improvements
• Booking.com: 10%
• Airbnb Search: 8%.

Another insight from split tests is that most ideas generate just a small positive effect, fewer create a medium effect, and far fewer deliver really big wins — an effect power law. (see (2016-09-22) Corey Haines Methodology Per Business)

To reflect this new information in our matrix we need to change the X and Y scales. Here’s what a realistic impact matrix may look like:

But It Gets Worse

Another lesson from A/B tests and real-world launches is that some ideas cause negative results.

Adding the negative options gives us three new impact areas, and they’re all bad:

The good news is that unlike the roulette, product development does not mandate blind bets. Small investments in research and product discovery — for example mapping out assumptions, interviewing users, running a usability test, launching an early adopter program — can greatly increase our odds to find the winners and drop the losers.