The Prioritization Matrix: Do spreadsheets really suck?

I'm a big fan of Roger Cauvin and his blog. If you don't read his stuff regularly, you should! He recently published a provocative blog post regarding spreadsheets (tables) many people use to prioritize product investments. I've had very different experiences with (and probably different expectations of) these tools so I thought I would share my personal findings. Not to sound defensive, but I consider this post just part of a conversation, not a rebuttal. BTW, my post will make much more sense if you read Roger's first.

I've been using an approach similar to what Roger describes for years. I call it a prioritization matrix. It's basically a list of things you want to prioritize scored based on a set of criteria. I'll structure this post based on the "three fatal flaws" Roger identified with this approach.

Organizational Dysfunction

In my experience, a simple, transparent method for prioritization helps address one of the key types of organizational dysfunction that leads to sub-optimal (read that as bad!) prioritization: political power or strong personalities trumping data and rational decision making. I've used prioritization matrices both for software investments and for investment decisions regarding entirely new businesses. On more occasions that I can count, I've seen senior leaders or overzealous advocates back off of entrenched positions when their pet investment fared poorly relative to others based on the matrix. This effect is partially dependent on the practice used to complete the matrix. More on that later.

Product Strategy Void

Roger contends that using the matrix exposes a lack of strategic alignment on the team. That hasn't been my experience. As a matter of fact, I've always used alignment with product/organizational strategy as one of the criteria. In my experience, some investments are clearly more "on strategy" than others. For this criterion to be meaningful, an organization must have an agreed upon strategy (something sorely missing all too often). I've written on the topic of the "business motivation model" before. This criterion becomes particularly relevant when some investments are being considered due to strong pressure from a single stakeholder like a key customer or highly opinionated exec.

Distraction from Unique Value Proposition

I agree with Roger's emphasis on the unique value proposition. Interestingly enough, in my experience at "power vendors", the unique value proposition rarely involved features and functions (often, prospects would rather do business with a name brand than a relatively small and unknown company, even if the latter has a superior product!). Regardless, much like product strategy, I would suggest making contribution to the unique value proposition one of the criteria.

I think my perspective on the flaws Roger enumerates is strongly influenced by the guidance I give clients and students regarding the use of such matrices and general expectations regarding the use of this kind of tool. Here are a couple points I consider critical:

Tools Can't Make Decisions

It's unrealistic to expect any tool, much less a simple one, to single-handedly drive complex prioritization. These tools are simply an input to the decision making process and can support a transparent process that makes decision making faster and more effective. As the person that is accountable for product success, you as a product manager will still have to make the final prioritization.

These Tools don't reliably do stack ranking

Far from truly stack-ranking proposed investments, in my experience, these tools tend to identify the following brackets or "buckets" relative to the items being prioritized:

• Investments that are clearly priorities
• Investments that probably aren't wise relative to other investments
• Items that need more thought or research

Put simply, a score of 39 (random number) doesn't mean an investment item is clearly superior to one with a score of 38.

Here are a few other relevant observations about the use of these tools that I've experienced:

• The key to using these tools is to have the right people at the table when using them and making decisions collaboratively (with time for discussion). Too many folks tends to lead to least common denominator decisions. 5 or so folks representing product management, development and executive leadership is usually a quorum.
• Once the team has used to the tool, scoring is extremely quick and efficient in my experience. I was on a team that prioritized relatively complex potential business ideas in 5 minutes or so per item. I'll write more about the practice we used in a future post.
• Assigning weights to the criteria and doing a weighted average can be valuable, although I've had really good luck using a simple average of 7-9 criteria.
• The process of completing the matrix and discussing it should be open, with efforts made to minimize undue influence by executive leaders. This is obviously a sticky wicket.
• The type of “items” being prioritized is important. Avoid prioritizing features as the matrix typically does a poor job of managing dependencies between them. From a product management perspective, I’ve had much better luck using user stories/epics etc. identifying a goal a type of user would like to achieve.
• I would consider using a scoring scale of 4 or 5 (not more). 4 is nice as it provides no “middle ground”.

In summary, I’ve been extremely happy with my use of these matrices. I've found them invaluable in keeping prioritization discussions structured and reasonably fair. Of course no tool or approach is perfect, so I think Roger’s post is valuable in identifying risks that should be considered and mitigated as necessary.

What is your experience using spreadsheets to prioritize product investments?