Training Banner

Thursday, August 6, 2015

The Prioritization Matrix: Do spreadsheets really suck?

I'm a big fan of Roger Cauvin and his blog. If you don't read his stuff regularly, you should! He recently published a provocative blog post regarding spreadsheets (tables) many people use to prioritize product investments. I've had very different experiences with (and probably different expectations of) these tools so I thought I would share my personal findings. Not to sound defensive, but I consider this post just part of a conversation, not a rebuttal. BTW, my post will make much more sense if you read Roger's first.

I've been using an approach similar to what Roger describes for years. I call it a prioritization matrix. It's basically a list of things you want to prioritize scored based on a set of criteria. I'll structure this post based on the "three fatal flaws" Roger identified with this approach.

Organizational Dysfunction
In my experience, a simple, transparent method for prioritization helps address one of the key types of organizational dysfunction that leads to sub-optimal (read that as bad!) prioritization: political power or strong personalities trumping data and rational decision making. I've used prioritization matrices both for software investments and for investment decisions regarding entirely new businesses. On more occasions that I can count, I've seen senior leaders or overzealous advocates back off of entrenched positions when their pet investment fared poorly relative to others based on the matrix. This effect is partially dependent on the practice used to complete the matrix. More on that later.

Product Strategy Void
Roger contends that using the matrix exposes a lack of strategic alignment on the team. That hasn't been my experience. As a matter of fact, I've always used alignment with product/organizational strategy as one of the criteria. In my experience, some investments are clearly more "on strategy" than others. For this criterion to be meaningful, an organization must have an agreed upon strategy (something sorely missing all too often). I've written on the topic of the "business motivation model" before. This criterion becomes particularly relevant when some investments are being considered due to strong pressure from a single stakeholder like a key customer or highly opinionated exec.

Distraction from Unique Value Proposition
I agree with Roger's emphasis on the unique value proposition. Interestingly enough, in my experience at "power vendors", the unique value proposition rarely involved features and functions (often, prospects would rather do business with a name brand than a relatively small and unknown company, even if the latter has a superior product!). Regardless, much like product strategy, I would suggest making contribution to the unique value proposition one of the criteria.

I think my perspective on the flaws Roger enumerates is strongly influenced by the guidance I give clients and students regarding the use of such matrices and general expectations regarding the use of this kind of tool. Here are a couple points I consider critical:

Tools Can't Make Decisions
It's unrealistic to expect any tool, much less a simple one, to single-handedly drive complex prioritization. These tools are simply an input to the decision making process and can support a transparent process that makes decision making faster and more effective. As the person that is accountable for product success, you as a product manager will still have to make the final prioritization.

These Tools don't reliably do stack ranking
Far from truly stack-ranking proposed investments, in my experience, these tools tend to identify the following brackets or "buckets" relative to the items being prioritized:
  • Investments that are clearly priorities
  • Investments that probably aren't wise relative to other investments
  • Items that need more thought or research
Put simply, a score of 39 (random number) doesn't mean an investment item is clearly superior to one with a score of 38.

Here are a few other relevant observations about the use of these tools that I've experienced:
  • The key to using these tools is to have the right people at the table when using them and making decisions collaboratively (with time for discussion). Too many folks tends to lead to least common denominator decisions. 5 or so folks representing product management, development and executive leadership is usually a quorum.
  • Once the team has used to the tool, scoring is extremely quick and efficient in my experience. I was on a team that prioritized relatively complex potential business ideas in 5 minutes or so per item. I'll write more about the practice we used in a future post.
  • Assigning weights to the criteria and doing a weighted average can be valuable, although I've had really good luck using a simple average of 7-9 criteria.
  • The process of completing the matrix and discussing it should be open, with efforts made to minimize undue influence by executive leaders. This is obviously a sticky wicket.
  • The type of “items” being prioritized is important. Avoid prioritizing features as the matrix typically does a poor job of managing dependencies between them. From a product management perspective, I’ve had much better luck using user stories/epics etc. identifying a goal a type of user would like to achieve.
  • I would consider using a scoring scale of 4 or 5 (not more). 4 is nice as it provides no “middle ground”.

In summary, I’ve been extremely happy with my use of these matrices. I've found them invaluable in keeping prioritization discussions structured and reasonably fair. Of course no tool or approach is perfect, so I think Roger’s post is valuable in identifying risks that should be considered and mitigated as necessary.

What is your experience using spreadsheets to prioritize product investments?

9 comments:

  1. BTW, I would add that prioritization matrices are most valuable very early in the planning/scoping process. They can be a very efficient means of "triaging" investments (see "buckets" enumerated above).

    ReplyDelete
  2. Greg, thanks for the thoughtful response to my blog entry. A few clarifications:

    First, you wrote that the data-driven approach to prioritizing can address the problems of political power and strong personalities. We may have very different experiences here. In my experience, The Spreadsheet doesn't solve either of these problems. The problems still manifest themselves. Those who abuse their strong personalities or political power in other contexts will do so in The Spreadsheet exercise:

    "I think we should include XXX as a column in The Spreadsheet."
    "I definitely think the rating in that column should be an 8."

    In a couple of companies where I've worked, some participants would even resort until waiting until other participants who differed left the room so they could get the prioritization outcome they wanted.

    So I contend that (1) the apparent need for a spreadsheet is often an indication that these problems exist, and (2) the problems run deeper than The Spreadsheet can fix.

    Second, putting "alignment with product strategy" on The Spreadsheet doesn't mean the team actually shares a prior understanding of what it is. On the contrary - if the team members did share such a prior shared understanding, the need for The Spreadsheet would largely disappear. As I wrote in my blog entry, decisions would naturally flow from the product strategy (especially the unique value proposition.

    Indeed, you wrote "an organization must have an agreed upon strategy (something sorely missing all too often)."

    Third, instead of spending time on The Spreadsheet, how about spending time developing an informed product strategy and ensuring that every member of the team understands and buys into it? Then, via the "empowering judgment with context" that Steve Johnson mentioned, we reduce the number of prioritization decisions that are controversial, and we resolve the rest of them through conversation, not spreadsheets.

    Though yours seems to differ, in my experience, a team that has consensus on the product strategy has robust but relatively quick conversations to get on the same page about any particular controversial prioritizing decision.

    Insofar as the root cause of prioritizing controversy is organizational dysfunction, the solution is to restructure the organization.

    ReplyDelete
  3. Hey Roger,
    Seems to me to come down to a difference in experiences (which makes me question more how the tool is used than whether it has inherent value). We are in total agreement on the need for an explicit strategy that is accepted by the team, a requirement independent of any prioritization approach. What I'm missing in your post is an effective technique for driving prioritization and making the criteria transparent (criteria {plural} exist whether you're conscious of them or not). The idea that consensus on priorities will naturally flow from shared understanding at the strategic level isn't consistent with my experience at all. My background is in huge shops with complex framework products (and plenty of people from different roles with strong opinions on strategy). I guess the value depends somewhat on the dynamics of strategy definition/ownership and team cohesiveness. Anyway, great conversation!

    ReplyDelete
  4. This comment has been removed by a blog administrator.

    ReplyDelete
  5. This comment has been removed by a blog administrator.

    ReplyDelete
  6. Greg, you mentioned previously that strategy is "missing all too often" in organizations. If that's been your experience, how can you be confident that product decisions don't flow naturally in organizations that do have strategy consensus (particularly around the unique value propositions of their products)?

    Indeed, you wrote:

    "My background is in huge shops with complex framework products (and plenty of people from different roles with strong opinions on strategy)."

    Sounds like there wasn't consensus on strategy, which supports one of my points.

    Moreover, regarding complexity, a defining characteristic of a good unique value proposition is that it is NOT complex:

    "No matter how complicated the product, no matter how complicated the needs of the market, it's always better to focus on one word or benefit than two or three or four." - Ries and Trout

    Battle it out over the unique value proposition, not over individual product ideas.

    I think I outlined pretty clearly in my blog entry the three transparent criteria for determining product priorities. When the team looks at a portrait of the key persona (call her "Lisa") on the wall and agrees, "This roadmap item is the most important to deliver our promised value (as reflected in the unique value proposition) to Lisa", it's transparent.

    I don't believe that numbers and formulas are the only form of, or path to, transparency.

    ReplyDelete
  7. Seems a bit oversimplified to me Roger. What if you don't just have to please Lisa, but a "visionary" CEO, the economic buyer at the customer, your development team and several different user types. No amount of strategic alignment will necessarily drive consensus around what the best investments should be for a given release. In my case, our UVP had to do with our existing relationships with customers and our credibility in the market. What makes more sense? Alignment with the other products in the portfolio or adopting the new UI technology identified as strategic at the corporate level. TONS of room for disagreement about which best supports our UVP. In these cases, the matrix can help drive structured, rational conversations that provide important input into final prioritization.

    For new products, you may have to identify "the best" 10 investments from 50 candidates, all of which seem perfectly reasonable on the surface. You need an efficient way of making sure that various stakeholders feel they've been heard and that their ideas, very often well-informed ones, have gotten a fair day in court. It's also nice to have a mechanism to keep yourself honest: we are humans and have our own biases that tools like the matrix can help expose. I teach the use of this tool and related best practices in my product management courses and it's very, very common for groups doing the related exercise to get a least one surprise in prioritization (an investment's scoring much higher or lower than expected).

    I would wrap up by saying any process or tool can suck. Nothing (in my mind) trumps experience and mine have been positive. In a future post, I'll write up a use case and describe the downside of not using a matrix (all approaches have their weaknesses). This topic obviously hit a nerve! Great dialog.

    ReplyDelete
  8. As I see it, the unique value proposition is really the crux of the issue here, Greg.

    If you don't believe in relentlessly simplifying the unique value proposition and focusing on as few personas as possible, then it will indeed make it difficult for the team to get on the same page and make common sense product decisions on a day-to-day basis.

    You alluded to the complexity of "pleasing" a diversity of stakeholders. Ries and Trout were well aware of this difficulty when they wrote Positioning: The Battle for Your Mind in 1981. Their advice: simplify, simplify, simplify.

    Ries and Trout also recognized that focus and simplicity have a halo effect. In your positioning, you can focus on one idea and one or two stakeholders and gain broad appeal across a much wider audience.

    Product managers should be dedicating much of their efforts to informing and building consensus for such a focused value proposition. If the diverse set of stakeholders on your team are struggling to agree on a unique value proposition, it is this problem the product manager should address.

    In the event (which you characterized as rare) that the team does have a product strategy consensus, and the unique value proposition component of that product strategy is focused and simple, then prioritizing individual items sure does become easier.

    But it sounds like you don't accept my central premise that Ries and Trout were (and still are) right about relentless focus and simplification of the value proposition.

    ReplyDelete
  9. I'm a big fan of simplification Roger. What I don't (seem to) agree with is that given consensus around a strategy or UVP, all stakeholders will automatically agree on priorities. In cases where there are differences of opinion (or likelihood thereof), I've found the matrix to be valuable. You must have had other experiences. I think that's normal. I like the title of your post for the purposes of provoking dialog, but (obviously) think it's overstated. Looking forward to more discussions like this in the PM community. I'm loving it!

    ReplyDelete