Criteria for Assessing a Software Product Management Organization

In a previous post, I talked about assessing products as they are being developed. In this post, I'd like to talk about a different kind of assessment: assessing (or "auditing" as some others call it) a software product management organization. Formal assessments are often requested by leaders of an organization who are concerned about the performance of the product management function. What I've realized since I began thinking deeply about the topic is that every time I've joined a new company, I've done a mental assessment to size the place up. When I began developing my assessment framework, I found myself trying to articulate or externalize this tacit mental model.

Before sharing my approach, I thought it made sense to share what I consider the primary requirements or criteria I would use to determine if the framework was ready for primetime. Whether you eventually agree with my approach or not, I hope these criteria can help you assess the multiple assessment approaches you are likely to encounter in the market.

An assessment framework or approach should be:

Simple

I believe the approach or method used to assess the product management organization should be reasonably simple: not to many heady concepts or moving parts. This requirement helps ensure those commissioning the assessment can easily understand the approach and its outputs. It also allows one to deliver an engagement in a fairly short period of time. My goal was to be able to deliver an assessment within a couple of weeks or less.

Comprehensive

I've read about assessment approaches that focus on the knowledge and skills of product managers. Although I too think professional competency is critical, it is just one the elements that impacts organizational effectiveness. By the way, I believe that the goal of the assessment should be to measure the organization, not simply the people in it. For example, I've spent a lot of my career creating platforms that model and execute business processes, so understanding the maturity of processes is important to me (and most organizations, I believe).

Quantifiable

I decided early on that beyond the qualitative aspects I can assess based on experience and intuition, measuring subsequent improvement would require that key elements of the framework be reducible to values that could be measured. That doesn't necessarily mean that the values are valid from a purely scientific perspective, but that changes in performance, both positive and negative, can be tracked over time in a convincing way.

Flexible

Experience with quite a few product management organizations has taught me that each organization values different aspects of the discipline. An assessment framework should be flexible enough that it can be adapted to incorporate an organization's priorities and culture.

Transferable

Although I believe it's difficult to generate assessment results that support an "apples to apples" comparison of any two product management organizations, the framework should capture common aspects of the product management discipline in such a way that reasonable comparisons can be made. This criterion is particularly important when assessment multiple organizations in the same company, e.g., different business units.

In a future post, I'll enumerate the key elements or "dimensions" I've chosen for my framework based on these criteria. I've shared my approach with several people whose opinion I value and am so far encouraged.

What is your experience with assessing product management organizations (particularly in the software domain)? What requirements or criteria are important to you?

Previous post: Overlooked Sources of Product Management Power