Your approach to validation is broken

There's been tons of buzz about "validation" in the past several years. Lean and design-centric approaches such as Design Thinking encourage entrepreneurs to build things and incrementally validate them, correcting course as necessary. When I think about how, at the beginning of my career, we worked for months before validating anything, this increased focus on getting feedback from those who will use our product is most welcome. However, this obsession to validate what we build has, in practice, at least a couple of massive flaws that I'd like to highlight.

Firstly, I see almost exclusive focus on end users. While the people who directly use our products are important, the fact is that they are just one type of stakeholder. This point is a little less obvious in the Internet-based consumer space, where the user and economic buyer are the same and the buying process is much, much simpler than in the enterprise space. An individual consumer paying a couple of bucks for an app may (often?) make a decision based on little more than impulse. Those toiling away in the complex problem space where millions may be invested in a solution and with a fiduciary responsibility to stockholders tend to have buying processes that are dizzyingly complicated. We, as product managers must, therefore, validate ideas not just with end users, but with most or all stakeholders. Have we expressed our value proposition in a way that resonates with executive leadership? Do we have the right business model? Does our pricing model provide enterprise purchasing departments with the required financial flexibility? Overlooking these types of questions for these stakeholders may result in massive failure of a product that delights its end users.

Secondly, I find that most of what's written about validation focuses on the solution space. The idea is to build something quickly, test it, and "pivot" while it's still relatively cheap to do so. The problem is that we rush through our analysis of the problem space without validating that we clearly understand our stakeholders' key pains. Just like in solution validation, we rarely get it right on the first pass. Often, our understanding comes from anecdotal evidence ("I was talking to a customer the other day…") and flawed intuition. The first thing we should validate with stakeholders (not just users), is that we truly understand their problems. And here I'm not just talking about the problems we intend to solve. Understanding stakeholders' pain broadly helps us uncover acute pain we may have overlooked and helps put the problems we intend to solve in proper context. Laser-focused on our objectives, we begin to assume we are solving stakeholders' most critical problems although in many cases our solution, although important, is way, way down on our stakeholders' list of most pernicious issues. Before we even conceive of a solution (much less validate it), it is wise to validate our understanding of the problems that are holding our stakeholders back.

Problem validation entails the following steps:

1. Open, problem-oriented discussions with all important stakeholders
2. Distilling problems into a concise, consumable form
3. Validating our understanding of problems directly with stakeholders
4. Adapting our expression and understanding of problems as necessary

Just as fixing a prototype is far less expensive than fixing a shipping product, correcting misinterpretations of problems is much cheaper that reformulating a misguided solution, even as a prototype, later. I've blogged previously about the "Problem Profile", a valuable tool for considering problems holistically and validating them with stakeholders.

Your challenge is to resist the urge to validate too late, once you've already invested in the solution space. You will find that problem-centric analysis will yield insights that will never come from a solution-oriented approach.

What is your experience? Do you have practices in place to ensure you have a grasp of stakeholders' problems before you begin trying to conceive of and develop solutions?

You can read more about my strategic approach to product management on my site.

An interview on orchestration with yours truly...

Product Manager: Are you being DDoS'd?

There are lots of reasons that product managers aren't as effective as they could be. However, in my experience, the number one culprit by a huge margin is "operational overload". I usually describe it (somewhat euphemistically) as "lack of strategic execution" to my consulting clients. The syndrome is all too familiar to most of us: we're being pulled in so many directions by so many stakeholders that we become highly reactive and, truth be told, ineffectual. Eventually, we're left with the feeling of running on the proverbial hamster wheel. At the end of a long day or week, we're exhausted but not at all convinced that we've made the difference we hope for. When you're in the middle of it, this situation can feel like a distributed denial of service (DDoS) attack: so many requests for your "resources" that you can't serve anyone!

Symptoms

• > 50% of your calendar is overbooked
• > 60% of your time spent in meetings
• > 60% of overall meeting content seems irrelevant to you
• A high level of stress (this can be dangerous as some folks thrive on this feeling)
• A sinking feeling that bad things are happening but you don't even have time to analyze what's happening
• You are convinced you're letting others down professionally and feel helpless to change it

It's very easy to feel helpless in this situation and lose hope. You might be surprised however that consistently applying some simple approaches and techniques can make your more effective, lower your stress level and make you a much happier professional.

IT'S YOUR ACCOUNTABILITY TO GET ON TOP OF THE SITUATION! Here are a few tips:

Understand your motivation and how success will be measured!

Write down your goals and objectives and invest in activities that will help you achieve them. If you're not prioritizing your time, you're setting yourself up for failure.

Actively manage expectations

As a PM, you have many stakeholders. Make sure you set clear expectations with those who are most important and adjust them as necessary. People can be very forgiving of missed deadlines if they know they're coming. Always apologizing after the fact will quickly erode your credibility.

Analyze your agenda for strategic activity

Go through the appointments in your calendar for a week and mark those that represent strategic work. You might be surprised how much of your time is occupied by operational issues.

Let go of lower priority stuff

One of the traits I've seen in professionals I've admired over the years is a willingness to simply abandon activities that won't help them achieve their goals. See if you can delegate some of these activities to others for whom they may be a growth opportunity. Carefully assess the consequences, but you might be surprised how little practical impact there is from simply not doing some of the low-priority tasks that are on your plate.

In summary, we as product managers seem especially vulnerable to the DDoS. In my experience, there's no one else who will step up to save you. If you don't get on top of your time, a career as a PM can be a very long, unrewarding road.

Why do we keep confusing customers and markets?

As a product management consultant and educator, I continue to be surprised by the conflation of the terms "customer" and "market". As anyone who has taken any of my courses knows, these terms are not synonymous! Let's start with some simple definitions:

market: the entities with the interest and means to buy your product

The term "market" is an aggregate concept. It is all the entities, consumers, for example, who might be interested in your product (because they assume it will delight them or solve their problems) who also have the means (financial means, usually) to purchase your product. Understanding markets must be done by finding proxies; the market doesn't have the means to express its pains to you. From a product management perspective, you must infer requirements from the market.

customer: An entity that has bought your product

Customers represent the segment of the market that has purchased your product. It's worth noting that the term "customer" is an aggregate concept referring to a multitude of customer stakeholders, e.g., economic buyer, influencers, end users. You can actually talk to customers and they, unlike markets, are capable of expressing their requirements and other desires. Communicating with customers, while itself challenging at times, is a much more straight-forward affair than divining the desires of a market.

So where do I see folks getting confused? Well-known examples include the traditional Venn diagram showing that PM lies at the intersection of customers, technology, and business. I would say that substituting "markets" for "customers" in this diagram makes much more sense. The Business Model Canvas also seems to confuse them. It defines a building block called "Customer Segments", for example. While it may be beneficial to segment your customers, I would say segmenting markets makes a lot more sense when you're fleshing out a business model.

The third case of confusion is a bit more subtle. I've worked with companies with significant market share, i.e., their customers comprise a large percentage of their market. In these cases, it is common for product managers to engage deeply with their customers at the expense of gaining an understanding of the broader market. This tendency is understandable but can hinder growth beyond your customer base, generating a type of chasm for even mature products.

I would say inferring the desires of a market is one of a PMs most challenging tasks. Although you can always try to validate your hypothesis, markets are, by their nature, unpredictable and often full of surprises. My challenge to the PM community is to think deeply about the distinctions between these two concepts and, as importantly, to stop conflating or confusing them.

What do you think? Are my definitions inline with yours? Have you seen people confuse these terms?

UPDATE: Began reading Blank's "The Four Steps to the Epiphany" and it becomes a bit clearer how these concepts get confused: "Broadly speaking, Customer Discovery focuses on testing whether a company’s business model is correct, specifically focused on whether the product solves customer problems and needs (this match of product features and customers is called Product/Market fit.)". I think the point is valid, but talking about customers and calling it "market fit" may not be the clearest way to express the idea.

Roadmapping: Your customers don't care about your team's development schedule

I continue to be surprised and somewhat disappointed in most of the roadmapping tools I see aimed at product managers. Working primarily in the enterprise space during my career, I created "milestone" style roadmaps that showed when the team would deliver new capabilities to the market. What I see in the tool market today are inward-facing Gantt style roadmaps which, I assume, help the team manage their development work. Although this style is no doubt valuable to the team, in my experience, customers, in general, didn't much care when or how I worked on features: they wanted to know when they would get them. I would also say that many, if not most, internal stakeholders weren't that interested in our development plan.

I'm sure some will say that changes in delivery models, e.g., SaaS, mean that teams continuously deliver to the market and milestones are now irrelevant. I maintain that customers still don't care about your development schedule and that when software is mission critical, continuous delivery isn't the boon it is in other contexts. There are certainly exceptions to this rule (in the security domain, for example).

What I believe is needed are tools that allow product managers to generate different product roadmaps based on the same set of "investments". Such a tool would allow certain things to be excluded or renamed and, above all, would show traditional delivery milestones.

What is your experience? Have I simply overlooked good roadmapping tools that already do what I describe?

Managing Complex Solutions: When Product Management Begins Breaking Down

I've spent most of my career working as a product manager in huge software companies (IBM, Microsoft, SAP) and as a product management consultant and trainer with vendors that deliver complex solutions in the B2B space. Perhaps it's just me, but when I look at most of the content being generated in the field of product management, I can't help but thinking that something is missing. To my eyes, most of it seems startup-centric (in the consumer space mostly), so, to someone who's been slugging it out in trenches of highly complex problem spaces, much of the talk of MVPs, A/B testing, continuous delivery and other hot topics seems a bit, dare I say, naïve.

 What I consider a lack of emphasis on what I've labeled the "enterprise space" has inspired me to begin thinking deeply about the unique challenges of effectively managing products that must address some of the world's stickiest business and technical problems. Often, these complex problems require complex organizations to deliver the associated solutions, which in itself brings a boatload of obstacles that a PM in a startup can only imagine (in nightmares for the most part).

 Here are just a few of the differences between the enterprise and consumer-oriented startups that are top of mind:

 The Myth of the Customer

In the enterprise space, the people who evaluate, buy and use the software are often different. Buying decisions may take years and the solution may be required to function (almost perfectly) for years or even decades. When we use the term "customer", it is a conversational convenience. The truth is, the enterprise customer comprises a complicated set of stakeholders we have to identify, understand and appease.

 The Dysfunctional Family

Big, stable organizations with highly specialized roles tend to create "families" that are (often highly) dysfunctional. Familiarity, procedural inertia and contention for resources make vendors that deliver complex products and solutions highly political, to the point that a majority of a PM's time may be spent on "political overhead", generating negative value for customers. Navigating the fraught waters of a politically charged and entrenched organizational hierarchy requires a level of diplomatic adeptness that is rarely optional.

 The Curse of Mission Criticality

A lot of the more nimble practices espoused by the startup community are quickly exposed as not viable when the products or solutions being delivered can cause losses of millions of dollars or much worse, lives, when they fail. Add the fact that many of these "high assurance" scenarios are strictly regulated and you have the proverbial straw that can easily break the product manager's back.

 In the coming months, I'll talk extensively about the enterprise space and offer some food for thought on how to tackle some of its most daunting challenges. Take heart enterprise product manager, together we can be inspired by what's happening in the startup world but move beyond it to make our community happier and more effective.

 You can read more about related topics on my site and blog.

The fine line between an interview question and free consulting

Absolutely LOVED this article by Liz Ryan in Forbes. I too have been put in the ridiculous situation of being asked to do specific strategy work in the context of a job interview. Luckily, I've never been desperate for a job when it happened. Probing questions that help a hiring manager determine if I have a solid grasp of some part of the job she expects me to perform make sense. Asking me to invest my time to formulate a strategy for your current situation in anything but the most general terms is not OK. Lest I belabor what I consider fairly obvious point:

1. If you're hiring: Using a job interview as an excuse to get free consulting from a candidate is simply not ethical behavior. There are plenty of ways to validate that someone knows their stuff without asking them for a free lunch.
2. If you're being interviewed: I realize how much stress a job search creates for a candidate, especially when you really need the work. However, resist the urge to give away your hard-won knowledge and creativity to someone who has essentially no skin in the game. An ethical organization and hiring manager will have better sense than to expect this.

I think there are more tactful ways to handle the situation described in the article (I assume the "victim" was convinced this guy was ONLY trying to pry knowledge out of her). For example, I was asked to develop a strategy for an online travel company thinking about expanding into business travel. I told the hiring manager that I was more than happy to discuss key factors that might increase their chances of success but that I simply wasn't willing to develop a strategy for him. I didn't get a call back and felt great about it. I didn't think I'd been rude in the least and managed to send him a clear signal that what he was asking for was unreasonable.

BTW, my opinion changes a bit for second-round interviews. If you're convinced you'be been selected from a broader field, you might consider being a bit more forthcoming. I would still stop short of doing free, strategic work for an organization who has invested little more in you than an interview slot or two.

I realize this is a touchy topic. What are your experiences?