Is Customer Development Pseudoscience?

The “Science” of Lean Startup

Lean startup practitioners embrace the scientific method, seeking the "truth" about what business model and strategy will lead to product success. We do so by:

• Formulating hypotheses
• Crafting and running experiments to test them
• Learning from the experiments
• Iteratively feeding our learnings back into revised hypotheses

Sounds pretty scientific, at least in spirit, doesn't it? Yet this process actually neglects a key ingredient in the scientists' mode of operation. To identify what’s missing, let’s examine “customer development”.

Customer Development

Steve Blank is one of the pioneers of the lean startup movement. He introduced into the lean startup lexicon the term “customer development”. Customer development consists of sessions and interactions with customers to test hypotheses.

For example, a product manager might interview a prospect, asking if she agrees with the product manager’s hypotheses about the problems she faces or the value the product might bring to the prospect. Or she might “pitch” her product ideas to a prospect, as a sales person might do, ask for payment or a commitment of some sort, and learn from the prospect’s reactions.

Customer development practitioners, according to Blank, start with a relatively specific set of preconceived hypotheses and focus customer sessions almost exclusively on assessing the “validity” and applicability of these hypotheses to customers. As practitioners test the hypotheses, they capture data and may stumble across some insights. But customer development sessions do not include open exploration into customers’ challenges, according to Blank.

In fact, Blank pigeonholes open exploration as part of “design thinking”. He suggests design thinking is, at least in common practice, a lengthy, heavyweight approach to learning, and therefore not appropriate for lean startups. (A comment by my former colleague, Jen van der Meer, aptly explains that this view of design thinking is a caricature.)

Even lean startups should include open exploration in their customer interview and observation sessions, lest they neglect a vast body of important potential insights.

Rummy’s Unknown Unknowns

Former Secretary of State Donald Rumsfeld is famous for, in a press conference, having distinguished among “known knowns”, “known unknowns”, and “unknown unknowns”:

"As we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don't know." - Donald Rumsfeld, February 12, 2002

Known known: Something we know we know.

Known unknown: Something we know we don’t know.

Unknown unknown: Something we don’t know we don’t know.

Learning is the process of converting unknowns (whether known or unknown) to known knowns.

Blank’s Misguided Focus

"Customer Development starts with, 'I have a technology/product, now who do I sell it to?'” - Steve Blank, "Driving Corporate Innovation" blog entry

By focusing customer development on testing preconceived hypotheses, Blank has us concentrate on the known unknowns, or the things we know we don’t know (with sufficient confidence). By definition, a preconceived idea or hypothesis is not, and cannot be, an unknown unknown.

If the body of unknown unknowns were very small relative to that of known unknowns, then we could employ Blank’s approach, and we might eventually converge on a set of hypotheses constituting a successful product strategy.

In reality, though, the typical body of knowledge (and lack thereof) might break down as follows:

In any given domain ripe for innovation, and particularly early in a lean startup process, the body of unknown unknowns may eclipse the body of known unknowns. Moreover, you may think you know the extent of the unknown unknowns, but how would you know you know? Consequently, even if we have a prior set of hypotheses - and even a product based on those hypotheses - we should focus some of our efforts on exploring the vast body of unknown unknowns that exist in a domain or industry.

Focus on the Unknown Unknowns

"Customer interviews are about exploring what you don't know you don't know." - Ash Maurya, Running Lean, page 71

We should focus some of our sessions with prospective or existing customers on exploring the situations and challenges they face, not exclusively on directly testing our preconceived notions of what they may be.

You likely do have a set of preconceived hypotheses, but these hypotheses exist within the context of a domain and set of possible stakeholders. This context helps you narrow the scope of your inquiry: the people you should observe and interview, and which aspects of their lives you should explore.

Much as a scientist conducts field studies, a lean startup practitioner should spend at least some time in the field to explore. Much as a therapist uncovers patients' problems by exploring their past and present lives, a lean startup practitioner should uncover unknown unknowns by exploring prospects' past and present ways of getting jobs done.

During interviews, ask prospects what they’ve done and what they currently do. Avoid leading questions, hypothetical questions, asking them what they “want”, having customers design your product, and ignoring change management issues.

By employing an exploratory approach grounded in a prospect’s reality, we’ll not only uncover insights we never anticipated, but we’ll also indirectly test the known unknowns. A prospect’s unprompted description of a painful experience is far more significant than a response to your eager “do you experience this pain” question.

Conclusion

There is no need to debate the semantics of “customer development” and “design thinking”. Steve Blank popularized the term “customer development”, and we can defer to his explanation of its definition and its focus. Regardless of the definitions, however, smart lean startup practitioners should include some open exploration in their approaches to interviewing and observing customers. Failure to do so risks missing important opportunities to create innovative and transformative product experiences.

Why Spreadsheets Suck for Prioritizing

The Goal

As a company executive, you want confidence that your product team (which includes all the people, from all departments, responsible for product success) has a sound basis for deciding which items are on the product roadmap. You also want confidence the team is prioritizing the items in a smart way.

What Should We Prioritize?

The items the team prioritizes could be features, user stories, epics, market problems, themes, or experiments. Melissa Perri makes an excellent case for a "problem roadmap", and, in general, I recommend focusing on the latter types of items. However, the topic of what types of items you should prioritize - and in what situations - is interesting and important but beyond the scope of this blog entry.

A Sad but Familiar Story

If there is significant controversy about priorities, then almost inevitably, a product manager or other member of the team decides to put together The Spreadsheet.

I've done it. Some of the most respected names in product management have done it:

• Quick Prioritization, by Steve Johnson
• How to Use a Scorecard to Prioritize Features, by Daniel Elizalde
• How to Build Roadmaps that Stick, by Bruce McCarthy

The Spreadsheet is a list of all the ideas for the product roadmap or "backlog", along with columns for every factor imaginable that might affect the priority of the items.

Columns in The Spreadsheet might include:

• Man-hours to implement
• Tool and other expenses required to implement
• Potential revenue
• Number of customer requests
• Number of lost deals due to absence of feature
• Alignment with quarterly or annual corporate strategic priorities

Someone creates a "magic formula" in the final column of the sheet that combines and weighs the various factors in the other columns and computes a "score". Then the product manager or team and you all get in a room and fill in the sheet. The group proceeds to debate the finer points of, for example, assigning a "5" or a "6" rating to the "potential revenue" column for a particular feature idea.

Halfway through the multi-hour exercise, you and the team are fatigued and decide to pick it up another day. Whether or not this follow-up meeting ever happens, the team ignores much of the priorities that result from the effort, and few if any updates ever occur to The Spreadsheet.

Why "The Spreadsheet" Sucks

The Spreadsheet approach to prioritization has at least three fatal flaws:

1. Organizational dysfunction. It attempts to address organizational dysfunction with formulas and analysis that ignore human factors.
2. Product strategy void. It indicates team members lack a shared understanding of the product strategy.
3. Distraction from unique value proposition. It distracts from your product's value proposition or from the target audience that derives the value from your product.

Let's examine each of these flaws in more detail.

Organizational Dysfunction

Step back and consider why one or more people on the team thought The Spreadsheet was a good idea. In all likelihood, it was due to difficulty determining or agreeing on priorities.

At some point, you may have asked your product manager, "Why aren't we working on feature Y instead of feature X? What data do you have to back up your decision?" Or members of the sales team bypass the product manager, talk directly to developers, and tell them how the next big deal will happen if only they would implement a killer feature. Or multiple product managers are competing for development resources and employ passive aggressive tactics to divert those resources to work on their products.

By now, you recognize the recurring theme: dysfunctional interactions among team members. The hope is that The Spreadsheet causes everyone to set aside their distrust and passive aggression and become "objective". If we just focus on the numbers and let the "magic formula" compute the score, all these dysfunctional interactions will go away, and we'll get on the same page.

Once you recognize the context (dysfunctional team interactions) and the proposed solution (objective analysis of priorities), it becomes obvious to anyone with a modicum of human sensibility that the proposed solution misses the mark.

Product Strategy Void

Why does this distrust and passive aggression exist in the first place? Or if no such dysfunctional behavior is occurring, why is it nevertheless so hard to get on the same page? It could be due to personalities and culture, or it could be that the team simply doesn't have a shared understanding of the product strategy.

One model of product management is that the product manager or product owner (The All-Knowing One or The Decider, as Teresa Torres might say in jest) knows best, prioritizes a backlog, and doles out the work to designers and developers. "I've prioritized it for you; just design and implement it."

This approach works fine when the product manager or owner is, in fact, all knowing, has the time to meticulously specify and prioritize every user story in the backlog, and when the designers and developers are machines that do exactly what they're told. Good luck with that.

What happens in the real world? The All-Knowing One is swamped, she has no time to talk to prospects and customers, and designers and developers keep pestering her with questions. When the product manager is wrong or has no sound basis for her answers, she loses credibility.

May I suggest another approach?

Teams are most productive when they share a strategic vision and each team member can "fill in the blanks". What if the product manager facilitated a shared understanding of the product strategy? Then each member of the team would feel confident making decisions without going back to The All-Knowing One. And the prioritizing that team members do on a daily, or even hourly, basis would flow naturally out of that strategy.

In the absence of a shared understanding of the product strategy, you, your product manager, and team members aren't likely to agree on priorities. The Spreadsheet isn't a strategy and is no substitute for a strategy. It won't fill this fundamental void.

Distraction from the Unique Value Proposition

Let's say you do have a product strategy and a shared team understanding of what it is. A fundamental part of your product strategy is the unique value proposition.

The unique value proposition is the value you envision your product uniquely delivering to a target market.

When the team chooses the product's unique value proposition, it selects a singular, overarching theme that implicitly embodies the set of market problems it plans to solve with the product. It has largely selected the high-level priorities and deliberately set aside problems and features that don't support the overarching theme.

For more information on unique value propositions and how to position your product relative to the competition, see this entry on competitive mindshare maps.

Imagine sales and customers bombard the team with requests for a "killer feature" that has little or nothing to do with the unique value proposition. How does the feature fare in The Spreadsheet?

Well, it scores well in the "number of customer requests" category. Perhaps it has great "revenue potential" as well, so bump up the score in that column, too. Sales claims it loses a lot of deals due to the lack of the feature. It looks like this feature might score very well indeed on The Spreadsheet.

But evaluating the feature through the lens of the unique value proposition might lead to a very different assessment of its priority. In fact, the feature might undermine, or be unrelated to, the unique value proposition.

Now, given the feedback you've gotten from customers and sales, you might want to consider whether to "pivot" and change the unique value proposition, and thereby the product strategy, for the product. Alternatively, you might consider spinning off another product or company with a fresh and separate set of product strategy hypotheses based on the new information.

But if you believe you already have a product strategy worth pursuing and testing, The Spreadsheet has led you astray. If you prioritize features without regard for the unique value proposition, you end up with a fractured product that distracts from - rather than delivers - its promised value. It may deliver bits and pieces of value, but those pieces have little relation to each other. They don't coalesce around a central theme.

While the "potential revenue" rating might nonetheless seem to justify the high priority, it neglects the power of brand: the long-term impact on revenue of a cohesive product that delivers a clear value proposition.

How to Prioritize

At this point, you're wondering how your team should prioritize, if not using The Spreadsheet.

I propose three primary questions to guide prioritization:

1. To what extent does this item (feature, user story, epic, market problem, theme, or experiment) support the unique value proposition?
2. What is the level of effort required to implement this item?
3. What will implementing this item enable us to learn?

(Heck, I might throw in a fourth one: how fun will it be to work on this item? It matters.)

The answers to these questions aren’t numbers team members plug into a formula. They form the basis for thought processes and conversations. Consider them holistically and always in the context of the unique value proposition. As Teresa Torres cautions, "If you use time-to-build to prioritize what to build next, you’ll end up with a product full of easy solutions." Teresa rightly urges us to emphasize impact on a goal. I contend the goal is delivering the unique value proposition, as I wrote in 2013.

Accordingly, your product manager should lead the process of determining the product strategy (including the unique value proposition) and foster a shared understanding of it, and the reasons for it, among the entire team (across all departments). The product manager may also lead the process of conceiving experiments to continually test the riskier assumptions behind the product strategy hypotheses.

This high-level strategic baseline sets the stage for every member of the team to make common sense day-to-day prioritization decisions without resorting to numbers and formulas and without going back to The All-Knowing One for guidance. And when the product manager or team prioritizes one item over another, it will flow naturally out of the shared understanding of the product strategy. Steve Johnson calls it "empowering judgment with context".

Healthy conversations about how my three considerations apply to an item should and will occur. But rarely will you need to ask for the "data" that justifies a prioritization decision. The Spreadsheet approach won't appeal to anyone, except perhaps to the most hopelessly analytical members of the team. God bless 'em.

What Is Design Thinking?

The Context

Over the years, various product management and development methods have come into vogue, most notably agile and lean startup methods. Agile methods addressed requirements, architecture, and development risks using frequent iterations. Lean startup methods take it a step further, iterating on the entire product strategy, including the unique value proposition, the target market, revenue and cost structure, and marketing and sales channels.

Recently, you may have heard more and more chatter about "design thinking". I certainly have. What is it, and how does it differ from agile and lean startup methods? How can companies leverage it to innovate and develop better products?

The Definition

If you do some web searches, you'll notice a lot of vague references to design thinking and a diversity of views about what it is. After exploring some definitions, I'll borrow from various descriptions and accounts of design thinking to form what I believe is a coherent explanation of what it is.

Let's see how Tim Brown, CEO of IDEO, defined "design thinking" in his seminal article in Harvard Business Review:

"[Design thinking is] a methodology that imbues the full spectrum of innovation activities with a human-centered design ethos."

Clear as mud, eh?

My former colleague, Davide Casali, defines it as follows:

This definition is concise and nicely aligns with the plain meanings of the words. It leads us to the key question: what are the characterizing activities of design thinkers? Indeed, Wikipedia states:

Hmm. I'm not sure about the "design-specific" part of this definition, but let's look at the activities that characterize design thinking.

The Activities

Note: these activities characterize design thinking but do not necessarily come in any particular sequence or imply a particular "process".

• Explore the problem domain. Interview prospective customers and users. Observe and experience the problem and its context first hand.
• Define the problem you are trying to solve. Half of the battle in designing a great solution is to clearly define the problem you're solving.
• Ideate to generate perspectives on the problem and ideas for solving it. Set aside analytical thinking and engage your right brain.
• Hypothesize about the problem, solution, and business model. What would have to be true for the problem to resonate with prospects and for the solution to solve the problem? How would you know that your hypotheses and assumptions are false?
• Experiment to test your hypotheses and assumptions. Employ prototypes and confront prospects with real and concrete choices.
• Learn from all your other activities, especially from exploration and experiments. Design thinking isn't (merely) about "creativity" and having a sense for the "right" solution. It's about learning and discovery.

Key Concepts

Design thinkers leverage key concepts that differentiate the solutions that emerge from their activities.

• Holism. In design thinking, the product is a set of experiences, not of features, that provide value. The nature and value of the experiences depends on a larger ecosystem, such as the one Apple created with iTunes.
• Iteration. You almost never get it right the first time, and you shouldn't spend a lot of time trying. While design thinkers don't necessarily employ a particular process, they iterate in their approaches to understanding the problem, the customer, and the solution.

Final Thoughts

Some product managers, agile teams, and lean startup practitioners engage in some of the same activities, and apply some of the same concepts, as design thinkers. In particular, iteration has long been a core part of agile processes. Lean startup practices are iterative and also incorporate hypothesizing, experimenting, and learning in a build-measure-learn loop. The best product managers learn about markets by interviewing and observing prospective customers and users.

But agile and lean startup methods differ from design thinking in several ways:

1. Though certain activities and concepts characterize design thinking, it's not a "process".
2. While agile methods are iterative and include some hypothesizing, experimenting, and learning, they don't necessarily include other design thinking activities or a holistic view of the product.
3. Open-ended exploration is not part of lean startup methods, though they arguably incorporate all other design thinking activities. (I'll address this controversial topic in a future blog entry. UPDATE: Read how the "customer development" concept in lean startup methods, at least according to Steve Blank, does NOT include open-ended exploration.)
4. You can employ lean startup methods without a holistic view of what constitutes a "product".

Many product teams suffer from tunnel vision, which stifles innovation. The tunnel vision stems from paying insufficient attention to raw exploration and obsessing over "validating" their preconceived ideas. And most of them treat products as collections of features instead of holistic sets of experiences. Product managers can up their game by leveraging all aspects of design thinking.

UPDATE: I covered design thinking at ProductCamp Austin 14. The title of my session was "WTF Is Design Thinking". You can get a link to the slides by tweeting this blog entry. Send this tweet:

What Is #designthinking? http://cauv.in/wtfdt

Thanks to Davide Casali and Shardul Mehta for reviewing and providing feedback on initial drafts of this blog entry.

What Product Managers Can Learn from the Apple iPod

The Story

When Apple unveiled its iPod digital music player back in October 2001, I dismissed it as a parity product. I already owned the Cowon iAUDIO CW100 MP3 player, loaded with my favorite tunes. There was Apple, generating great hype over the iPod as if it were a breakthrough product.

The idea of a portable digital music player was nothing new. The first mass-produced MP3 players came out in 1998. In late 2001, the concept may have been new to a lot of Apple customers, but it wasn't new to me. I proudly showed my MP3 player to friends when they gushed about the iPod.

Thus Apple's iPod was not an innovative product in and of itself. Years later, however, I realized the significance of ecosystem of which the iPod was a part. Apple had released iTunes (with technology purchased from SoundJam MP) and created the iTunes Store for finding and downloading music. Unlike Napster, it was a safe and legal way of distributing and acquiring music.

The prior way of playing portable digital music required a user first:

1. "Rip" a song from CD or download the MP3 file (illegally, in many cases) from Napster.
2. Connect a USB or other cable and transfer the file to a portable MP3 player (after installing the appropriate drivers).

No big deal for me. But the vast majority of users didn't know how, or want, to mess around with ripping CDs, installing drivers, or downloading and transferring files. They wanted to acquire and listen to songs, not files. Before Apple's iTunes and iPod, it was about "ripping", drivers, files, and transfers.

Apple had created an ecosystem. Its product managers and designers considered the entire set of user experiences, not just users' direct experiences with digital music players. If you viewed the iPod as a product in isolation, it wasn't innovative. The innovation was in creating a set of elegant (and legal) user experiences around acquiring and playing digital music.

The Lesson

This story makes you wonder: what is a product? Is a product a collection of features? Or is a product, in a sense, a set of user experiences? Product managers should focus on enabling users to have a set of experiences. Doing so requires considering (and building, if necessary) the entire ecosystem, not just a piece of it. The real innovations often lie outside of what product managers typically consider to be the product.

Website Product Management

Managing a website, whether the target visitors are internal to an organization or are in the public at large, is not merely a matter of slapping together some web pages and linking them together. It's also not merely about design.

No, managing a website includes such challenges as:

• How do you elicit and prioritize the requirements for the website?
• How do you position and market the website to the target audience?
• How do you test your assumptions and continuously adjust to the needs of your target audience?

Note that those who manage any product face the same challenges. In his recently-published book, Website Product Management, David Hobbs teaches us how to manage websites as the "products" they are.

David was gracious enough to allow me to interview him about website product management and post his answers here. Enjoy!

Q. Why is product management important for websites?

A. Organizations are usually stuck in the rut of thinking of their web presences as a series of projects. On the face of it this may not seem like a problem: after all, some changes to websites are big enough to warrant project management techniques. That said, the emphasis causes problems in a variety of ways: a) it leads people to believe that maintaining a website is more of a technical problem, b) a focus on what’s possible (or fits within a budget) rather than what’s important, c) it overlooks ongoing change (and probably even maintenance), d) it comes too late to integrators / development shops, e) and it leads to disjointed efforts. Product management needs to drive what all those projects are doing, and also to ensure that the organization is set up for other types of changes that are smaller than projects. Fundamentally, product management needs to make sure the web presence is coherent.

Q. How is website product management different from product management elsewhere?

A. First off, it’s extremely easy to make changes to websites. Of course, some readers may think “it’s not simple to change my website,” but that is a process or implementation issue. At its core, it’s easy to make website changes. Since it’s always “live,” if you make some simple changes to HTML (or other components of the site) then the whole world sees the change. But this could also be said of products that are delivered via the web. So the real differentiator is the natural inclination for a wide range of folks to feel like they “own” parts of the site. Sometimes the ownership is legitimate, and sometimes it is less so. Of course any product has a range of stakeholders, but usually pretty much everyone (and certainly every major group) in an organization feels they own chunks of the website. That’s the reason in the book I focus on the need for everyone to think of their website as a product, and not just some central team. 

Q. When managing a website, what are the very most important things teams can do to manage them more like products?

A. For website product management, I break it down into three key ways of thinking: business first, long term, and broadly. So perhaps the most important thing teams can do is to consider each of these three aspects when considering any change. So when you are thinking about changes:

Ask first what the business need of the change is. If that isn't clear, then you probably should not do it (even if it is “easy” to do).
How will this work long term? Is this implemented in a way that will be difficult to maintain? Does it box you in for future change? Is it unnecessarily adding complexity to the implementation (or to the site visitor)?
Is this just going to benefit one group, or will it help the website broadly? 

You don’t have to buy the book to get a free flowchart to better handle website change.

Q. What activities of website product managers are the most strategic?

A. I don’t necessarily advocate that someone has the title of “website product manager.” Depending on the size of the web presence, an existing role could take on this extra responsibility. Regardless, I advocate for broad acceptance within an organization of product management thinking. 

I think the most strategic activities are: 1) defining the vision for the site, and 2) rising above the day-to-day requests in order to set a solid foundation (what I think of as getting the bones right). There is a subtle dance between these two. You can’t just declare a vision that isn't implementable. Actually, you can and people frequently do — they just aren't very helpful and usually counterproductive. The vision needs to be inspiring, grounded in the business need, and implementable. To be implementable, you need to think about how long term and on an ongoing basis how your goals will be met (just implementing once isn't enough). In particular, one of the most damaging things that can happen to a web presence are one-offs, and these happen all the time. Frequently one-offs are lauded as victories. But they often lead to an even more disjointed experience. The bones need to be defined to accommodate the types of changes that will be happening and not break.

Join Me for ProductCamp Austin 13

UPDATE: The presentation won "Best Session" at ProductCamp Austin 13. Check out the slides here.

Join me Saturday, August 2nd, 2014 for ProductCamp Austin 13.

I think I missed only one ProductCamp Austin "unconference" since helping to organize the first event. At ProductCamp, product management and marketing professionals teach, learn, and network.

For the upcoming event, I've proposed a session called "Google or search.com? Why We Suck at Naming Products and Companies".

What the heck is a hopdoddy?









Did you know that scientists have studied what makes a great brand name? The findings may surprise you. Our intuitions about brand names are the opposite of what the science tells us.

In the session, we’ll answer:

• What goals should you strive to achieve when choosing a name for your product or company?
• What does the science say about what types of names best accomplish these goals?
• How should you choose a name?

Prepare to challenge your intuitions.

As I write this blog entry, registration for ProductCamp Austin 13 is open.  If you'd like to present or lead a conversation at the event, propose a session. UPDATE: Check out this list of proposed sessions.

Hope to see you at ProductCamp Austin 13!

Stop Validating and Start Falsifying

The product management and startup worlds are buzzing about the importance of "validation". In this entry, I'll explain how this idea originated and why it's leading organizations astray.

Why Validate?

In lean startup circles, you constantly hear about "validated learning" and "validating" product ideas:

The assumption is that you have a great product idea and seek validation from customers before expending vast resources to build and bring it to market.

Indeed, it makes sense to transcend conventional approaches to making product decisions. Intuition, sales anecdotes, feature requests from customers, backward industry thinking, and spreadsheets don't form the basis for sound product decisions. Incorporating lean startup concepts, and a more scientific approach to learning markets, is undoubtedly a sounder approach.

Moreover, in larger organizations, sometimes further in the product life-cycle, everyone seems to have an opinion about such aspects of the business model as:

1. The most pervasive and urgent market problems the product should solve.
2. Whether the solution truly addresses the market problems.
3. What price customers would be willing to pay.
4. What tactics will most effectively drive prospects to buy.

Presumably, the "validated learning" approach of lean startup enables the organization to identify which opinions are factual and not mere speculation.

Validation in Practice

But let's consider what our naive lean startup practitioner (we'll call her "Cameron") does in practice.

Present the idea






Cameron puts together a slide deck, minimum viable product (MVP), or demo, presents her idea to one or more prospects, and eagerly awaits feedback. The prospects say, "I like it!" or "I would buy it!" Cameron feels warm inside, as the prospects have "validated" her and her idea.

Ask about the "pain point"






Since solving an urgent and pervasive problem or "pain point" is usually a prerequisite for product success, Cameron visits with prospects and asks them if they experience one of the problems she envisions her product would solve. "Yes," reply the prospects. Feeling "validated", Cameron does an internal high-five with herself.

Name a price






Cameron asks prospects what they would pay for a solution with the features she envisions for her product. Prospects "name their price". Or, in equally naive fashion, Cameron herself names a price and asks, "Would you pay X for the product?" Prospects reply affirmatively. Cameron can hardly contain her giddyness as her pricing assumptions are "validated".

Cameron feels extra special for having "gotten out of the building" to visit customers.

What's Naive about Cameron's Approach?

The results of all of Cameron's efforts are practically worthless, aside from the emotional affirmation she may feel. When you visit prospects, you don't get reliable information by posing hypothetical questions. When you seek validation from someone, you will tend to get it. Cameron unwittingly designed her interactions with prospects in a manner likely to confirm her preconceived ideas, and her interpretations of the results were naive. Let's call it "validation bias", an insidious psychological disorder that has infected the lean startup community.

Note: had Cameron conducted a survey, though seductive because it would have yielded quantitative data, the results would still have been suspect to the extent the questions were hypothetical and failed to confront prospects with real choices and commitments.

What's the Alternative?

If you want to be scientific in your approach to product decisions, you craft experiments and make falsifiable predictions, with the intention of testing, not "validating", the hypotheses and underlying assumptions.

Philosopher Karl Popper is famous for having championed falsificationism, a set of principles that shifted the emphasis from verifying scientific beliefs to ensuring they are possible to falsify via experiments:

According to Wikipedia:

"A statement is called falsifiable if it is possible to conceive an observation or an argument which proves the statement in question to be false."

In fact, Popper argued that a statement isn't scientifically factual at all if it isn't falsifiable.

What Cameron Could Do Differently

Instead of asking hypothetical questions, Cameron could ask the prospects what they actually do (and have done) instead of what they would do. She could probe into their current and past experiences and note whether the supposed problems manifested themselves in those experiences. She could sit with prospects in their native environments and observe (ethnography), first hand, their situations and challenges. Cameron could test her value proposition and pricing hypotheses by quantifying the costs that problems and their workarounds impose on prospects. She could ask for an actual commitment to pay for the product once it's released.

Cameron could also work with her team to craft "digital" experiments and predict how those experiments will turn out. For example, her team could author "how-to" guides, downloadable on landing pages, that help prospects solve the problems she assumes they face. The team could predict how many people will visit those landing pages and how many people will proceed to download the guides. (To make prospects aware of the how-to guides, the team could run Facebook ads or Google AdWords to drive people to those landing pages and see if the predicted number of page visits and downloads materializes.)

The possibilities are endless, but they require a mindset that emphasizes falsifying product ideas and business model hypotheses, not "validating" them.

Join Me at ProductCamp Austin 12

Join me Saturday, February 15th, 2014 for ProductCamp Austin 12.

By now, if you're a product management, marketing, or technology professional, you've probably heard of ProductCamp. ProductCamp is an "unconference" where product management and marketing professionals teach, learn, and network.

ProductCamp depends on volunteers to organize it, propose and lead sessions, provide lively conversation and debate, set up and tear down the day of the event, and recruit sponsors to keep it free. Participants can propose sessions prior to the event, and participants vote the morning of the event to determine which sessions they will have the opportunity to attend throughout the day.

As I write this blog entry, registration for ProductCamp Austin 12 is open. If you're ready to commit to participating in the product management and marketing conversation, I suggest you register now. The slots usually fill up quickly. If you'd like to present or lead a conversation at the event, propose a session.

I've proposed a session called "5 Ways Companies Make Product Decisions".

Is your company making ad hoc or informed, deliberate product decisions?

In the session, we'll look at five ways companies make strategic and ongoing tactical decisions about how they develop, market, and sell products and solutions. How do they decide what features to include in their products, what messages they will use to articulate the value of their products, what marketing tactics they will use, what prospective customers they will target, and other day-to-day choices?

We'll discuss the pros and cons of each method and explore other methods that may be more likely to result in product success.

I'd love to hear your perspective and see you at ProductCamp Austin 12.

Competitive Mindshare Maps

Why You Need a Competitive Mindshare Map

The core of your product strategy lies in your product's positioning and unique value proposition (UVP). It should drive nearly all product decisions, including the roadmap, feature prioritization, marketing messages, and sales approaches.

A sound unique value proposition depends on:

1. Value. It should convey the value of your product. Value is rooted in the problems your product solves.
2. Competitive landscape. It should differentiate your product from available alternatives.
3. Perception. It should acknowledge perceived weaknesses of the product and perceived strengths of competing products.

Surprisingly, most companies take the value of their products for granted and don't bother to explicitly formulate a unique value proposition. For others, determining a unique value proposition is haphazard, with little or no process guiding the decision. At best, they focus on the value and pay insufficient attention to perceptions and the competitive landscape.

Enter the competitive mindshare map. A competitive mindshare map helps you determine a unique value proposition for your product and ensures it properly accounts for the competitive landscape and market perceptions.

What is a Competitive Mindshare Map?

A competitive mindshare map is a visualization of how products are positioned in the competitive landscape. The competitive landscape, however, is not a set of line items or numbers in a spreadsheet or chart. The competitive landscape is the mind of the prospect.

The fourth law among Ries and Trout's 22 immutable laws of marketing is the Law of Perception.  It states:

"Marketing is not a battle of products, it's a battle of perceptions."

A competitive mindshare map divides the mind of the prospect into territories and places each offering into the territory it occupies. Just as a country, no matter how great its military might, can't occupy an entire continent, your product can't occupy every territory in the prospect's mind. You must acknowledge the perceived strengths of competing products and the perceived weaknesses of your product. You must cede the territory your product can't realistically occupy.

Positioning your product is a matter of fortifying and defending the territory it already occupies, invading weakly-defended territory, or capturing uncontested territory your product can hold.

Composing a Competitive Mindshare Map

To compose a competitive mindshare map, you start with an image of a head and a brain (you can get a template below). I suggest using Google Drawings, but you can use any drawing tool, even a white board and markers if you like. For your product and each competing product, you then:

1. Place a logo in a distinct region of the brain. You carve up the brain into any size, shape, and number of regions. Typically, you can extract logos from competitor web pages.
2. Provide a short (no more than three major words) category name or theme that conveys the perceived strength the product "owns" in the minds of prospects.
3. Provide a terse explanation of the product's perceived strength.

Depending on the tool you're using to compose the map, you can make the logos hyperlink to web pages with more information (such as actual product pages or more detailed competitive intelligence documentation).

After your first draft, take a step back and ask yourself:

1. Does the map include the major competitors, including in-house solutions that customers might build themselves?
2. Does your category name or theme convey a promise that your product actually delivers (or will deliver in the future)?
3. Do the category names (or themes) and descriptions of competitors accurately represent their perceived strengths?

Positioning Guidelines

Positioning your product doesn't have to be "black magic". A number of concrete factors should guide the positioning of your product and the selection of a unique value proposition. This article will help you take a methodical approach to positioning as you compose your competitive mindshare maps.

Above all, keep in mind that the more focused the category, the more powerful and easier it is to defend. Indeed, the Law of Focus states:

"No matter how complicated the product, no matter how complicated the needs of the market, it's always better to focus on one word or benefit than two or three or four."

Your brand gains power from sacrifice.  As Al Ries and Laura Ries state in the Law of Expansion (from The 22 Immutable Laws of Branding):

"The power of a brand is inversely proportional to its scope."

and in the Law of Contraction:

"A brand becomes stronger when you narrow the focus."

You'll face strong pressure to extend and expand the territory your product occupies. As a general rule, it's best to resist it. Ironically, the power gained by narrowing the brand's focus has a halo effect that increases - not decreases - its reach into the minds of prospects.

Get the Template

The easiest way to start is to get the template. Click one of the options below. Simply create a copy of the template in Google Drawings or download the template in PNG image format.

Next Steps

To the extent you didn't collaborate with others to compose the competitive mindshare map, schedule sessions to review it with the executive team, marcom, sales people, and customers.

If you're torn between two value propositions, you can devise experiments (such as A/B tests with landing pages) to test how they resonate with prospects.

Incorporate the unique value proposition into the business model canvas for your product. (Go here for an overview and downloadable model of these and other lean startup concepts.)

Ideally, your team evaluates every product decision in terms of the extent to which it fulfills the promise of the unique value proposition or helps to instill, in the minds of prospects, the association between your product and its unique value proposition.

Lean Startup Concepts

Now that you've had a largely buzzword-free introduction to lean startup methods, you may be interested in a bit deeper understanding of the concepts and terminology.  Lean startup methods incorporate scientific methods and principles of agile development to help practitioners learn markets quickly and reliably and deliver successful products.  But lean startup enthusiasts and practitioners throw around a lot of terminology and concepts that may seem alien or not particularly meaningful to you.

Let's examine and demystify the basic lean startup terminology with a rich and concise conceptual model. To view the model of lean startup concepts, click the image below:

Lean startup concepts fall within four different clusters: hypotheses, learning, market, and product.




Hypotheses

Forming hypotheses is a component of lean startup methods. The business model represents the strategy driving more tactical product decisions, and it consists of a set of hypotheses. These hypotheses are rooted in the market problems the product is intended to solve. The product serves as a set of solutions to these problems.

Prospects and customers typically have existing alternatives to at least partially address or work around the same problems. The unique value proposition conveys an overarching theme of addressing the problems, and it leverages the unfair advantage the company possesses relative to any competition. A high-level concept summarizes the unique value proposition using a metaphor likely to be familiar to members of the targeted customer segments.

Metrics measure the key activities from which the company and customers derive value, such as adoption, usage, and purchases. Customer segments represent the target market for the product, and sales and marketing channels reach these customer segments. Revenue streams come from such sources as product purchases and advertising, and they offset the cost structure for the product.




Learning

Lean startup methods are first, and foremost, about efficient learning. We conduct interviews of subjects, which include prospects and existing customers, to uncover initial insights. Hypotheses rest on assumptions. These assumptions entail predictions that we can test with purposeful experiments. The results of these experiments yield further insights that we use to adjust our assumptions and hypotheses.




Market

We sell products to prospects who face problems, and those prospects become customers. Early adopters are initial customers that can be key partners in helping lean startup practitioners form, test, and refine hypotheses.





Product

The business model outlines the strategy for a successful product. Developing a minimum viable product (MVP) enables us to minimize the amount of time needed to deliver value to customers, to determine whether prospects will pay to solve their problems, to determine whether the product satisfactorily addresses those problems, and to determine whether the product is usable.


Parting Thoughts

Most lean startup concepts aren't new. Taken as a whole, however, they incorporate and enhance the common ways that companies make product decisions in a manner that can accelerate, and improve the reliability of, learning and implementation. The scientific and agile approach fosters a greater likelihood of product success.

Is your company incorporating all or most of these concepts in the ways it makes product decisions? If so, what sorts of successes and challenges have you had? If not, why not?

What Is Lean Startup? Here's What You Need to Know

You've probably heard a lot of buzz about "lean startups".  You may wonder if it's mere hype, applies just to tiny bootstrap ventures, or if adopting some lean startup methods might actually benefit your company.

One of the top problems companies face as they make product decisions
is that the process of learning the market, and learning what makes the product successful, is slow and unreliable. Sometimes they suffer analysis paralysis, swerve from one big deal to the next, allow conventional industry wisdom to stifle innovation, or squabble over uninformed personal opinions. In other cases, they make decisions quickly but fail to learn from their inevitable mistakes until after they've invested exorbitant amounts of time and money.

If you find that your organization is facing this problem, it's worthwhile to consider lean startup methods.

Just as scientists use the scientific method to learn how the universe works, your product team can apply lean startup methods to learn what business model for your product will work in the market. Lean startup practitioners:

1. Collect data. Practitioners observe and interview prospects to gain qualitative insights into their situations, challenges, and the ways they operate. They also examine quantitative data for further insights.
2. Form hypotheses. Based on their observations and insights, practitioners form and capture hypotheses about the business model for their products. These hypotheses include the problems to solve, the key elements of the solution, the unique value proposition, the buyer and user personas, key metrics or user activities, costs, and revenue streams. Hypotheses may also include measurable predictions of the impact of various marketing or sales tactics.
3. Conduct experiments. Recognizing that at least some of their initial business model hypotheses are likely to be wrong, practitioners design and run experiments to test the hypotheses. Often these experiments confront prospects with real-world choices (such as functional product) and measure how prospects behave when confronted with these choices.
4. Learn and adjust. Having conducted experiments to test hypotheses, practitioners analyze the results and adjust their hypotheses.

These activities often occur in parallel and not necessarily in this sequence. For example, entrepreneurs commonly think of product ideas prior to formally collecting market data.

Applying these methods in an iterative or continuous fashion enables product teams to confront product strategy risks and quickly and reliably learn markets with more targeted - and ultimately, less wasteful - business investments.

Lean startup practitioners essentially apply agile methods to the entire business for a product. Most software companies have adopted at least some agile development practices. But unlike companies that iterate merely on the development of the product, lean startups iterate on the requirements, the positioning, the target markets and personas, and sales and marketing tactics.  They monitor and optimize the sales and usage funnels. They emphasize the delivery of working product and prospect-touching experiments over exhaustive planning and documentation devoid of external feedback.

Summary

Learning markets reliably and expeditiously, and learning what will make products successful, is one of the primary challenges many companies face, whether those companies are startups or large, established corporations. By applying scientific practices and agile principles across the entire business for a product, lean startup is designed to address this challenge.

How does your company currently make product decisions?  How would you know it's time to add new approaches or practices to the mix? The next blog entry will explain how you can begin to put some lean startup methods into practice right away, once your team is ready for some change.

Thanks to Koen Bosma, Ash Maurya, and Emiliano Villareal for providing helpful feedback on this blog entry, and for their thought leadership on the topic of lean startup methods.