Tuesday, October 06, 2015

Is Customer Development Pseudoscience?

The “Science” of Lean Startup

Lean startup practitioners embrace the scientific method, seeking the "truth" about what business model and strategy will lead to product success. We do so by:

  • Formulating hypotheses
  • Crafting and running experiments to test them
  • Learning from the experiments
  • Iteratively feeding our learnings back into revised hypotheses

Sounds pretty scientific, at least in spirit, doesn't it? Yet this process actually neglects a key ingredient in the scientists' mode of operation. To identify what’s missing, let’s examine “customer development”.

Customer Development

Steve Blank is one of the pioneers of the lean startup movement. He introduced into the lean startup lexicon the term “customer development”. Customer development consists of sessions and interactions with customers to test hypotheses.

For example, a product manager might interview a prospect, asking if she agrees with the product manager’s hypotheses about the problems she faces or the value the product might bring to the prospect. Or she might “pitch” her product ideas to a prospect, as a sales person might do, ask for payment or a commitment of some sort, and learn from the prospect’s reactions.

Customer development practitioners, according to Blank, start with a relatively specific set of preconceived hypotheses and focus customer sessions almost exclusively on assessing the “validity” and applicability of these hypotheses to customers. As practitioners test the hypotheses, they capture data and may stumble across some insights. But customer development sessions do not include open exploration into customers’ challenges, according to Blank.

In fact, Blank pigeonholes open exploration as part of “design thinking”. He suggests design thinking is, at least in common practice, a lengthy, heavyweight approach to learning, and therefore not appropriate for lean startups. (A comment by my former colleague, Jen van der Meer, aptly explains that this view of design thinking is a caricature.)

Even lean startups should include open exploration in their customer interview and observation sessions, lest they neglect a vast body of important potential insights.

Rummy’s Unknown Unknowns

Former Secretary of State Donald Rumsfeld is famous for, in a press conference, having distinguished among “known knowns”, “known unknowns”, and “unknown unknowns”:

"As we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don't know." - Donald Rumsfeld, February 12, 2002

Known known: Something we know we know.
Known unknown: Something we know we don’t know.
Unknown unknown: Something we don’t know we don’t know.

Learning is the process of converting unknowns (whether known or unknown) to known knowns.

Blank’s Misguided Focus

"Customer Development starts with, 'I have a technology/product, now who do I sell it to?'” - Steve Blank, "Driving Corporate Innovation" blog entry

By focusing customer development on testing preconceived hypotheses, Blank has us concentrate on the known unknowns, or the things we know we don’t know (with sufficient confidence). By definition, a preconceived idea or hypothesis is not, and cannot be, an unknown unknown.

If the body of unknown unknowns were very small relative to that of known unknowns, then we could employ Blank’s approach, and we might eventually converge on a set of hypotheses constituting a successful product strategy.

In reality, though, the typical body of knowledge (and lack thereof) might break down as follows:

In any given domain ripe for innovation, and particularly early in a lean startup process, the body of unknown unknowns may eclipse the body of known unknowns. Moreover, you may think you know the extent of the unknown unknowns, but how would you know you know? Consequently, even if we have a prior set of hypotheses - and even a product based on those hypotheses - we should focus some of our efforts on exploring the vast body of unknown unknowns that exist in a domain or industry.

Focus on the Unknown Unknowns

"Customer interviews are about exploring what you don't know you don't know." - Ash Maurya, Running Lean, page 71

We should focus some of our sessions with prospective or existing customers on exploring the situations and challenges they face, not exclusively on directly testing our preconceived notions of what they may be.

You likely do have a set of preconceived hypotheses, but these hypotheses exist within the context of a domain and set of possible stakeholders. This context helps you narrow the scope of your inquiry: the people you should observe and interview, and which aspects of their lives you should explore.

Much as a scientist conducts field studies, a lean startup practitioner should spend at least some time in the field to explore. Much as a therapist uncovers patients' problems by exploring their past and present lives, a lean startup practitioner should uncover unknown unknowns by exploring prospects' past and present ways of getting jobs done.

By employing an exploratory approach grounded in a prospect’s reality, we’ll not only uncover insights we never anticipated, but we’ll also indirectly test the known unknowns. A prospect’s unprompted description of a painful experience is far more significant than a response to your eager “do you experience this pain” question.


There is no need to debate the semantics of “customer development” and “design thinking”. Steve Blank popularized the term “customer development”, and we can defer to his explanation of its definition and its focus. Regardless of the definitions, however, smart lean startup practitioners should include some open exploration in their approaches to interviewing and observing customers. Failure to do so risks missing important opportunities to create innovative and transformative product experiences.


Mark Waddoups said...

Good points and I have found the same to be true. You can gain a lot by asking exploritory questions and Ash's book Running Lean has some scripts to build off of that are great starting points. When I am more focused on uncovering problems and using the hypothesis as just a tool or starting point I am better able to identify true opportunities to solve customer pain and pivot faster to the right solution.

Roger Cauvin said...

Thanks for the comment, Mark. I like your description of using the hypotheses as a "starting point" for exploration.

kenny! said...

Your opinion on ethnography with respect to customer development?

Roger Cauvin said...

Kenny, I mentioned observation a few times in the blog entry. Observing customers in their native environments can be an excellent way to uncover insights, since it focuses on what customers actually do, and not on what they say they might do under hypothetical conditions. In some cases, we may notice things about customers' behavior that the customers themselves don't notice.

Greg Prickril said...

Hey Roger,
Enjoyed the post as usual. What struck me most was a phrase you used early on: "hypotheses about the problems she faces...". I find product managers far too often fall victim to the human tendency to jump to the solution before we've done due diligence on understanding the problem. In my experience, what you first assume is "the problem" is typically a set of problem with highly relevant underlying causal factors. The Blank quote on customer development would seem to perpetuate this error, although it may be intentional (I'll have to digest the blog entry you link to).

Roger Cauvin said...

Greg, thanks for commenting. You're right that product managers and many founders often jump right to the solution without having formed and articulated the hypotheses regarding the market problems the solution is intended to solve. Blank at least urges entrepreneurs to document their hypotheses (not just the solution).

It's amazing how few companies have composed a lean canvas or business model canvas. One of the first things I do when I go to work for a company is sit key executive and internal stakeholders down and say, "We have a wealth of market and strategy knowledge in this room, but it's all in our heads. Let's get all of our assumptions down in a one-page canvas."

But I also immediately begin setting the wheels in motion to conduct prospect interviews and to focus on the unknown unknowns - not so much the prior hypotheses - in these interviews.

Luke said...

Dear Roger, have you met Cindy Alvarez' approach? I find it quite appealing because it starts with a hypothesis and then leads you to understand the problems in the hypothesis, not the solutions... The unknown unknowns about problems are gold

Roger Cauvin said...

Luke, I'm a big fan of Cindy Alvarez and her approach. She wrote a great blog entry in 2010 about the types of questions to ask in customer development interviews.

I should reiterate, however, that we shouldn't just be seeking to understand the problems in our preconceived hypotheses. Our exploration should also be about uncovering new insights that may bear little relation to our hypotheses.

Jim said...

Good points here, Roger. I'm glad to find your blog!

We try to do both on our team, focused research and open-ended conversations to understand the top-of-the-list pain points for our audience.