Tuesday, October 06, 2015

Is Customer Development Pseudoscience?

The “Science” of Lean Startup

Lean startup practitioners embrace the scientific method, seeking the "truth" about what business model and strategy will lead to product success. We do so by:

  • Formulating hypotheses
  • Crafting and running experiments to test them
  • Learning from the experiments
  • Iteratively feeding our learnings back into revised hypotheses

Sounds pretty scientific, at least in spirit, doesn't it? Yet this process actually neglects a key ingredient in the scientists' mode of operation. To identify what’s missing, let’s examine “customer development”.

Customer Development

Steve Blank is one of the pioneers of the lean startup movement. He introduced into the lean startup lexicon the term “customer development”. Customer development consists of sessions and interactions with customers to test hypotheses.

For example, a product manager might interview a prospect, asking if she agrees with the product manager’s hypotheses about the problems she faces or the value the product might bring to the prospect. Or she might “pitch” her product ideas to a prospect, as a sales person might do, ask for payment or a commitment of some sort, and learn from the prospect’s reactions.

Customer development practitioners, according to Blank, start with a relatively specific set of preconceived hypotheses and focus customer sessions almost exclusively on assessing the “validity” and applicability of these hypotheses to customers. As practitioners test the hypotheses, they capture data and may stumble across some insights. But customer development sessions do not include open exploration into customers’ challenges, according to Blank.

In fact, Blank pigeonholes open exploration as part of “design thinking”. He suggests design thinking is, at least in common practice, a lengthy, heavyweight approach to learning, and therefore not appropriate for lean startups. (A comment by my former colleague, Jen van der Meer, aptly explains that this view of design thinking is a caricature.)

Even lean startups should include open exploration in their customer interview and observation sessions, lest they neglect a vast body of important potential insights.

Rummy’s Unknown Unknowns

Former Secretary of State Donald Rumsfeld is famous for, in a press conference, having distinguished among “known knowns”, “known unknowns”, and “unknown unknowns”:

"As we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don't know." - Donald Rumsfeld, February 12, 2002

Known known: Something we know we know.
Known unknown: Something we know we don’t know.
Unknown unknown: Something we don’t know we don’t know.

Learning is the process of converting unknowns (whether known or unknown) to known knowns.

Blank’s Misguided Focus

"Customer Development starts with, 'I have a technology/product, now who do I sell it to?'” - Steve Blank, "Driving Corporate Innovation" blog entry

By focusing customer development on testing preconceived hypotheses, Blank has us concentrate on the known unknowns, or the things we know we don’t know (with sufficient confidence). By definition, a preconceived idea or hypothesis is not, and cannot be, an unknown unknown.

If the body of unknown unknowns were very small relative to that of known unknowns, then we could employ Blank’s approach, and we might eventually converge on a set of hypotheses constituting a successful product strategy.

In reality, though, the typical body of knowledge (and lack thereof) might break down as follows:

In any given domain ripe for innovation, and particularly early in a lean startup process, the body of unknown unknowns may eclipse the body of known unknowns. Moreover, you may think you know the extent of the unknown unknowns, but how would you know you know? Consequently, even if we have a prior set of hypotheses - and even a product based on those hypotheses - we should focus some of our efforts on exploring the vast body of unknown unknowns that exist in a domain or industry.

Focus on the Unknown Unknowns

"Customer interviews are about exploring what you don't know you don't know." - Ash Maurya, Running Lean, page 71

We should focus some of our sessions with prospective or existing customers on exploring the situations and challenges they face, not exclusively on directly testing our preconceived notions of what they may be.

You likely do have a set of preconceived hypotheses, but these hypotheses exist within the context of a domain and set of possible stakeholders. This context helps you narrow the scope of your inquiry: the people you should observe and interview, and which aspects of their lives you should explore.

Much as a scientist conducts field studies, a lean startup practitioner should spend at least some time in the field to explore. Much as a therapist uncovers patients' problems by exploring their past and present lives, a lean startup practitioner should uncover unknown unknowns by exploring prospects' past and present ways of getting jobs done.

By employing an exploratory approach grounded in a prospect’s reality, we’ll not only uncover insights we never anticipated, but we’ll also indirectly test the known unknowns. A prospect’s unprompted description of a painful experience is far more significant than a response to your eager “do you experience this pain” question.


There is no need to debate the semantics of “customer development” and “design thinking”. Steve Blank popularized the term “customer development”, and we can defer to his explanation of its definition and its focus. Regardless of the definitions, however, smart lean startup practitioners should include some open exploration in their approaches to interviewing and observing customers. Failure to do so risks missing important opportunities to create innovative and transformative product experiences.