The “Science” of Lean Startup
Lean startup practitioners embrace the scientific method, seeking the "truth" about what business model and strategy will lead to product success. We do so by:
- Formulating hypotheses
- Crafting and running experiments to test them
- Learning from the experiments
- Iteratively feeding our learnings back into revised hypotheses
Sounds pretty scientific, at least in spirit, doesn't it? Yet this process actually neglects a key ingredient in the scientists' mode of operation. To identify what’s missing, let’s examine “customer development”.
Customer Development
Steve Blank is one of the pioneers of the lean startup movement. He introduced into the lean startup lexicon the term “customer development”. Customer development consists of sessions and interactions with customers to test hypotheses.
For example, a product manager might interview a prospect, asking if she agrees with the product manager’s hypotheses about the problems she faces or the value the product might bring to the prospect. Or she might “pitch” her product ideas to a prospect, as a sales person might do, ask for payment or a commitment of some sort, and learn from the prospect’s reactions.
Customer development practitioners, according to Blank, start with a relatively specific set of preconceived hypotheses and focus customer sessions almost exclusively on assessing the “validity” and applicability of these hypotheses to customers. As practitioners test the hypotheses, they capture data and may stumble across some insights. But customer development sessions do not include open exploration into customers’ challenges, according to Blank.
In fact, Blank pigeonholes open exploration as part of “design thinking”. He suggests design thinking is, at least in common practice, a lengthy, heavyweight approach to learning, and therefore not appropriate for lean startups. (A comment by my former colleague, Jen van der Meer, aptly explains that this view of design thinking is a caricature.)
Even lean startups should include open exploration in their customer interview and observation sessions, lest they neglect a vast body of important potential insights.
Rummy’s Unknown Unknowns
Former Secretary of State Donald Rumsfeld is famous for, in a press conference, having distinguished among “known knowns”, “known unknowns”, and “unknown unknowns”:
"As we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don't know." - Donald Rumsfeld, February 12, 2002
Known known: Something we know we know.
Known unknown: Something we know we don’t know.
Unknown unknown: Something we don’t know we don’t know.
Learning is the process of converting unknowns (whether known or unknown) to known knowns.
Blank’s Misguided Focus
"Customer Development starts with, 'I have a technology/product, now who do I sell it to?'” - Steve Blank, "Driving Corporate Innovation" blog entry
By focusing customer development on testing preconceived hypotheses, Blank has us concentrate on the known unknowns, or the things we know we don’t know (with sufficient confidence). By definition, a preconceived idea or hypothesis is not, and cannot be, an unknown unknown.
If the body of unknown unknowns were very small relative to that of known unknowns, then we could employ Blank’s approach, and we might eventually converge on a set of hypotheses constituting a successful product strategy.
In reality, though, the typical body of knowledge (and lack thereof) might break down as follows:
In any given domain ripe for innovation, and particularly early in a lean startup process, the body of unknown unknowns may eclipse the body of known unknowns. Moreover, you may think you know the extent of the unknown unknowns, but how would you know you know? Consequently, even if we have a prior set of hypotheses - and even a product based on those hypotheses - we should focus some of our efforts on exploring the vast body of unknown unknowns that exist in a domain or industry.
Focus on the Unknown Unknowns
"Customer interviews are about exploring what you don't know you don't know." - Ash Maurya, Running Lean, page 71
We should focus some of our sessions with prospective or existing customers on exploring the situations and challenges they face, not exclusively on directly testing our preconceived notions of what they may be.
You likely do have a set of preconceived hypotheses, but these hypotheses exist within the context of a domain and set of possible stakeholders. This context helps you narrow the scope of your inquiry: the people you should observe and interview, and which aspects of their lives you should explore.
Much as a scientist conducts field studies, a lean startup practitioner should spend at least some time in the field to explore. Much as a therapist uncovers patients' problems by exploring their past and present lives, a lean startup practitioner should uncover unknown unknowns by exploring prospects' past and present ways of getting jobs done.
During interviews, ask prospects what they’ve done and what they currently do. Avoid leading questions, hypothetical questions, asking them what they “want”, having customers design your product, and ignoring change management issues.
By employing an exploratory approach grounded in a prospect’s reality, we’ll not only uncover insights we never anticipated, but we’ll also indirectly test the known unknowns. A prospect’s unprompted description of a painful experience is far more significant than a response to your eager “do you experience this pain” question.
Conclusion
There is no need to debate the semantics of “customer development” and “design thinking”. Steve Blank popularized the term “customer development”, and we can defer to his explanation of its definition and its focus. Regardless of the definitions, however, smart lean startup practitioners should include some open exploration in their approaches to interviewing and observing customers. Failure to do so risks missing important opportunities to create innovative and transformative product experiences.
Comments
Enjoyed the post as usual. What struck me most was a phrase you used early on: "hypotheses about the problems she faces...". I find product managers far too often fall victim to the human tendency to jump to the solution before we've done due diligence on understanding the problem. In my experience, what you first assume is "the problem" is typically a set of problem with highly relevant underlying causal factors. The Blank quote on customer development would seem to perpetuate this error, although it may be intentional (I'll have to digest the blog entry you link to).
It's amazing how few companies have composed a lean canvas or business model canvas. One of the first things I do when I go to work for a company is sit key executive and internal stakeholders down and say, "We have a wealth of market and strategy knowledge in this room, but it's all in our heads. Let's get all of our assumptions down in a one-page canvas."
But I also immediately begin setting the wheels in motion to conduct prospect interviews and to focus on the unknown unknowns - not so much the prior hypotheses - in these interviews.
I should reiterate, however, that we shouldn't just be seeking to understand the problems in our preconceived hypotheses. Our exploration should also be about uncovering new insights that may bear little relation to our hypotheses.
We try to do both on our team, focused research and open-ended conversations to understand the top-of-the-list pain points for our audience.
-Jim