Wednesday, June 10, 2009

Why Product Management Interviews Suck

Before becoming a product manager, I was a software engineer for about eleven years. During my career as a software engineer, I interviewed for many different positions and many different companies. Some of the companies had perfected their interview process; they employed such methods as:
  • Analysis and design sessions
  • Coding quizzes
  • Design pattern questions
  • Development process question/answer sessions
The candidate's performance during each segment was fairly objective and straightforward to assess, and hiring managers felt confident that a candidate would excel on the job if she performed well. Any software engineering "rock star" felt confident that she would come close to acing these exercises and quizzes.

Now, as an experienced product manager having recently interviewed at various companies, I'm struck that 95% of product manager interviews yield almost no useful or reliable information for assessing how well the product manager would perform on the job.

Unfortunately, most interviewers concentrate on broad and fluffy questions about previous experience but don't probe into what methods and principles a candidate employs to make key decisions as a product manager. In particular, notably absent from these interviews have been such tools as:
  • Requirements elicitation exercises
  • Product positioning exercises
  • Quizzes on product management principles, methods, and concepts
  • Product management process quiz (or Scrum quiz for companies that practice it)
In fact, the only quiz on product management I've ever seen is the one that Pragmatic Marketing uses for its certification exam. (Seilevel has a somewhat rigorous set of exercises and questions in its interview process, but it focuses more on product specification than on product management.)

If you're hiring a product manager, try creating a rigorous set of tools for assessing how well a candidate knows product management. You'll not only make better hiring decisions, it will help build a better understanding of why you're hiring a product manager in the first place.

10 comments :

Cindy said...

It's almost impossible to find good questions/exercises online if you're searching for them, too.

We'd be doing a huge service to the Product Management community if we collectively came up with question or exercise suggestions (collective = could cover different industries/domains).

Mike Lunt said...

From what I've witnessed, the product manager role varies greatly from company to company. In many cases, unfortunately, this role is setup to fill varying gaps between marketing and engineering. Often, the most important role of converting customer needs into product is being subsumed by one or more individuals who may be reluctant to give up this control.

All of these variables add up to make the product management interview a mine field of questions and agendas. Having a firm grasp of the organization's needs and the people involved may be more important for landing this type of job than most others.

Roger L. Cauvin said...

Cindy, I like your idea of collectively coming up with questions and exercises for use in product management interviews. Perhaps we could use a series of entries on our blogs to brainstorm them?

Roger L. Cauvin said...

Mike, I totally agree with you that the product manager role varies, and that the role is usually a way of filling a tactical gap or set of immediate needs.

In fact, in a recent interview, it became apparent that the hiring manager was simply looking for someone to perform a hodge-podge of tactical activities that she had become too busy to do herself.

I happened to talk to three other product managers with whom she discussed the position, and we all realized that we told her she was barking up the wrong tree.

In short, it's not too difficult to figure out what the company needs, but it's very difficult to overcome the hiring manager's preconceptions of what the company needs.

Sandra said...

Roger,

A perspective from someone who knows nothing about product management...

Very often we make decisions based on intuition (vs. reason, i.e. intellectualizing). In his book "How We Decide" http://bit.ly/nOf8k Lehrer explains that intuition is a collection of positively reinforced behaviors that have been internalized. In other words, intuition is a shortcut to a feeling derived after many years of thousands of positively reinforced thoughts. Sometimes it's helpful to intellectualize our decision but many times it's wise to go with our more "emotional" intuition.

Roger L. Cauvin said...

Sandra, I totally agree that not all decision-making can or should be formalized.

Product management interviews are on the extreme side of the spectrum. There are not even any de facto or informal standards and almost no probing into the candidates' knowledge of established principles of product management and marketing.

If inteviewers are unable to test or measure the quality of candidates with even a modicum of rigor (aside from credentials listed on a resume, which are unreliable), then it strongly suggests that they simply don't understand the role and why they need to fill it.

Len said...

8 Non-Useless Interview Questions for Product Managers is a good read on this topic.

http://www.cindyalvarez.com/psychology/8-non-useless-interview-questions-for-product-managers

Now only if more hiring managers read this!

Rich said...

I fish for "tell me a PM mistake you made and how you fixed it" to find out if the candidate has been on a product long enough to live with consequences of decisions. Can naturally follow with "how successful was your product and why?" to listen for not-my-fault-it-failed

Scott said...

Roger,
Great topic with great comments from all.

In my recent experience looking for my next PM gig I've found both good and bad in terms of a companies interviewing plan/strategy and as a result the kind of questions I get asked. I've been asked ones close to what Rich suggested. A good one I did get was "Tell me about a product you've recently started using in your personal/professional life and tell me what you would do to make it better, why and how would you make those decisions?"

I've also been asked to whiteboard my plan for scaling/leveraging PM function within a fast growing company given certain goals and constraints as inputs to the problem. This was fun for me and the VP.

I will definitely go read Cindy's questions and would love to contribute to creating questions/exercises for the community.

How can we start this? Should we blog it as Roger suggested?

Sarah Throlson said...

I think all job interviews, both for the interviewer and the interviewee. If only there were a way we could just skip the process altogether... well, we'd probably end up with businesses full of unqualified employees. I guess we'll have to stick to doing interviews for now ;)