Introduction

Interviewing is an essential component of the customer discovery process that the lean startup methodology uses [1]. Unfortunately, many entrepreneurs focus on their products, and not on their customers. They try to fit a product to a particular market or customer. As a result, they fail to gather valuable information and learnings that can make or break the success of the product and firm. In the end, they develop products that fall short of addressing customer requirements. Such outcomes are because many entrepreneurs lack strong customer discovery skills and processes, in which proper interviewing techniques are essential.

Investigators of the lean startup process report challenges concerning the proper use of questions, the gathering of insights, and the interpretation of the interview findings by entrepreneurs [2,3]. In examining how Indian startups implement of the lean startup methodology, Rao highlights the difficulties involved with obtaining and conducting productive interviews [3]. Chen and colleagues discuss several cognitive biases with Indonesian entrepreneurs when adopting the “Get-Out-of-the-Building” approach [2].

A previous paper on this topic of interviewing and customer discovery focused on the strategic aspects of planning and conducting interviews [4]. This publication examined the four-quadrant planning approach, along with the “FUNNEL,” and the “SPEECH” techniques. These provided useful frameworks for entrepreneurs to use.

This paper builds on this initial publication that explores interviewing strategies. It focuses on critical tactical pieces involved with interviewing: (1) customer selection and engagement; (2) the interview process, documentation, and analysis; (3) questions; (4) interviewing and question biases; and (4) practical examples of effective and ineffective questions as identified in Rob Fitzpatrick’s book, The Mom Test [5].

Targeting the Right Customers

Identifying the right customer or customers is essential for conducting compelling interviews. When entrepreneurs go out to interview, they do not necessarily focus on the appropriate individual or persona who defines the target audience. This point is critical because the target sample should be representative. Sometimes, the entrepreneur fails to recognize that multiple customers (and personas) may participate in deciding on a particular product or service. This consideration is especially important in the “Business-to-Business” (B2B)and and the “Business-to-Consumer” (B2C) setting.. The interviewer needs to consider the different roles with respect to the purchase process.These include the beneficiary, the influencer, the saboteur (e.g., a competitor), and the decision makers. Hence, one needs to engage such players, particularly the beneficiaries and decision makers, and consider each stakeholder’s situation and motivation when interviewing these individuals. Such can help to optimize the customer acquisition process and the user experience, which is essential to loyalty and repeated purchases.

Getting the Interview

Once the interviewer identifies the target customers, one needs to engage them to conduct the “Painful” interview. Many times, this effort will include some communication to set up a time. The invitation can be via a telephone message, email, or letter.

Engagement communications involve three parts, which should take only thirty seconds or less (Figure 1). Part 1 sets the frame. It can include a “Hook.” The interviewer should personalize the opening with the customer’s name and including one’s own name and position in the first sentence. One should state how he/she obtained the interviewee’s name (e.g., referral or expert opinion) or a relevant, insightful question to prompt interest. The frame then covers the relevant information and the interview’s purpose. Remember, this opening needs to be concise. Part 2 covers what the interview would entail, including the time needed, a high-level description of the topic, and whether compensation is involved. One should try to keep the proposed interview times short (e.g., 10-15 minutes for uncompensated and 30 minutes for compensated interviews). Part 3 puts forth an action step. One should ask about availability or, even better, propose a few times from which the interviewee can choose. The interviewer should include one’s contact info (e.g., a telephone number, email, text number, or an RSVP weblink) or a response. The bottom line is to make this initial engagement relevant, concise, and easy to respond.

Sometimes one can even engage an individual on an ad hoc basis, such as at a convention, meeting, mall, or on the street. Such events or opportunities offer a fertile ground for conducting multiple customer interviews. When engaging customers in these settings, one should remember to ask whether they have five to ten minutes or whether the present time is suitable for an interview. People are busy going from place to place; they feel rushed. Therefore, interviewers speed through these conversations to accommodate. If one can get the interview, remember to say within the promised time frame. Sometimes the interviewer can get five minutes to gather some insights, but not a full interview. Hence, if further discussion is needed, one should ask about a follow-up time for a little more detailed conversation.

Conducting the Interview

The interview consists of several parts− a beginning, middle, and end. Ideally, one should conduct it in person to watch for non-verbal signals in response to the questions asked and to ensure that the interviewee is not distracted. If not live, then the interviewer should try to use some Internet-based platform to allow for a face-to-face exchange. Useful platforms include Skype, Zoom, or Facetime. Zoom is particularly attractive because the interviewer can record the interview to the cloud and receive in a few hours MP4 and transcript files [7].

The interview should involve one interviewee at a time. This approach prevents “Groupthink” or the overt influence by vocal or more senior individuals within a group. The individual interview approach allows for a diversity of ideas and opinions and customization of the interview to each interview, both of which can lead to better results in less time for each interview.

When starting off the interview, one should thank the individual and frame the purpose of the conversation. This approach helps the interviewer to build rapport and establish a frame of reference for the discussion. In the beginning, the interviewer might start with simple screening and “Softball” questions to keep it human. The interviewer should focus on the first two “FUNNELS,” the target audience and the individual’s attitudes concerning the topic area, to determine whether one has the right customer and to build rapport.

As the interview progresses, the interviewer will want to expand to more significant questions. One should use the middle “FUNNELS” to delve into interviewee’s experiences, “Pain Points,” “Job-to-Do,” and solutions that the interviewee. During this portion, the interviewer might ask the interviewee to tell a story, propose solutions, and identify priorities. As the conversation progresses, the interviewer should explore the interviewee’s experience, efforts to address the problem, and requirements for a potential solution. Great questions might include the following: (1) Tell me about the last time it happened?; (2) How large of a pain is this one?; (3) How has the interviewee addressed the problem; and (4) What would the solution look like if the individual had a “Magic Wand,” blank piece of paper, or an iPhone wireframe.

During the interview, one should try to make it conversational. Griff Constable, the author of Talking with Humans, points out that this effort is critical to disarming the interviewee [6]. Knowing the topic helps the interviewer build credibility with the interviewee and make the interview more of a conversation; one is not just asking questions, but instead is conversing back and forth with the customer. As a result, the interviewee begins to feel more comfortable, lets one’s guard down, forgets about time, and enjoys the exchange.

As the discussion progresses along, the interviewer needs to listen actively. In doing so, one observes the interviewee’s body language and facial expressions, acknowledges points, urges for expansion of points, asks questions, and summarizes comments. Concerning body language, the interviewer should note when the interviewee is nodding or leaning in.

The interviewer should be empathetic and display the ability to understand and share the feelings of another. In essence, empathy allows the interviewer to step into the interviewee’s shoes and view things from another person’s point of view. It is critical for the entrepreneur who is the interviewer to step away from oneself and one’s product. Instead, one should dive into the interviewee’s life and challenges.

As part of an empathetic or “Painful” interview, one should respect the interviewee’s time. Thus, the interviewer needs to be watching the time (or have a partner who is doing so) to respect the interviewee’s time and to pace the interview. One can make the most of the time by planning out what key “FUNNELS” or questions to explore with the interviewee. To optimize the interview, the interviewer should not be talking too much, but instead framing and asking questions, rather than trying to “Pitch” the product (such efforts turn off the interviewee and result in misleading information). The interviewer can monitor time by using either the timer in one’s watch or smartphone.

During the interview, some individuals will go off on tangents, which can impact time. Sometimes the tangents provide useful insights. At other times, these responses may reflect the interviewee’s lack of clarity with the question or lack of desire to address the specific query. However, tangents can be challenging. The interviewer needs to work adeptly redirect the conversation towards the core questions by listening actively and diplomatically looking for places to segue back to the primary areas of discussion, while not being too brusque because such an action could shut down the interview. One can also use further follow up questions, called “Drill Downs,” to help discern the lack of desire or understanding, and redirect back to the primary flow of conversation. Another technique is to acknowledge the points as excellent insights to make a note of, thank the individual, and pivot back to the primary areas of discussion. One can also indicate that the interview can return to the topic a little later.

The interviewer successfully closes out the discussion by being mindful of a few essential wraps up actions. First, thank the interviewee. Second, try to summarize the critical takeaway points to confirm accuracy. Third, highlight any specific action points. Fourth, ask two crucial questions: (1) Is there any question I should have asked but did not?; and (2) Who else should I talk with (and could you extend the referral)? Both of these questions can lead to essential insights and additional references with a recommendation from a colleague. Finally, the interviewer should always send along a follow-up note to thank the interviewee and highlight any follow-up actions.

After the Interview: Documentation, Analysis, Application, and Communication

Following the interview, the interviewer needs to document, analyze, apply, and (5) communicate critical findings.

The first issue to address is that of taking notes. To aid documentation, one can utilize one of three strategies to take notes. First, if the conversation involves a live face-to-face interaction, the interviewer can record the interview with a digital audio recorder or on one’s smartphone. This approach is the “Gold standard.” It allows for a complete record for further review. The interviewer can take the digital file and upload it to a transcription service, such as Rev.com, which can turn around the notes in 24 hours at $1 per minute transcribed [8]. There are also several apps that the interviewer can use as well. If the interview is via the web, then check to see if one’s platform will record and transcribe. For example, with Zoom, an interviewer can record the engagement to the cloud and obtain a transcript along with the digital audiovisual file within a few hours [7]. However, if one is going to record the meeting, remember to obtain permission. In some states, such as in California, the taping an interview requires that the interviewer obtains the consent from the interviewee to record the engagement. Finally, make sure one’s recorder works and had adequate storage space and battery life.

The second technique involves taking notes. It is a useful approach, especially with ad hoc interviews. Taking notes helps with active listening. However, one needs to be aware of the bias of the interviewer in interpreting responses, and having to multitask can limit and bias this effort [2].

The third technique involves having two people. Many entrepreneurship educators encourage this approach, and McKinsey Consulting uses it [9]. It overcomes the previously mentioned interviewer’s bias, interpretation, and multitasking issues. It is helpful when one cannot record the interview as well. In this case, the notetaker can take more detailed notes, allow for an additional perspective, and make observations concerning the interviewee’s nonverbal responses. Still, with notetaking by a second individual, the potential for interpretation bias does exist [2]. Also, interviews with two people can sometimes intimidate and influence the interviewee’s responses.

An interviewer should document the interview quickly while it is still fresh in one’s mind. It would be ideal for fashioning a summary within a day. One approach is to write up a summary with headlines and bullet points, which McKinsey Consulting uses [9]. It contains an executive summary, which is followed by the expansion of each major bullet area. Generally, these summaries range from one and a half to three. The interviewer can then share the results rapidly with stakeholders, clients, and teammates for analytic and action purposes. Another pragmatic approach is to lay out the critical “FUNNEL” areas, put bullet responses under each, then tie in what learned and how applied (e.g., updating a Business Model Canvas [BMC] or Value Proposition Canvas [VPC]. A final approach is to use an interview summary template (Figure 2a) covering the functional, emotional, and social “Pains” and “Jobs-to-Do” and a debrief template (Figure 2b) to translate the interview learnings into a customer story.

(a) Interview Summary Worksheet Template

Once  aninterviewer conducts several interviews, he/she can summarize the findings in a table (Figure 3). This table condenses critical pieces of information− number of conversations, customer personas, critical learnings, applications to the BMC and VPC, and pivots (and why). This process helps to inform insights, applications, and decisions to make.

Another way to make sense of the data is to put multiple interviews into an Excel spreadsheet, code the response, and perform a keyword analysis. This approach becomes more useful when one collects data from forty to fifty interviews, or when one is participating in the National Science Foundation Innovation CORPS™ program where the team needs to conduct 100 or more engagements [12]. Once the interviewer inputs the data, one can then turn the qualitative information into something more quantitative. This effort can lead to more data visualization via figures or word clouds. It allows for detecting patterns or trends within the data, which the interviewer can pursue further with additional meetings or through surveys.

However, it is essential to watch for false positives within the data. Thus, any analysis should employ questions such as “So what” and techniques, such as the “Truth curve,” to identify what is relevant and truthful, and what that might be misleading. This view is essential because interviewees might inform the interviewer of one behavior, but instead, when observed, display actions that reflect something completely different. Hence, one needs to view such data with the requirement to conduct further interviews, tests, and experiments based on learnings gained from customer discovery.

The Power of Questions

In conducting interviews, one needs to have a set of tools. Questions are these tools. In considering who are effective interviewers, at the top of the list are attorneys, consultants, journalists, physicians, and salespeople. One of the critical attributes of individuals who are successful in those occupations is that they “think in questions.”

Why is that important to think in questions. In the Harvard Business Review article by Brooks and John, “The Surprising Power of Questions,” the authors highlight several benefits [13]. First, questions spur learning and idea exchange. Second, it fuels innovation and better performance. Third, they build trust among team members and for it mitigates risk by uncovering unforeseen pitfalls and hazard. The authors also point out the need to ask the right question. It is not about one’s company and its product. For example, as Constable illustrates in his book, Talking to Humans, an example where an individual is interviewing a customer, and rather than exploring the customer’s needs, the interviewer is asking about the product and packaging [6].

The Fundamentals

When thinking about questioning fundamentals, one needs to start with the “Journalist’s Six”− Who, What, When, Where, How, and Why? The strength of the “Journalist’s Six”− is that these questions provide a flexible array of areas for the interviewer to probe the interviewee to draw out the facts that underly a story.

Of the “Journalist’s Six,” the question “Why” is the most important. It is considered to be the “Power Question.” “Why” is essential because it helps to define motivation. It also can define the “Root Cause” of a problem or “Pain” by using the “Five Whys” technique [14].

The next area focuses on open-ended versus closed-ended questions. Open-ended queries are very critical because they allow for the interviewee to elaborate and provide greater insight. A good list of general open-ended questions include the following queries: (1) What would happen?; (2) I wonder if?; (3) What would you think about that?; (4) in what way?; (5) Tell me about X?; (6) What would you do?; (7) How can we?; (8) How did you in what way?; and (9) What do you suppose? However, the “Watch Out” with open-ended questions is for the interviewee to avoid the question or move on to another issue (i.e., “Going Off the Reservation”).

In contrast, closed-ended questions are generally not ideal. Yes/no types of questions are very limiting. When using this type, however, add on a “Why” to avoid a terse and limiting response. Sometimes, there is a value using the yes/no to confirm the position. This approach is accurate and what it does is to verify what the interviewer has heard. For this type of yes/no question, active listening is essential. So, repeat what one hears to clarify and confirm the response.

As an alternative to using yes/no types of questions, one can change to a graded scale to understand better the rationale and degree of adherence to the belief. For rating, use either a one to seven scaled or a “Hot,” “Warm,” or “Cold” response. Another technique is to use a rank order response. These types of questions can also let an individual obtain both quantitative as well as qualitative information. For example, is the issue related to where something fits into the “Top Five” pains to gauge motivation and to quantify the problem.

The next area of interview questions involves “Drill Downs.” These types of questions involve when an interviewer starts with an initial question and then pursues with a second question based on what one hears from the interviewee. This approach, however, involves active listening to be effective. It provides a path to add value to one’s customer insights. One can guide the customer for clarity as well. Example “Drill Down” questions can include: (1) Tell me more about that?; (2) How did you do that what works?; and (3) Why or tell me that the last time that happened?”

Questioning Techniques

Several additional questioning techniques build on the fundaments. The first one is  the “Boomerang.” This approach is when one receives a question and responds with a question. Thus, it functions similar to a “Boomerang” that has been thrown out and returns. For example, when getting a tough question, the interviewer can respond with, “Oh, that is an interesting question, but could you please explain the rationale for asking about this issue.” This approach changes the control of the exchange and offers the opportunity to obtain valuable insights.

A second approach is getting input from the interviewee. For instance, one can ask whether one would offer an opinion. This approach provides an excellent strategy to put the respondent on a pedestal since the interviewer is asking directly for a particular perspective on a topic.

A third strategy is the empathy card. For example, if one were to step into someone else's shoes, then the individual might wish to query about: (1) How would one feel about this issue?; and (2) How one would respond to the other approach? The interviewee's response could provide a glimpse to high-value propositions. An interviewer could also use this strategy with an interviewee to have the individual project one’s reactions as if one were another type of customer. For example, if the interviewer first asked for a response from an interviewee as a particular type of medical specialist, one could then follow up with the question as to how the interviewee would respond if one were a generalist. The responses could be quite different.

The final strategy involves other ways to understand an interviewee’s rationale for a particular perspective or response. For instance, one could use the query, please help “Me” (the interviewer) understand the approach involved here. Another example might involve the response, “I” (the interviewer) am a little confused here in terms of what “I” (the interviewer) have heard, could “You” (the interviewee) clarify this issue or your response for “Me” (the interviewer). In many ways, this approach is just another way of asking “Why.”

Questioning Biases

Biases exist in both the conduct of the interview and the interpretation of the interviewee’s responses. Several reports highlight this problem and related concerns [2,15,16]. Grace Ng, the co-Founder of Javelin.com, a software and services company that implements lean startup, reinforces these concerns in her 2014 Forbes piece [15]. She highlights that asking the wrong questions one of the five common pitfalls into which startup falls. She explains that these practices exist because the startup focuses its discovery phase on selling the product, instead of exploring current customer behaviors and insights. Ng points out several mistakes including that of leading questions, selling too soon, talking too much, and not digging deeper when insights surface.

Interestingly, York and Danes highlight in their Journal of Small Business review article that entrepreneur biases and heuristics exert significant risk in the customer discovery process [16]. They are significant watch-outs to prevent the interviewer from finding the truth. In their paper, they note that entrepreneurs, constrained by time and expenses, fail to obtain critical information available for making a proper decision. They continue on to observe that many entrepreneurial activities rely on a subjective view and limited data.

Furthermore, they identify several biases and heuristics that carry significant risk to the discovery or customer development process. The first involves that of selection (i.e., friends and family), which involves the identification of individuals close to the entrepreneur and give the expected answer. The second involves that of confirmation (i.e., leading, establishing, closed-ended questions). This bias involves the setup of questions that confirm one’s assumptions. The third is that of overconfidence (i.e., overestimation of knowledge, skills, and precision of one’s data). The individual has blinders (i.e., does not consider a 95% confidence interval) and does not consider other options (e.g., devil’s advocates). The fourth bias is optimism (i.e., unlikeliness to experience failure). Because entrepreneurs tend to be overly optimistic, they fall into “Sales Mode,” rather than sitting back to explore for and analyze responses for the truth. The fifth involves that of representativeness (i.e., a generalization of findings from small samples). The final bias is that of acquiescence (i.e., respondents offering answers they think the entrepreneur wants to hear rather than their genuine opinions). Entrepreneurs can get their way as they “Pound the Customer” to get the answer for which they seek. In many ways, acquiescence can be similar to confirmation bias.

In a related paper, Chen and colleagues examine the cognitive biases that can influence the entrepreneur’s judgment of product/market fit [2]. The authors highlight several of these biases when adopting the “Get-Out-of-the-Building” approach− (1) for face-to-face interviews involving saliency, vividness, and inappropriate cues; or analogy (2) for sequential interactions involving recency and primacy, and the contrast effect; (3) for large samples, overconfidence, redundancy, and dilution; and (4) for the entrepreneur, biased processing. Chen and colleagues note that the essential point of the interview is to explore, not validate. They confirm York and Danes point that entrepreneurs who conduct their interviews bring in biases in their questions, interpretation of the responses, and analysis of the data. Chen and colleagues add that while preferences are difficult to eliminate, they suggest that techniques could be employed to reduce the effect of such predispositions when interviewing customers.

Related to York and Danes’ confirmation bias is that of framing. Tversky and Kahneman highlight this issue in their timeless piece, “The Framing of Decisions and the Psychology of Choice” [17]. In this paper, they discuss an experiment in which the investigators offer the participants two sets of choices (four options) related to the treatment of an outbreak of an Asian disease and estimates of the outcome (Figure 4). While the results are the same with each option, the characterization of each choice and its comparison to another option can drive responses to one specific response within pairs and even within all four options where compared.

For choice set 1, most individuals chose Program A because it was definitive with a positive outcome. In contrast, for choice set 2, the majority of participants preferred Program D because it presents a more a definite possibility of a negative result. Finally, when putting all choices together, individuals selected Program A because it was a defined positive outcome.

Thus, interviewers can drive responses based on how an interviewer sets up and presents one’s questions and answer. Both pollsters in politics and advertisers display this bias, which tends to drive misleading data. For example, when presented with two different presentations of the same frozen yogurt, which would one choose 20% fat or 80% fat-free (fat-free always resonates)?

Another framing bias involves the fallacy of anchoring and the appeal to authority. Concerning anchoring, interviewers can drive responses by the reference points they set. For example, an interviewer (or a survey) can obtain higher numeric answers when one anchors them off the lower limit of a higher range versus one selected if the array was at a lesser point [18]. Concerning the appeal to authority or obedience to authority, use of an authoritative source or influencer to frame up the question could bias an interviewee’s response [19-21]. For example, according to the New York Times, Drug A is the most promising cancer treatment; therefore, does one think so? Attorneys use this technique in court; it is called “Leading the Witness.” Thus, such efforts set up the answer and can inject bias and, ultimately, drive responses that are misleading and inaccurate.

Using Fitzpatrick’s The Mom Test to Highlight Ineffective and Effective Questions

Rob Fitzpatrick in his book, The Mom Test, identifies such biases and question pitfalls [5]. The essence of the book is for the interviewer is to engage the customer to talk about one’s life instead of the entrepreneur’s idea. He encourages entrepreneurs to ask interviewees about specifics in the past instead of generics or opinions about the future, which is another common mistake. He adds that entrepreneurs need to listen more to identify customer needs, deflect compliments, anchor fluff, and dig beneath ideas and insights to get at the truth. He also recommends “Googling for Trivia” to allow for more robust queries and allow for more effective preparation.

Fitzpatrick highlights a list of helpful and inadequate questions (Table 1) [5]. His discussion offers suggestions to improve inappropriate or weak questions. Of these, the most concerning are that of “Pitching Types” of questions, which he notes the entrepreneur should reserve for later in the discovery process once one has identified the customer’s needs and developed a minimum viable product or MVP. He points out the familiarity bias, which York and Danes highlight, through the book’s title since most individuals understand that moms (along with dads and friends) will reinforce the one’s view of the product is terrific [5]. His point concerning “Pitching” ties into York and Danes bias of acquiescence [16]. Fitzpatrick advocates that interviewers should deflect compliments and anchor fluff, as these type responses are false positives or reactions to get out of the interview. He also includes in the list of more imperfect questions are those related to willingness to pay and pricing. These are detrimental when the entrepreneur is in the customer discovery phase.