Validating Product Ideas. Tomer Sharon

up and then not use the product?

      • What are some use cases we should plan for?

      • Would people use the product?

      • How do people choose what to use among similar options?

      • Does the product solve a problem people care enough about?

      • Which customer needs does the product satisfy?

      • How do we define the right requirements for the product?

      • How will the product solve people’s pain points?

      • Which features should be included in an MVP?

      Clarification: MVP is an experiment that has a goal of learning as much as you can while making the smallest amount of effort. It’s a process, not an end goal. Therefore, although you might be asking yourself which features to include in an MVP, understanding needs and problems real people have through experience sampling (and other methods) puts you in a better position to come up with an experimentation process.

      • What are some feature ideas our customers have?

      • Which features are most important?

      • Should we build [specific feature]?

      • How often would people use the product?

      • Would people from other cultures use the product?

How to Answer the Question

      The following is a how-to guide that takes you step-by-step through the process of using experience sampling to answer the question “What do people need?”

STEP 1: Define the scope and phrase the experience sampling question.

      “What do people need?” is a very broad question that probably has an infinite number of answers. To make sure that your study results are effective and useful, first define a very clear scope for your inquiry. You probably have a domain in mind such as grocery shopping, photography, or enterprise sales. This means you will be focusing on this area during the study.

      Try to get even more specific. What is it about grocery shopping that you want to learn? What would help you make a decision about your photography-related app? What aspect of enterprise sales are you interested in? Be as specific as possible when you decide how to best use research. For example, maybe your study scope involves challenges in remembering what to buy at the grocery store, or opportunities for saving time spent on uploading photos from a camera to a desktop, or ways in which salespeople prepare for a sales pitch to a potential customer. After you define the scope, it’s time to work on the question you will ask your study participants.

      The specific question you ask experience sampling study participants must be carefully phrased and tested. As you phrase the question (or come up with a few alternatives), bounce it off other team members early on and run a pre-launch pilot before releasing it to the whole sample group. The most important thing is that the question has to speak to participants’ behavior that repeats itself. Remember, you are going to ask that question over and over, and you will expect different answers (both across the participant population and from the same individuals at different times) that provide insights into people’s needs, pain points, and delights.

      Here are some good examples of experience sampling questions that target repeated behaviors:

      • What was the reason you recently used a piece of paper to write something down?

      • What was the reason you recently updated your website?

      • What did you want to know recently?

      • What was the reason for the last phone call you initiated?

      • What were you frustrated about recently when you went grocery shopping?

      The experience sampling question must also be specific; otherwise, you are destined to get a lot of irrelevant answers. For example, if you ask, “The last time you went to a shoe store, what did you do?”—that’s too general and vague. People will tell you they met a friend and talked about their recent spring break vacation or called their mother. If you are interested in uncovering needs related to buying shoes, be specific about it. For example, ask, “The last time you went to a shoe store, what frustrated you the most about buying shoes?”

      The following are some questions to ask about behavior that is probably not so repetitive:

      • What annoyed you the most the last time you moved to a new house or apartment?

      • What was the primary reason for choosing your most recent vacation destination?

      • Which laptop computer did you buy recently?

      While the above questions are legitimate, good questions to ask, they are not recommended for experience sampling because the frequency of the behavior doesn’t make sense for this research format. If you are looking to uncover needs related to behaviors that are not repeated very frequently, consider interviewing (Chapter 2), observation (Chapter 3), or a diary study (Chapter 4).

      Make sure that the question you ask helps you figure out what users need. There are several types of questions you should avoid in an experience sampling study because they will never help you uncover needs:

      1 Questions about opinions: Asking for someone’s opinion about something several times a day or week is useless. Opinions don’t change five times a day or week, and there’s no point in asking for them that much. Examples might include:• What do you think about hiring a Web developer for updating your website?• Should links be blue or black?

      2 Questions that speak to “average” behavior: A common trap people who phrase questions for experience sampling fall into is asking them in a way that vaguely refers to a time frame about which the question is asked. For example, when you ask, “What frustrates you most about boarding a train?” you are setting yourself up for failure. Let’s assume the person who tries to answer this question has boarded trains 300 times during her lifetime, 50 of them in the past year, one last week. This person will probably not remember all of the frustrations about boarding a train and will try to come up with an answer that “averages” the ones she does remember (maybe the last five). She will also try to satisfy you with the answer and give you a real interesting one with a nice anecdote. The answer will probably not represent many real frustrations very well. The best way to avoid this trap is to ask about the last time the behavior happened. This way, her memory is still fresh, and it is less likely the participant will pick and choose an answer she thinks will satisfy you. Here are some examples for questions that ask people to “average” their behavior:• What frustrates you the most when you board a plane?• How do you spend time while you wait in a long line?

      3 Questions that are too general: While taking a broad approach to learning from people is usually a good thing to do, asking a general question in experience sampling is going to force you to deal with a lot of noise. If you are interested in one aspect of a topic, ask about that aspect rather than asking about the entire topic. For example, if you decided that the scope of your research was uncovering needs related to finding a parking spot, don’t ask, “What annoys you about driving in a city?” Instead, ask, “What was the most frustrating thing that happened to you the last time you were looking for a parking space?” Here are some more examples of questions that are too general:• What frustrated you recently?• How did you decide which smartphone to purchase?• What is email good for?

      4 Yes/no questions: Experience sampling is a research method that integrates