Doorbells, Danger, and Dead Batteries. Steve Portigal

and on. After a while, we tried to send a subliminal message by asking whether she would prefer to meet another time since she seemed busy (read: distracted and totally unengaged). The young lady looked at us (finally!) and said, “No, it’s fine. Let’s do it now.” (Read: this is tedious already; you interrupted my beauty sleep, so let’s get it over and done with it.)

      So . . . we went on. After a while, I started having the giggles, intrigued by the evidently dysfunctional situation. I felt tempted to suggest we refocus the interview on her mobile phone usage (evidently her passion), to learn something useful instead of pushing a cart into such a void-of-usefulness corner. But I didn’t suggest it. Instead, I kept going. Maybe I was in shock. Maybe I was so entertained by her behaviors that I wanted to see how far she could go. Maybe both. Hard to tell—these things happen fast, and it is often easier to think about the right thing to do retrospectively.

      Anyway, the rest of the interview continued along the same lines, with the exception of the home tour part, where my pictures were not of a user handling her phone while on a couch, but those of shoulders hunched over a phone. During the tour, the verbal part of the interview shifted from yes/no answers to a number of grunts and monosyllables.

      After three hours, we finally left. We looked at each other in total puzzlement, unsure if we should cry, laugh, or have a meltdown. We would have been totally entitled to have one. Instead, we kept our cool and moved on (aside from a few colorful words that I won’t put in writing). It was a testing-your-patience-thresholds kind of day.

      But the fun part was not over yet, because after a week, we had a second interview scheduled!

      This second interview was definitely much more colorful. Instead of taking pictures of a participant and her phone, I managed to take pictures of serious multitasking in action: for instance, send texts and check your social network on the phone with the left hand while checking the stock exchange on the laptop with the right hand. And do not forget the yes/no/grunt answers and minimal level of auditory attention paid to us.

      If you were to ask me what I learned from this participant, I would say lots.

      If you instead asked: Did you learn anything useful for your project? Not a thing would be my answer.

      Regardless, this experience led to a good story to share and helped me have a different appreciation for researchers’ thresholds.

Patricia Colley: The Hidden Persuader

      In 1984, I was 23 and working for a market and social research firm in San Antonio, Texas. They sent me down to McAllen to collect voter opinions on the upcoming national elections. McAllen is a sleepy little town near the bottom tip of the state, just a few miles from the Mexican border, mainly populated with low-to-moderate income Hispanic families.

      I was on my second day of door-to-door polling, asking voters their opinions on policy matters and their thoughts on the state and presidential candidates. I was getting a high rate of interview completions, with lots of useful data. After four years in market and social research, I was quite confident in my neutral, non-threatening “aw shucks, I’m just one of you” act, and its ability to deliver great results.

      But my confidence was shaken when I met Maria, a shy housewife in her early 30s.

      It was about 4 p.m. on a warm, dry Thursday afternoon when I knocked on the door of a modest, well-kept ranch house in a suburban section of McAllen. Maria opened the door partway. She was half-hiding behind it, sizing me up like a rabbit peering through tall grass at a coyote in the distance . . . curious, but poised to flee.

      I explained that I was gathering public opinions on the upcoming elections, and after she agreed, we began the interview.

      Me: “Now, thinking about (Candidate X), what comes to mind?”

      Maria: “Uhh, I don’t know? Is he a good guy?”

      Me (shrinking): “Well, I really don’t have any thoughts on (Candidate X). Besides, my bosses didn’t send me all this way to talk about my opinions. He wants to know your opinion.”

      Maria: “I don’t know. He seems OK?”

      Now, I didn’t think Maria was incapable of forming opinions. I suspect she had simply never been asked to share her thoughts about such important things. And she might never be asked again. But on this day, I was determined to make her opinion count.

      Me: “Well, you’ve heard of him, maybe seen him on TV?”

      Maria: “Yes.”

      Me: “So, what did you think of him? Is he someone you would vote for?”

      Maria: “Um . . . (pause)”

      Her eyes darted across my face, scanning every crease and twitch, searching for clues. Those big rabbit eyes begged mutely for help. I stared back, apologetically. I took a few slow breaths, trying to ground us both, so she might relax into talking more naturally. Each time she hesitated, I carefully repeated the question, altering the wording and inflection to make it sound as simple and benign as possible.

      Me: “Really, we’re just interested in what you think. Whatever you think is fine. Do you think you’ll vote for him, or not?”

      Maria: “Uh . . . yes?” (seeing no reaction from me) “No?”

      Me: “OK, that’s fine. Alright. Now, thinking about (Issue A), is that important to you? Do you think it’s good or bad?”

      Maria: “Uhh . . . I think it’s good?”

      The back and forth went on for several minutes. I tried to be neutral and free of any emotional expression, but my contortions only intensified the awkwardness. I was failing miserably to collect any genuine responses from Maria. A hot wave of panic washed over me. How can I get this back on track?

      I quit fighting it, and fell back on connecting with Maria as a person. I began riffing on her responses, affirming and adding detail to them. While trying not to reveal my personal opinions, I offered supportive words and gestures to elevate everything she said, so that she might open up and elaborate. Eventually, her answers flowed a bit more freely.

      Me: “So, what about the presidential candidates?”

      Maria: “I guess I’ll vote for (presidential candidate B).”

      Me: “Great! Is it because he is for (issue B)?”

      Maria: “Oh, that’s good. Yeah, (B) is good for us.”

      I felt I was way off book. It seemed impossible not to sway her answers. Well, at least she was talking. Finally, we got to the end. Walking back to my car, I breathed a huge sigh of relief. The hardest interview I’d ever done was over. I went out for a well-earned drink and a tragicomic debriefing with my co-workers.

      Sometimes you just get a dud subject, but something about that 15-minute exchange with Maria struck a deeper chord in me. As I drove out of town, troubling questions lingered. What is the value of a skewed interview? Was this the only time I’d failed to be impartial? Or had this been happening all along, in more subtle ways? How can I ever know that the data I’m collecting is pure?

      Maria taught me two important things that day.

      • People make stuff up as they go along. And we can’t always see the flaws in self-reporting.

      • The observer effect is unavoidable. Interviewers shade their work in unpredictable ways.

      I’m as diligent as ever about delivering valuable insight through my research. But ever since that incident in McAllen, I draw my conclusions with a fuzzy border, in humble deference to flawed inputs and shadow projections, on both sides of the clipboard.

Cordy Swope: A Crisis of Credibility

      IDEO. NYC. Early 2010.

      I