Doorbells, Danger, and Dead Batteries. Steve Portigal

Doug Cooke: Knock-Knock! Who’s There?

      In a recent research and strategy project focused on defining a new global platform for a medical device, our research plan required us to shadow clinicians and others as they used existing devices in the “context of care.” With issues like HIPAA² protecting patient privacy and other security issues at big urban hospitals in the U.S., our team decided that conducting research in Europe provided a better opportunity to understand these devices and their users.

      Planning started with all the usual steps: multi-day client sessions to assess the domain, issues, and problems; auditing reams of client data and documents; becoming familiar with competitive products, etc. We developed a research protocol that went through many rounds of revision with a large, multi-location client team arriving at a clear understanding of relevant and important user issues. We developed screening criteria for participating medical institutions. Pilot studies were run at United States hospitals. Months of preparation were spent in making sure our research team was fully prepared to bring back insights and perspectives that would help define the next generation global respirator platform. Ready, set, on to Europe!

      Our first stop was at a hospital in Wales. They had lined up the appropriate people for us to shadow and interview, including department heads, physicians, and medical techs. We spent two days shadowing, probing, and gathering, and everything worked according to plan. Wahoo!

      At our second stop in London (hauling two large model cases that would not fit into London’s very spacious cabs), we arrived at the check-in desk and asked to see Dr. Smith (or so we’ll call him). Upon arrival at his department wing, we learned that Dr. Smith was not in. Even more concerning was that Dr. Smith was out of the country at a conference and had not let anyone else know we were coming. After speaking with a few more people, the answer was “Please come back at another time when the doctor is in.” Ouch! In spite of all the planning, effort, and resources to get here, a few uncooperative people were about to jeopardize our research program.

      WIKIPEDIA

      How could this happen? Well, I ignored one of my primary rules: never let the client take on a critical path item that could endanger the project’s success and my firm’s reputation. Specifically, because of the difficulty of gaining access to the right people and institutions, and extremely high costs if we were to use a traditional recruiting process, our client took on the responsibility of arranging our visits to hospitals throughout Europe. Few clients understand the level of effort needed to screen, schedule, and triple-confirm each participant. When the “research gig” is complex and requires the participation of a number of people carefully choreographed in a short time, it is essential to have a dedicated, experienced resource to make that happen.

      We made it all work in the end. With no Dr. Smith and an apparent dead-end, we literally started on-the-spot networking, walking up and introducing ourselves to doctor after doctor until we had made some friends who would grant us two days of access in the ICU and ER. It worked out in the end, but presented unforeseen delays and stress to an already pressure-filled project. Painful, but constructive outcomes, nonetheless.

      The rest of the trip in Germany and Italy presented various levels of preparedness on the part of the hospitals we visited. Some hospitals were planning on hosting us for our full two-day itinerary, and some were expecting only a few hours’ meeting (which we were able to extend by turning on our best charm).

      I have always been a very careful planner and can fastidiously orchestrate research logistics. I know what it takes to gather user insights. But the lessons learned from this European research foray are a clear reminder that whenever I can, I must control the recruiting and scheduling process. I hope to never again knock on any unsuspecting doors.

Daria Loi: Researcher Thresholds

      Several years ago, a colleague and I traveled to Sweden, Indonesia, and China for a study of storage practices in homes. We were particularly interested in observing everyday activities related to the “stuff” one owns, like clustering, archiving, organizing, disposing, sharing, holding, recycling, and so on. The goal was to gather useful insights from the analog world to better understand how people might deal with data in the digital one.

      In each city, we recruited a number of participants to be interviewed twice and to complete a cultural probe during the week between the interviews. The first interview (about three hours) started by focusing on baseline data for the first 60–90 minutes, and then shifted to a home tour in which we would go room by room, observing the environment and asking questions arising from what we noticed or from what the participant indicated during the baseline interview. During this part of the first interview, we would often find ourselves opening drawers, cupboards, wardrobes, and the like, with participants’ permission, of course.

      There is nothing more fascinating than seeing what people do with their stuff. To some extent, you see yourself and your own behaviors in action, and in other cases, you have to be prepared to find the most obscure things in those drawers—so obscure that even their owners are perplexed when they rediscover them!

      I have fond memories of a young and bright Swedish woman laughing with puzzled surprise when she discovered the enormous amount of candles she managed to accumulate and that all those candles were in the same drawer as a flyswatter she did not recall owning. I still giggle when I think of a beautiful Indonesian family taking us in their storage room, to discover they had six to seven identical broken appliances. I still remember the puzzlement of the husband, trying to work out how on earth that accumulation happened. And again, I always smile with affection and admiration when I think of a Chinese painter and his lovely wife showing us their feng-shui based order of things.

      During this quite long study (a bit more than two weeks in the field for each country, long for corporate research), one of our many adventures was about a Chengdu-based participant, often code-named “the interviewee from hell.”

      It all started in the morning at 9 a.m.—the first interview for that day. We rang the bell of an apartment, but no response. After a few minutes, we tried again, but still nothing. We started feeling edgy, as we did not want to be culturally inappropriate or pushy. Yet suddenly the door opened to reveal a young pajama-clad woman with puffy eyes who was evidently just out of bed.

      The young lady, who here will remain unnamed, looked at us evidently annoyed, flashing “how dare you wake me up” eyes, and asked us what we wanted. The translator explained that we were there for the interview, and she told us she was pretty sure we were one day early.

      My colleague and I began thinking of ways to accommodate her interview another day, but the participant let us in—even though we feared this was not the best premise for the best interview.

      After the usual preambles and consent form sign-offs, we set up our video gear and proceeded with the first part of the interview. I should have immediately realized something was off when I saw the participant clutching her mobile phone with great intimacy—the glued-on-my-body type of intimacy. But no, her behavior did not immediately ring the “this is going to be a disaster” bell, and we started with the interview, with me taking charge of picture/video taking activities.

      There is something rather cool about framing another human being through a camera. You observe little details even more deeply. And now, all the little details immediately rang the infamous “this is going to be a disaster interview” bell. For the rest of the interview, the following scenario occurred over and over again:

      • Colleague asked a question while participant checked her phone (text, emails, Internet).

      • Participant responded with “yes,” “no,” and “hmm . . . I think so” type of answers.

      • Our facial expressions were incredulous.

      • Participant continued checking her phone, rarely looking