Remote Research. Tony Tulathimutte

the moderator. You want to encourage observers to contribute helpful observations and advice to the moderator during the session, but advise them not to flood the moderator with distracting messages and have them generally defer to the moderator’s judgment in matters of running the session.

      This letter assumes you’re using Adobe Connect, but other variations can be found on our Web site at http://remoteusability.com.

      ---------------------------------------

      Hi everyone,

      This email contains detailed instructions for how to observe the remote user research sessions, which will begin Next week.

      [How to Set Up Adobe Connect]

      For most of the screen sharing with our participants, we’ll be using a Web service called Adobe Connect. You don’t have to install anything now, but you will have to follow certain steps before each individual session. Here’s how you’ll be using it:

      1 Go to the Web address at the beginning of the test session: connect.acrobat.com/XXXXX

      2 Enter as a guest. Use a neutral guest name like “usability” or “user testing” so that the participant isn’t distracted by seeing an unfamiliar name.

      3 Dial in to the conference number: (XXX) XXX-XXXX

      4 Mute your phone! This is very important; otherwise, the user will be able to hear you, which will disrupt the session.

      [Other Things to Note]

      —Recruiting and testing will be conducted between the hours of Noon EST (9 AM PST) and 8 PM EST (5 PM PST), with a brief break around 3 PM EST so the researchers can get some lunch. Please be aware that for setup and recruiting processes, there may be several minutes of downtime between each session, so feel free to have something to work on while you wait for a new session to begin.

      —Phone headsets are encouraged so that you don’t have to hold the phone with your shoulder all day.

      —I can be reached on AOL Instant Messager at “aimscreenname.” To keep from swamping me with requests from multiple people at once, please designate one person who will direct any questions and comments to me as the sessions are ongoing. Feel free to point out any and all issues you think are important. I may not be able to handle all your requests, but I’ll try my best!

      —If you have any other questions about how to follow along with the sessions or communicate with me during the sessions, feel free to email me.

      That’s all! Looking forward to getting this study started!

      Note

      Researchers and Stakeholders

      In this book we occasionally mention “researchers” and “stakeholders” because often-times the people conducting the research (“researchers”) are doing it on behalf of other people who have commissioned the research (“stakeholders”). Stakeholders—who can include business executives, managers who dictate the research budget, and so on—are usually untrained in research methods (never mind remote research methods), but they’re essential for defining the parameters of the study and should be involved in all steps of the testing. Of course, there are also situations in which the researchers are the stakeholders—e.g., academic research or companies in which the developers also do their own user testing. So, just to be clear: there aren’t always business stakeholders, and in those cases you can ignore the whole researcher/stakeholder dichotomy and the issues of “sign-off and approval” we occasionally bring up.

Chapter Summary

      There’s always the chance you may have to go back and adjust something in the project goals or adjust the schedule for whatever reason; however, you can consider yourself pretty much finished with the setup phase and ready to move on to recruiting once you’ve completed the following:

       Selected, installed, and familiarized yourself with a screen sharing tool (and recording tool, if necessary).

       Set up your equipment: computer, high-speed wired Internet connection, monitor(s), two-line desk phone, phone headset, and any other necessary equipment.

       Arranged backup screen sharing tools, as well as any optional tools (recording, note taking, etc.).

       Completed standard user research project management steps (check out Chapter 5 of Mike Kuniavsky’s Observing the User Experience).

       Completed the recruiting screener design and recruiting test (covered in the next chapter).

       Drafted and familiarized yourself with the facilitator guide.

       Briefed observers on how to participate during testing days.

       Conducted a practice run of the test, to make sure everything’s working (especially if it’s your first time).

      Got all that? Then you’re ready to start testing!

[1] Chapter 5 covers the finer points of moderating in detail; for more details on traditional moderating techniques, see Joseph Dumas and Beth Loring’s Moderating Usability Tests: Principles and Practices for Interacting.

      Chapter 3

      Recruiting for Remote Studies

       What’s “Live Recruiting”?

       Live Recruiting Using Forms and Pop-ups

       Designing the Screener

       Paying Remote Recruits

       Choosing Good Users and Spotting the Fakers

       Recruiting Slow? Don’t Sit Around—Fix the Screener

       Wait! Read On…

       Chapter Summary

      Recruiting research participants is notoriously frustrating and easy to mess up. If you do mess it up, you risk blowing the validity of the study, and even if you don’t mess it up, it can still be a big drain on time and money. Many researchers are glad to hand over recruiting to a professional third-party recruiting agency, paying anywhere from $100 to $800 per recruit (depending on the stringency of the recruiting criteria). Others rely on in-house email lists, academic volunteer pools, paid participant panels, or personal contacts, each of which may not provide a representative or unbiased sample, for a number of reasons. And worst of all are online classified ads like those on craigslist, where the recruits are usually biased or, at best, solely interested (as opposed to mostly interested) in collecting an incentive check.

      But then there’s the rest of the Web: a huge pool of anonymous, disinterested, ordinary people who wouldn’t necessarily consider participating in a research study, much less join a panel of standby participants—in other words, very promising research participants. This section will teach you how to recruit those people, confirm that they’re qualified, contact them, and convince them to take 40 minutes out of their day to participate in a research study, all in a reliable, ethical, and nonirritating way.

      Note that you can use these methods to recruit for any kind of study, whether moderated or automated, in-person or remote, live or scheduled. And we’ll also explain why doing it this way is worthwhile.

What’s “Live Recruiting”?

      Live recruiting is using your Web site to collect voluntarily submitted user info and then using that