The Official Fluid Blog

Jared Spool recently wrote that pretending and roleplay in usability testing is not as effective as recruiting people with an actual need or real interest in a product or activity.

We routinely ask participants in prototype testing to pretend they need to complete an action by creating a believable scenario for them (e.g., you just bought a new MP3 player and want to download some new music for it.) Jared Spool’s research has found that people who are not really interested in an activity behave differently on a web site than people who are genuinely interested in that activity.
What we need to be aware of (and what Jared Spool nearly alludes to) is that the bigger the difference between what people want to do and what we ask them to pretend to do, the greater the possibility that they will not behave accordingly. So, asking a Wall Street executive to pretend to be interested in The Jonas Brothers will probably be less accurate than asking female highschool students to pretend to be interested.

Jared also finds that asking people to pretend to shop for something they do not want or need is less accurate then asking them to shop for something they do want or need. At Fluid we sometimes find ourselves in this situation in prototype testing where not all of the participants actually need the products being sold on the site being testing, but there are a few things we can do to improve the accuracy and relevancy of our testing observations:

1. Recruit from real customers who already visit and shop on the site, because they are already interested in the content, products, and features,
2. Recruit for the appropriate demographics and interests in participants to improve the probability that even if they are not actual customers they will be still be interested in the content, products, and features of the site,
3. Recruit for specific needs (e.g., recruit people for customizing wedding invitations right at the time they are planning a wedding and need invitations.) Obviously this is more difficult, but it should produce more accurate observations.

One of the limitations we have with prototyping testing is that we cannot adapt the task to the participant after they arrive. Prototypes typically have a pre-defined set of tasks and available options, which means we need to be careful and recruit the appropriate participants who will match the prototype tasks rather than adapt the task to the participant.

If we were testing a fully functional and available web site, we would have more opportunity to adapt the tasks to the participant (e.g., ask someone to shop for a backpack rather than shop for a dress), but we still need to recruit participants who would be interested in the available range of products. If the only products a web site sells are shoes, then we cannot recruit for just any shopping need; we should try to recruit people who need and like to shop for shoes.

When we are preparing for prototype testing, we ask all of our clients to provide actual customer lists for participant recruitment so that we know our participants are at least interested in the content, features, and products of the sites and prototypes being tested. If we cannot recruit for a specific need, we are at least able to recruit for characteristics that make that need more likely to be present in participants.

This entry was posted on Monday, September 29th, 2008 at 3:41 pm and is filed under E-Commerce, Information Design, Interaction Design, Usability. You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site.