- Nielsen, Alertbox (2010): Interviewing Users [URL]
As we have planned on both observing users and interviewing users (actually administering a questionnaire), I am focusing on the Preece et al. and Nielsen readings this week.
Nielsen:
"What users say and what they do are different"
Nielsen points out that the critical failure of user interviews is that "you're asking people to either remember past use or speculate on future use of a system. Both types of responses are extraordinarily weak and often misleading."
He goes on to suggest that "dilemmas relating to specific UI elements can be resolved only by watching users interact with a design that implements a specific solution so that you can see how well it works in real use."
I wonder if the same holds true for questionnaires? For example, in our evaluation stage, we are planning to give users a questionnaire after engaging with our parallel designs. Questions that I wonder about include:
-Will users remember the designs clearly enough to be able to provide accurate responses?
-Will asking about the 'clarity of buttons or icons' provide us with any useful information?
Nielsen suggests "if you are compelled to ask users how much they like your features, then be sure to include a few non-existent features to collect a baseline" - will we have done this with our questionnaire? Is this something that we should be doing with our questionnaire?
I am reassured by Nielsen's statement that "interviews are also useful when you want to explore users' general attitudes or how they think about a problem" - which is part what we would like to learn.
"Beware the Query Effect" - Ha! hadn't heard of this before, but it makes sense. When you ask users for their opinions, watch out for the query effect: "People can make up an opinion about anything, and they'll do so if asked" - which makes me think that we need to be quite careful and considerate about the kinds of questions we ask.
Nielsen gives a good example of the above:
"For example, if you quiz people about your visual design, you'll inevitably get comments about the colors, even if they're not particularly important to the users. On the other hand, if you hear people mention the colors (unprompted) while they're using the site, then there's probably something to consider. (Say, a comment like "Wow, this neon blue really hurts the eyes," or a more positive statement like "The ultramarine is nice and calming.")" ... something to watch for!
Food for thought:
"Each method has its strengths and weaknesses. Taking the best input from each method will give you a much richer understanding than you could gain from any one method alone. Also, by supplementing each method with a range of other approaches, you can triangulate the findings and guard against misleading outcomes."
And:
Interviews are great supplements to other usability methods. If you could do only one thing, I would always recommend user testing. But why limit yourself to one method?
With our evaluation, we are planning to do use user testing, observation, and survey/questionnaire - looks like good triangulation!
And just for Norman and Intelligent Design:
2. Preece, Rogers & Sharp, Ch 12 "Observing Users"
Preece et al. offer a number of useful suggestions to keep in mind while observing users. They state, "Observation involves watching and listening to users. Observing users interacting with software, even casual observing, can tell you an enormous amount about what they do, the context in which they do it, how well technology supports them, and what other support is needed (Preece et al. 2002, p. 359).
In our evaluation phase, observing users will play a key role in our data gathering. As noted by Preece et al., goals and questions should guide all evaluation studies. Knowing what we are evaluating and the information that we would like to gather is key in the process. I think that it is also important to be able to focus your goals - you cannot gather information on too many topics. By focusing our attention on key aspects, we will be able to gather more useful information that can be used to inform our design.
 |
| Add caption |
Once our data is gathered, it remains to tell what Preece et al. describe as a "convincing story". To quote, "Much of the power of analyzing descriptive data lies in being able to tell a convincing story, illustrated with powerful examples that help to confirm the main points and will be credible to the development team" (p. 380), which will be our intent as we analyze our notes and the responses from users.
No comments:
Post a Comment