Asking Users: Questionnaires - Preece, Rogers, & Sharp - ePortofolio Chunk #10

March 21, 2011 - Preece, Rogers, & Sharp, Chapter 13 - focus on "Asking users: Questionnaires"

In our product evaluation, we are planning to use observation, usability testing,  and a questionnaire, which, according to Preece et al. (2002) "can be used on their own or in conjunction with other methods to clarify or deepen understandings" (p. 398).  Preece et al. offer a number of guidelines for designing questionnaires, many of which we have adopted, including:

• avoiding complex multiple questions
• making questions clear and specific
• beginning with general questions, followed by specific questions
• providing clear instructions
• using consistent rating scales
• aiming for brevity over length to encourage completion

Nielsen (2004) adds to this, suggesting that in order to ensure high response rates and avoid misleading results, you should "keep your surveys short and ensure that your questions are well written and easy to answer."  

He adds, "the highest response rates come when surveys are quick and painless.  And the best way to reduce users' time and suffering is to reduce the number of questions" (2004).  Another good point that Nielsen makes is that an interactive questionnaire is a user interface - which in and of itself should be "designed on the basis of user testing and follow standard usability guidelines."  

One of Nielsen's points that really resonated with me is the idea of "survey bloat" - particularly when working in a group, there is perhaps a tendency to try and satisfy everyone's wants - each person coming at it from a different perspective.  I imagine that a design team would be similar, which serves to reinforce the importance of knowing exactly what it is you are evaluating for - you can't evaluate everything at one time!  As Nielsen notes, "please resist the temptation to collect all the information that anybody could ever want.  You will end up with no information (or misleading information) instead." 

Related to this is the 'Query Effect", also referred to by Nielsen (2010)  - "whenever you do ask users for their opinions, watch out for the query effect: People can make up an opinion about anything, and they'll do so if asked".   The lesson for our group?  Be careful what we ask - make sure it is information that we want to have and that matters in our design. 

Norman Chapter 5 - The Role of Automation

I add this cartoon as it reminds me of myself as I text messages to my niece on my iPhone.  The auto-correct feature on the phone sometimes make me wonder.  I've sent nonsense messages such as, "How r u horsy", "Xhetr", "do you have docks on?", and "you all teddy?"  As Norman notes, we need automation for a number of purposes, such as eliminating the dull, the dangerous, and the dirty.  To this list he adds "to simplify a complex task, to reduce the work force, to entertain -- or simply because it can be done" (p. 117).  I suppose the auto correct on my phone is intended to make the task easier, however I spend more time correcting the corrections than I do actually sending messages!

I appreciated Norman's example of the automatic coffee machine - I have one and I love it.  In my case, I am able to enjoy my daily lattes (well, actually 2) at the push of a button, and leave the maintenance to someone else - best of both worlds!  However, in addition to other problems with automation that Norman notes, what about when things break down?  Just this morning, as I was attempting to froth my milk, the frother stopped working.  Before long I could smell that unpleasant odour that machines emit when they are working too hard or are about to blow.  Automation is great, but with no understanding of what is going on inside that machine, I am now at a loss as to what I will do for my morning coffee.

Norman notes two different directions in which research on smart things can move:  "one is toward intelligent autonomy, systems that attempt to infer the intentions of people" - as in the 'smart home', and the other is toward intelligent augmentation, providing useful tools but letting people decide when and where they are to be used."  (p. 130). At first glance, I lean towards augmentative tools, as I prefer to make my own decisions about activity.  Perhaps this is only because I haven't experienced much intelligent autonomy and don't really have a sense of what it is like.  Another reason, as no doubt my mother would note, is a fierce independence and a desire to be in control (I admit it).  Nevertheless, the examples of autonomous devices that Norman gives seem justifiable and even preferable to the alternatives - such as in search-and-rescue missions in dangerous situations.  I chuckled at the baggage-handling system example that clearly illustrates why sometimes you "need real people!"

This chapter also reminds me of J.D. Robb's books featuring Eve Dallas (yes, I admit, I have read some of these trashy novels - but only to learn about the design of future things!) - a futuristic detective, living in 2060.  In Robb's books, automation is fully functional in many aspects of daily life - cars fly and drive themselves, autochefs prepare favourite meals, lights respond to voice commands, as do showers.  It certainly will be interesting to see what the future brings!