Charline's Icon Usability Study

On Friday at the GNOME London UX Hackfest, Charline from Canonical gave us some details on an icon usability study she had run recently for the Launchpad icon set. Here’s my notes from the session:

Methodology

• The study was done as a surveymonkey.com survey.
• The study was for Launchpad, so a link to the survey was posted to Launchpad’s blog to attract Launchpad users. After 3 days, the survey had gathered 125 respondents.
• The icons were presented in context, since the context would inform the user’s interpretation of the icon in real usage
• Then users were asked to help interpret what each icon meant
  • First question: “this icon means….” and asked the user to fill out, free-form
  • Second question: “i have the following percentage of confidence in my answer” so we can tell how much of a guess it was on the user’s part or how sure they were of their interpretation.
  • Third: “when do you expect to find this icon?”
• Then, users were asked to provide a second / alternate interpretation of the icon, filling out the same three fields for it: this icon means… percentage certainty/confidence…. where do you expect to find

Lessons Learned

• You can’t do more than 5 or 6 icons per survey. It’s too much work for the respondents… the rate of people who don’t complete the survey grows high if you ask too much of them.
• After only 3 days, the survey had 125 respondents – a very fast response rate! Posting to a blog that users follow was an effective way to get respondents in this case.
• We can start to understand how people read icons and see some design rules and guidelines if you analyze enough icon interpretations. For example, with the edit icon, most users realized that it meant edit, but they also gave it a strong association with attention / warning / danger. Very different meanings…. a recommendation would be to change the color or shape so it wasn’t yellow and didn’t have an ‘!’-like design in it.