Charline's Empathy Usability Report

This morning Charline from Canonical presented a report on an empathy usability test she’s completed. These are rough notes split into topics:

Usability reports in general

• There was some discussion of the usage of user quotes in usability reports and their utility
• Audiences for test:
  • People who report bugs, developers
  • Some confusion over how testing on Ubuntu would have effected the test results – e.g. how did notify-osd behave to the user…
  • Usability folks
• Consider referencing products / screenshots that show a solution to the problems presented as inspiration
• When users fail to do a task, it would be useful to document the paths they tried / the mental model they had that failed. Even things they wanted to do that could never work in the current implementation – could be good ideas for a way the UI could work.
• Indicate how many users said something. If only one user suggested something, that should be clear from what all or most users said.
• We need help to get these filed as bugs. Would community members be willing to support usability by splitting reports like these into actual bug reports?

On-Board Experience

The on-board experience in empathy confusing. There’s one status for multiple accounts, but sometimes accounts don’t share the same status.
Ideas:

• present list of networks on left pane
• sniff out accounts from web browser
• allow users to authenticate using web browser when possible
• sniff out accounts from evolution

Identity information & Display of user information

Overall users were anxious because their own information was not being displayed in empathy and they felt a lack of control over it.

• People didn’t know how they appeared to other people
• “I used 2 different names and would like to know how my name is displayed. I don’t know how to check which name is in use!”
• “I can’t see me that worries me bc I don’t know how exactly how I am being seen or not seen.”
• “Available” status dropdown at drop looks like a filter and not status.
• people not sure if the camera was turned on or off – worried people can see me without my knowing
• Is this person willing to be contacted by camera? having a camera doesn’t mean you’re willing be contacted by it (e.g. if they are at work)
• Some discussion of camera icon – wasn’t a problem for users to interpret in empathy but apparently the gmail camera icon sucks and confuses people
• People didn’t have a problem mechanically changing their alias but they didn’t understand where it was put into place. What is the difference between login ID and alias? There are actually local and remote aliases in empathy… in actuality, its usage is inconsistent based on the networks involved.
• “I’m not sure what I’m doing. I cannot fill in my name as I want it to be shown.”
• “Now I have a light bulb and no green light beside my nework. I don’t know if I am available anymore. How come Jenny is available and I am not?”

Displaying information around people

They expected to find all the information concerning them in one place and the information concern others accessible through their names. They didn’t see aggregate information about people.

Interacting with Others

They didn’t understand how to chat with two people at the same time. “If you can add an individual contact, you should be able to add a group in the same way.”
The technical challenge here is every service has different concepts of group chat. Some have ad-hoc group chat, some have an inflexible concept of rooms. What if the two users you want to chat with together are on different networks. If one friend is on MSN and one is on Jabber – then it’s not possible.
To create a group, people try to invite others to a room. The Skype application uses the concept of a room really well.

Sending and receiving files

• Confusion around the icon for file sending. They didn’t associate the arrow icon with receiving a file.
• Once the file was received – they would click on the sender’s name, expecting the file to be showing there. They looked around to find it. Where did I download it to?
• GNOME’s way of handling bluetooth files is awesome and is a model maybe we should follow here?
• People would have liked to get some information about the file so they would know where to store it – is it a photo? music?
• In Google Talk, you can send someone a file if they are offline / about to go offline and they can get it when they come back online?
• Some mention of Yahoo! messaging’s shake-window feature and how irritating it is

Creating Groups – treating groups differently

User: If I have the same contact with many different accounts, I could group them under the same group? The system should indicate if I can treat all my contacts in the same way or if there are some things I can’t do depending on which account.
I tried to create a group and I can’t create it – is it because they’re from different accounts? So I can’t put them together?
I can’t see which people can be treated the same way and which can’t be.

Tab interaction

• Very positively received. They felt it was very natural.
• A known area of failure is having 20+ conversations at once.
• One tab issue that came up – where users detached the tabs and put them back together – their instinct was to drag & drop to put tabs together, but they dragged and dropped from the top and it didn’t work. People didn’t figure you have to drag from where the name is. One way Mac tried to handle this is to have a drag and droppable icon to represent the document in a document window up in the titlebar. Makes it easy to move the folder you’ve already got open rather than have to go one level up to grab the folder icon.
• Some discussion about if you’re talking with person a in one tab and person b in the other, concern about confidential info that person in tab a can see but not person in tab b, especially with copying & pasting.

Searching conversations

People weren’t sure what they’d be searching – all the conversations you had forever? Or would you be searching for a specific person? They had to try it a few times to figure out what kinds of results they would get. They liked the feature where they could see conversations on the calendar, but confused about what to expect at first.
The conversation view is a time-focused view. We wonder if they would want a more people-centric view.

Minor issues / Wishlist / bugs

• When you first open empathy, it starts the icon but it doesn’t open a window. People didn’t know if it was open or not. Grey icon not noticeable.
• Default settings were an issue – people didn’t expect that their contact list would be in alphabetical order. They expected it to be sorted by status.
• The ‘offline’ grey box icon looks like a checkbox, some users thought it was clickable
• People didn’t understand the little broom icon for clearing fields. Better to have a grey X within the entry field itself. Some people thought they had to click the broom to activate their account and lost what they had just typed.
• People couldn’t figure out how to create favorite status messages. The empty star icon was too pale and was not noticed.
• People had an interesting way to use customized messages. They thought the message was like a status update like in facebook rather than be associated with the icon. So they typed ‘please don’t disturb’ next to the green icon.
• People didn’t understand the dialing pad sidebar was for. They clicked on sidebar and saw it and were mystified.
• Sometimes button in the account dialog is ‘save’ and sometimes it’s ‘login’ – sometimes they didn’t want to log in, they just wanted to create the account.
• One icon conveys several meanings for status – invisible and offline use the same icon.
• There’s an edit custom messages – but no favorites.
• When wanting to add a contact, users didn’t understand what ‘identifier’ meant. Vocabulary was confusing.
• “It is one thing to have a camera, but another to know if [my buddy] wants to use it.”
• Button missing besides status box. There’s an invisible button beside the status area in the main window that is a shortcut to the accounts dialog.
• Notifications – users liked that it flashed to signify urgency? When you first go online there’s a notification of who in your contacts is online. One of the users was a very heavy IM user, and when he gets online he’d have 300 people there and that notification would be unmanageable for him.
• Sometimes there’s a ‘find’ button to search, sometimes you have to click ‘enter’ and there is no button to search. Between the ‘search’ and ‘conversations’ tab of the search window.
• There’s not a lot of distinction between sound and notification, so why are they in two different tabs in preferences?
• Renaming accounts – people had a tough time getting this to work. They didn’t understand what alias meant, and they wanted to rename their accounts. They didn’t think in terms of their username, but in terms of the network.
• Users would like to know when someone is offline or busy – how long ago did that status change take place? E.g. if someone got offline 2 minutes ago, might be able to still send an email or phone call – but if it was 16 hours ago, different situation.
• People thought the use of color in looking at previous conversations is not contrasting enough at a glance – the difference between green and blue is not contrasty enough.
• One person asked for some branding – if they knew the application was ’empathy’ – it doesn’t call itself empathy anywhere. They didn’t know where they were. One user said, ‘the interaction in the application, feels more like an email than with IM. IM is friendlier, and less formal. More feedback and interaction when using the app – but this feels more like email and more formal.’ A little discussion here about extroverted apps like AOL vs more introverted apps like email.

Charline will be making the slides available shortly.