Usability the problem of testing in labs

While reviewing usability tools and looking at the best way to test mobile (cell phone) apps I came across this handy summary that sums up how I feel about lab testing.

“Usability researchers and practitioners have been concerned that laboratory evaluations do not simulate the context where mobile phones are used (Johnson 1998) and lack the desired ecological validity. Interruptions, movement, noise, multitasking etc. (Tamminen et al. 2004) that could affect the users’ performance are not present in laboratory tests. The surrounding environment and mobility are assumed to set special requirements for mobile applications. Usability testing should take these requirements into account.”

Although the conclusion of this paper is that when testing mobile applications you will identify all the issues. The thing you will miss is the incidental observations about user behaviour – stepping out of a crowd to do complex tasks – but nothing that would effect the identification of problems the user has your application.

Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing

Dark Patterns

A well explained and compelling presentation at http://wiki.darkpatterns.org/Home on how some organisation exploit UX in unethical ways. It is fascinating to see how incompetence and dumb luck in terms of web form design is evolving. As we gain understanding of the psychological and cognitive processes people are learning how to exploit misunderstanding and trust. The humble web form becomes a muggers paradise.

The line between encouraging people to do what you want and tricking them into doing things that they don’t want to is very grey. As DarkPatterns.org explain many companies will have a poor moral compass and without pressure from outside. For example shoddy tricks to make company websites appear higher up search engine results are only kept to a minimum because the search engine software is regularly updated to punish people that seek to ‘cheat’ their way to the top. There is no equivalent in terms of UX. I think the only that companies can be dissuaded from making ££££s with little effort is to publicly name and shame them. When the use of dark patterns is seen as the confidence trick that it is we have some hope of limiting there use to murkier areas of the interweb.

Analytics is not the answer

A CEO I know has finally come round to the idea that endlessly changing things and adding new features is not the way forward – Hazaar! He now thinks that analytics will provide all the answers. As the company hops from the frying pan of whimsical change into the hellish fire of stats I have been arguing for user testing to be thrown into the mix. This is an uphill struggle.

The CEO threw up the idea of analytics identifying the cancer that we can chop out. I suggested that, pushing the analogy further, if analytics was the x-ray that identified the cancer then user testing is equivalent of talking to the patient.

It will be interesting to see if this patient survives.

UX note: Display default review as 5 stars

Example of play.com displaying default review as 5 stars

Example of play.com displaying default review as 5 stars. Even products with no reviews look like they are great!


Most shopping or review sites set the review to 0 or hide the start rating until enough reviews have been made. By showing 5 stars and asking people to review they have given people, quickly scanning the page, the idea that this item is great. This seems underhand, but I bet it is effective.