Usable....but is it useful
Andrew Swartz, managing consultant from Serco Usability Service has written on how to test how useful a product or service is as well as how usable it is.
As an example he cites public internet terminals, an initiative which has spectacularly failed to take off (Even narrow focused terminals such as reward point terminal in supermarkets are being remove as quickly as possible).
In his article Andy offer 4 point on testing for usefulness.
- Limits of self-reporting : Users can be poor at predicting their own behaviour. We often see participants in usability studies claiming that they would love a new product only to see the product fail in the marketplace, or expressing disdain for a new product only to see it succeed. Take what users say with a grain of salt.
- Analyse based on context of study: Most lab-based usability studies give us the opportunity to see users' first 60-90 minutes interacting with a new product or service. First reactions are important, but they are not everything. Poor initial reaction means only that - that users coming cold to the product are not receptive. In practice, it could be either that the product simply cannot succeed, or it could be that users need more background to make them appreciate the product's value. (Think what a marketing team could do with information like that.)
- Consider different kinds of studies: Some products look remarkably different in the lab than in the real world. Some products require long start-up periods. For example, voice recognition products require an initial training period followed by a month or so in which the user intensively corrects the software. An hour in a test lab may tell you about the obstacles in the training process, but not much about the ultimate usefulness of such a product. Other products rely on 'network effects'. For example, one person with a video-calling phone on his own isn't going to find it very interesting; but a large group of sports fans may come to think of it as indispensable. Design your studies accordingly - longer-term studies for products that require it, and group studies for network effects
- Beware puppy love: Some features make users fall in love at first sight, but not every love lasts. Clippy, the infamous talking paper clip in Microsoft Office products (serving as a painfully eager help system), is typically adored by naïve users the first time they see it, and desperately despised within a week or two, once users become aware how intrusive it can be. If you had conducted a standard user test before Clippy shipped, it probably would have predicted great success; but now you can type "hate Clippy" in Google to see how a user interface can inspire genuine passion.