Chapter Introduction | Web Resources | Assignment Comments | Teaching Materials

Imagine you have designed a new shared web space intended for advertising second-hand goods. How would you find out whether householders would be able to use it to find what they wanted and whether it was a reliable and effective service? What evaluation methods would you employ? In this chapter we describe two very different approaches—usability testing and field studies. The first you would use to evaluate its effectiveness, ease of use, and satisfaction, and the second you would use to assess how it would support householders finding desired items. The first, usability testing, takes place mainly in a controlled environment; the second, a field study, examines how a product is used in natural home environments.

Typically, the performance of users interacting with a product is measured for set tasks in a usability test. This allows the system’s effectiveness to be assessed and usability problems to be identified, e.g. problems printing a document or finding information. Usability testing varies in terms of the amount of control over the study. At one end are experiments, typically carried out in laboratory settings, that test user performance (e.g. reading) on one or two variables (e.g. font size, resolution) on a number of participants. At the other end are more opportunistic studies, that investigate a subset of usability and user experience goals, e.g. satisfaction, fun, frustration, for a handful of users. Between these two extremes are controlled tasks that examine such things as the ease of searching for and locating information items.

Recently, it has become more common for field studies to be conducted in natural environments, such as homes, offices, and outdoors. This is especially so for the evaluation of new technologies, such as mobile devices. Observation, diaries, and interviews are commonly used methods. Importantly, they take place in the setting where the product is used.

The main aims of this chapter are to: