Chapter 13: Introducing Evaluation
Chapter Introduction | Web Resources | Assignment Comments | Teaching Materials
Objectives
The main aims of this chapter are to:
- Explain the key concepts and terms used in evaluation.
- Introduce a range of different types of evaluation methods.
- Show how different evaluation methods are used for different purposes at different stages of the design process and in different contexts of use.
- Show how evaluators mix and modify methods to meet the demands of evaluating novel systems.
- Discuss some of the practical challenges that evaluators have to con-sider when doing evaluation.
- Illustrate through short case studies how methods discussed in more depth in Chapters 7 and 8 are used in evaluation and describe some methods that are specific to evaluation.
Introduction
Imagine you have designed an app for teenagers to share music, gossip, and photos. You have prototyped your first design and implemented the core functionality. How would you find out whether it would appeal to them and if they will use it? You would need to evaluate it – but how? This chapter presents an introduction to the main types of evaluation and the methods you can use.
Evaluation is integral to the design process. Evaluators collect information about users’ or potential users’ experiences when interacting with a prototype, an app, a computer system, a component of a computer system, an application, or a design artifact such as a screen sketch. They do this in order to improve its design. Evaluation focuses on both the usability of the system (e.g. how easy it is to learn and to use) and on the users’ experience when interacting with it (e.g. how satisfying, enjoyable, or motivating the interaction is).
Devices like smartphones, iPods, iPads, and e-readers have heightened awareness about usability, but many designers still assume that if they and their colleagues can use a product and find it attractive, others will, too. The problem with this assumption is that designers may design only for themselves. Evaluation enables them to check that their design is appropriate and acceptable for the wider user population.
There are many different evaluation methods. Which to use depends on the goals of the evaluation. Evaluations can occur in a range of places such as laboratories, people’s homes, outdoors, and work settings. Evaluations usually involve observing participants and measuring their performance – in usability testing, experiments, or field studies. There are other methods, however, that do not involve participants, such as modeling user behavior. These tend to be approximations of what users might do when interacting with an interface, often done as a quick and cheap way of assessing different interface configurations. The level of control on what is evaluated varies; sometimes there is none, such as in field studies, and in others there is considerable control over which tasks are performed and the context, such as in experiments.
In this chapter we discuss why evaluation is important, what needs to be evaluated, where evaluation should take place, and when in the product lifecycle evaluation is needed. The different types are then illustrated by short case studies.