Overview

Clarisse de Souza (2005) and her colleagues have developed a theory of HCI—semiotic engineering—that provides a series of tools for HCI design and evaluation. In semiotics thefundamental entity is the sign, which can take the form of a gesture, a symbol, or words, forexample. One way or another all of our communication is through signs, even when we don’tintend to produce signs, our mere existence conveysmessages about us—how we dress, how oldwe are, our gender, the way we speak, etc., can all be interpreted as signs that carry information.

Semiotic engineering views human–computer interaction in terms of communication between the designers of the artifact and the user. In linguistic terms the designer and the user are thought of as interlocutors and the artifact is thought of as a message from the designer to users. Of main interest here is how the theory is applied in evaluation, which focuses on identifying breakdowns in communication between the user and the designer.
These breakdowns occur when the user fails to understand the message (i.e. the design of the artifact) sent by the designer—in other words, the problems that the user has interacting with the artifact—i.e. the communicability of a design. The method used is communicability evaluation.

Like usability testing and field studies, evaluating communicability is based on observing a user’s experiences with an application either directly or, more usually, recorded on video or audio. Using a predefined set of tags the evaluator analyzes the user’s behavior, focusing on breakdowns in which the user either could not understand the designer’s intentions (as encoded in the interface) or could not make herself understood by the application. The first step in the communicability evaluation involves tagging the user’s interaction with communicability utterances. In other words, it consists of ‘‘putting words into the user’s mouth’’ in a kind of reverse protocol analysis (de Souza, 2005, p. 126). The evaluator looks for patterns of behaviour that correspond to tags such as: ‘‘Oops!,’’ ‘‘Where is it?,’’ ‘‘I can do it this way,’’ ‘‘I can do otherwise,’’ ‘‘Looks fine to me.’’ The figure below presents a schematic image of communicability utterances for a few frames of recorded video. Thirteen such tags have been identified and you can see how they are applied in this case study.

References

De Souza, S.C. (2005) The Semiotic Engineering of Human–Computer Interaction. MIT Press, Cambridge, MA.

These movies have been created using TechSmith Camtasia. To view them you will need to download the TSCC.exe Codec from Techsmith's website at:http://www.techsmith.com/codecs/tscc/default.asp

Arach1a.avi

Spider1a.avi