External Representations Contribute to the Dynamic Construction of Ideas

External representations such as diagrams, sketches, charts, graphs and scribbles on napkins play facilitatory roles in inference, problem-solving and understanding (e.g. [1],[2],[3],[4],[5],[6],[7],[8],[9]). How does the externality and visibility of representations facilitate inference and problem-solving? One benefit of external representations is on memory. They reduce working memory load by providing external tokens for the elements that must otherwise be kept in mind. This frees working memory to perform mental calculations on the elements rather than both keeping elements in mind and operating on them [2],[9]. External representations also serve as visuo-spatial retrieval cues for long term memory, evoking relevant information that might not otherwise be retrieved. Another benefit of external representations is to promote discovery and inference, both visuo-spatial and metaphorical. Perceptual judgements about size, distance, and direction are easily made from external representations (e.g.[4]). In a Venn diagram, set relations such as inclusion are abstractly mapped onto visuo-spatial diagrammatic features, enabling direct perceptual calculation. Visuospatial features such as proximity, connectivity, and alignment provide useful hints to selection of appropriate inference paths (e.g.[1],[6],[8]) and to proper understanding of the structure of a target system (e.g.[5]). Calculations requiring counting, sorting, or ordering are easily made by rearranging external spaces (e.g. [7]).To serve these functions of memory, inference, and calculation, the interpretation of the external representation is static; it must stay the same in order not to introduce error in the operations performed from the external representation. External representations, however, are visuo-spatial displays, and it is known from research on perception that such displays, especially vague and ambiguous ones, can be interpreted and reinterpreted. Are there situations where the very instability of visuo-spatial displays can be used to advantage?One situation where the instability of interpretations of external representations can be beneficial is design. Among the earliest of commentators on this was Schon [10], who proposed that freehand sketches serve as a medium for the dynamic generation of new design ideas. In developing ideas for new projects, designers do not draw sketches to externally represent ideas that are already consolidated in their minds. Rather, they draw sketches to try out ideas, usually vague and uncertain ones. By examining the externalizations, designers can spot problems they may not have anticipated. More than that, they can see new features and relations among elements that they have drawn, ones not intended in the original sketch. These unintended discoveries promote new ideas and refine current ones. This process is iterative as design progresses. Seeing unintended relations and features in sketches requires release from previous interpretations. Previous interpretations can have a strong hold on observers, so preventing fixation and encouraging new interpretations are perceptual processes desirable for designers to acquire.In recent years we have explored ways that designers use external representations to discover and develop design ideas. The project has used both naturalistic and experimental methods. We present data from both these projects that are relevant to the current analysis. In a large naturalistic study, novice and experienced architects were filmed as they designed a museum. Later, they watched their design session, and commented on every stroke of the pen. These protocols have been analyzed in detail [11],[12],[13]. In the experimental study, designers and novices were shown ambiguous sketches and asked to produce as many interpretations of them as possible.The protocols from the design sessions showed clearly that new design ideas were likely to be generated immediately after discovering new relations and features in one's own sketches. Notably these new relations and features were unintentional byproducts of the aspects of the sketches drawn for other reasons. We call this process detection of unintended relations and features [12]. Detection of unintended relations and features is a significant impetus for the generation of new ideas. The generation of new ideas, in turn, was likely to become an impetus for further detection of unintended relations and features, so that each component process drives the other [12]. The joint occurrence of generation of new ideas and detection of unintended relations and features constituted the core cognitive processes of designers as they worked. As is common in discovery problems, the designers themselves were not able to predict what kind of unintended relations and features they were going to detect and what kinds of ideas they were going to generate. Productive design is situated in the physical setting of sketches. They are not merely a static medium for externalizing internal visions, but rather a physical environment from which ideas are generated on the fly.The key to creative design, then, is the cyclic pattern of generating new ideas and detecting unintended relations and features in external representations. More generally, it can be regarded as a coordinated co-generation of new conceptual thoughts and perceptual discoveries in external representations. This appears to be a general phenomenon occurring in a broader context involving any kind of creation, not just in design processes [13]. An important issue is how to facilitate the co-generation of reinterpretations and novel ideas in inspecting external representations, given that it is by no means automatic. We argued that the cognitive skill we have called constructive perception promotes the discovery of new interpretations in external representations [14]. By constructive perception, we mean self-awareness of the ways that perception underlies interpretations of external representations. The self-awareness allows searching for other ways to perceive, enabling reorganization of the external representation to promote novel interpretations. Experienced designers are superior to laypeople in this skill [14]. This finding raises two issues, one cognitive, the other didactic. What constitutes the ability of constructive perception? How can people be trained to use it? Research on these will promote successful use of external representations for creative purposes.