Graphical Revelations: Comparing Students’ Translation Errors in Graphics and Logic

We are interested in developing a better understanding of what it is that students find difficult in learning logic. We use both natural language and diagram-based methods for teaching students the formal language of first-order logic. In this paper, we present some initial results that demonstrate that, when we look at how students construct diagrammatic representations of information expressed in natural language (

nl

) sentences, the error patterns are different from those observed when students translate from

nl

to first-order logic (

fol

). In the

nl

-to-diagram construction task, errors associated with the interpretation of the expression

not a small dodecahedron

were manifested much more frequently with respect to the object’s size than with respect to its shape. In the

nl

-to-

fol

task, however, no such asymmetry was observed. We hypothesize a number of possible factors that might be implicated here: differences between the

nl

-to-diagram and

nl

-to-

fol

tasks; the reduced expressivity of diagrams compared to language; scoping errors in participants’

nl

parsing; and the visuospatial properties of the blocks-world domain. In sum, constructing a diagram requires the student to provide an instantiated representation of the meaning of a natural language sentence; this tests their understanding in a way that translation into first-order logic does not, by ensuring that they are not simply carrying out a symbol manipulation exercise.