Diagrammatic Integration of Abstract Operations into Software Work Contexts

Most software applications present information to the user in a WYSIWYG form, where the main representation on the screen is made as close as possible to a visual facsimile of the final work product. Wherever possible, users specify required transformations of the product by directly selecting and manipulating areas of this visual facsimile. This brings great usability advantages, but is not adequate for the specification of abstract operations such as generalization and inference commands, which are usually represented linguistically in menus and dialogues. We report a series of experimental implementations exploring alternatives to menus and dialogues. In these six systems, abstract functionality is integrated into the work context through two techniques: diagrammatic interpretation of user’s actions (gestures) and diagrammatic overlays superimposed as semi-transparent layers over the visual presentation of the work product. We discuss the diagrammatic justifications and consequences of these alternatives, and present results of preliminary user studies suggesting that both forms of interaction may in future be valuable techniques for exploiting diagrammatic formalisms in software interaction.