Skip to main content

Über dieses Buch

What the book is about This book is about the theory and practice of the use of multimedia, multimodal interfaces for leaming. Yet it is not about technology as such, at least in the sense that the authors do not subscribe to the idea that one should do something just because it is technologically possible. 'Multimedia' has been adopted in some commercial quarters to mean little more than a computer with some form of audio ar (more usually) video attachment. This is a trend which ought to be resisted, as exemplified by the material in this book. Rather than merely using a new technology 'because it is there', there is a need to examine how people leam and eommunicate, and to study diverse ways in which computers ean harness text, sounds, speech, images, moving pietures, gestures, touch, etc. , to promote effective human leaming. We need to identify which media, in whieh combinations, using what mappings of domain to representation, are appropriate far which educational purposes . . The word 'multimodal ' in the title underlies this perspective. The intention is to focus attention less on the technology and more on how to strueture different kinds of information via different sensory channels in order to yield the best possible quality of communication and educational interaction. (Though the reader should refer to Chapter 1 for a discussion of the use of the word 'multimodal' . ) Historically there was little problem.



Chapter 1. The ‘M-Word’: Multimedia Interfaces and Their Role in Interactive Learning Systems

As discussed in the Editors’ Preface, many writers use terms with an apparently imprecise idea of their meaning — and of what the reader will understand by them. This chapter makes a valuable contribution by proposing definitions which distinguish between two important concepts: medium and mode. On that basis it is possible to go on to make consistent statements about interfaces which use multiple media and multiple modes, and the author does this in the context of describing the development of novel interfaces. This review looks at interfaces from a number of viewpoints. First, developments in the technology of human-computer communication are considered, along with recent attempts to develop the somewhat neglected modalities of touch and audition in the human-computer interaction (HCI) context. Thus, for example, the imminent arrival of digital video is discussed, as well as recent work on using non-speech audio, touch and gesture, in interfaces. Next, an attempt is made to point to. the likely effects of multimedia interaction on the user by turning to the human-performance literature and by attempting to derive some broad guidelines. In the last section of this chapter, which focuses on the implications for interactive learning, the question is put: what can we claim for multimedia? The answer is that it improves the quality of HCI dialogue, and it gives a greater opportunity to engage the learner’s interest. In the context of a long history of disappointments in the use of technology for learning, these must be considered as highly promising gains.
J. Terry Mayes

Chapter 2. Hunting Musical Knowledge in Darkest Medialand

The development of multimedia interfaces often appears to be technology-driven. So it is that developers often think that they are dealing with new phenomena, new problems. However, in the field of music, people have been struggling with many of the same problems for centuries, including adaptation of what might be called the ‘musician-instrument interface’ as well as how they might improve on traditional inadequate and non-intuitive notations. In this chapter, the author presents a very personal historical view. The chapter focuses on three decades of work on the application of artificial intelligence techniques and human-computer interface design to music education, against a background of four centuries of musical experience, and draws lessons applicable to the development of learning in any field.
R. Sterling Beckwith

Chapter 3. ‘Rights in the Mirror’: An Interactive Video Drama Programme About Human Rights Education

This chapter describes a case study that is unusual both in its theoretical inspiration and in the domain of-education to which it is applied. The multimedia system ‘Rights in the Mirror’ assists student teachers to learn about teaching human rights issues. It has been asserted that interfaces of the future will make use of a theoretical metaphor, using agents. Taking the metaphor of interfaces as drama in a different direction, ‘Rights in the Mirror’ consists of a videodisc- and hypertext-based system that is explicitly structured according to dramatic principles and metaphors with particular reference to the theory and practice of Greek dramatists and the ‘learning plays’ of Brecht. The system was developed in association with the Utrecht School of Art, an international centre conducting both education in the creative arts and research into artificial intelligence (see Chapter 2).
Joseph Nolthuis

Chapter 4. On-Site Distance Learning: A Multimedia Full-Scale Experiment in Denmark

It is important to remember that not all education takes place in classrooms with young students., Continuing, up-dating education is becoming increasingly important with developments in technology and the consequent changes in the work people do. This presents a different set of challenges and problems: the students are more heterogeneous; they may not have received formal education for some years; they are valuable workers who cannot be spared from their jobs for long periods of time, and so on. This chapter describes one approach to meeting such requirements. There was a need for continuing education of factory workers, and it was decided that some of the above-mentioned constraints dictated that the appropriate educational delivery strategy was on-site training at a distance. In such a situation, a multimedia approach seems particularly appropriate. This chapter reports on the implementation of such a system. The implementation was based on currently available technology, including databases, interactive video, computer conferencing, computer-aided instruction (CAI) and slow-scan television, but used them in such a way as to simulate the kind of learning environment which will be available in the future, when the technology has been developed further. Multimedia systems were used not just to facilitate human-machine interaction, but to facilitate learning through peer interaction — by participants who may not have been in the same building. The approach described here draws on the long practical and theoretical tradition in Denmark that treats education as a social, collaborative process. This chapter discusses how ‘virtual classrooms’ are found to facilitate new learning approaches: for example the teacher/trainer may become more of a consultant or director than a traditional teacher. Unlike many pilot multimedia schemes, this system was subjected to extensive practical testing and evaluation.
Mette Ringsted

Chapter 5. ‘Playing’ Graphics in the Design and Use of Multimedia Courseware

Another growing area of application of computers in education is the development of Computer Assisted Learning (cal), and more recently intelligent cal. It is only natural that these two approaches should be combined, to make potentially very powerful education systems. This chapter presents the extensive work which has already been carried out in this area in one group in France. The chapter addresses some of the theoretical questions, such as the relationship between the participants: the author, the teacher and the learner, as well as describing tools which have been developed to facilitate die development of courseware.
Jacqueline Vacherand-Revel, Christian Bessière

Chapter 6. Design Guidelines for Electronic Book Production

By many definitions a book may be considered to have a multimedia interface. (See the definitions in Chapter 1). The educational power and value of books would not be disputed, but there is clearly scope for improvement. In particular, a book is not interactive. It is a channel for very remote communication between author and reader. Information technology makes it possible to take all that is good about books and enhance them with the power and interactivity of a computer. This chapter discusses such ‘electronic books’. A significant amount of experience has already been accumulated in this area. Design guidelines have been formulated which are presented in the chapter within the context of four case studies. Some potential future directions of development are also outlined.
Philip Barker

Chapter 7. Computer-Controlled Video for Intelligent Interactive Use: a Description Methodology

Regardless of any definition which one might adopt for terms such as ‘multimedia’, a qualitative difference between the interfaces described in this book and traditional ones is their richness. However, richness implies complexity, which has to be controlled. Video is an example of such richness and this chapter describes one approach to manage its complexity. Before one can control any phenomenon, one has to be able to refer to its components, to describe them systematically and consistently. Video-based intelligent tutoring systems that can cope flexibly with unanticipated teaching situations may need to use artificial intelligence techniques to enable them to draw inferences for themselves about the scope and relevance of pre-recorded video sequences. Parkes, working within the discipline of artificial intelligence and education addresses this and related problems. His chapter includes introduction of a description methodology for application to both still and moving images, and illustrates this with some examples. Implications for the development of multimedia education systems in general are also noted.
Alan P. Parkes

Chapter 8. Representing Musical Relationships in the Harmony Grid

One of the most difficult questions of multimedia design is when is it appropriate to use a given medium or mode of interaction. Sometimes the answer is not as obvious as it might seem. For instance, a program which manipulates music obviously has to be capable of playing music, presenting auditory information, but to what extent should the interface to the program be an auditory one? This chapter describes one such program which teaches about music, but with which the student interacts using a two-dimensional spatial representation of musical relationships. In other words a cross-modality mapping occurs. This kind of mapping appears to be especially successful because variants of it have been applied to good educational effect more than once. Indeed, Chapter 12 describes a successful use of a different but closely related cross-modality mapping. A very interesting and open question is: why is this mapping appropriate, where others may not be?
David Levitt

Chapter 9. Communicating and Learning Through Non-speech Audio

Multimedia communication is the rule rather than the exception in most human interaction other than with computers. There is a great deal which can be learned from traditional communication and applied to human-computer interaction in an educational context. This chapter is based upon such an approach, looking at the use of non-speech sounds. This is a good example of the expansion of the interface in that previously auditory output from computers have generally been restricted to undifferentiated ‘beeps’, which implies that one of the most powerful channels of communication has been virtually ignored.
Meera M. Blattner, Robert M. Greenberg

Chapter 10. Redundancy and Adaptability

Extending an interaction to use more than one medium (or mode or channel) introduces the possibility of there being redundancy in the communication. This is an important opportunity because redundancy is a natural part of human communication. However, this chapter makes the additional point that redundancy in an interface also improves the possibilities for adapting it for use by people who have disabilities. Adapting computers for use by people with sensory disabilities is very much a problem of the interface. It seems likely that if several channels are utilized, then it should be possible to communicate (almost) as much information to a user who has a restriction in one as to an able-bodied user. If that is the case, then the users’ internal representations are likely to be almost identical. The deficits of disabled users’ cognitive models of the systems with which they interact are therefore due to the limits on the information they receive, rather than any inherent limitation in their ability to master complex systems. Vision has a very high bandwidth and tends therefore to dominate (as discussed in Chapter 1) and yet interface designers do not by any means make use of the full bandwidth of the other senses. For the average user some of that added information will be superfluous, redundant, but for others it may mean that communication is possible where it was not so previously.
Alistair D. N. Edwards

Chapter 11. Multimodal Communication, Natural Language and Direct Manipulation (Gestures) in Human-Computer Interaction

Human-computer communication is quite impoverished compared to human-human communication, both in its form and its effectiveness. In many cases the aim of designing a multimedia interface is to produce an interaction which is more like that which takes place between people. Natural language is a highly developed form of (human-human) communication, one which is often supplemented by body language, one of the most important components of which is hand gesture. For example, people often use pointing (deixis) to avoid having to describe something that is complex, inexact or inaccessible to them. On the other hand, natural language can be much more refined at specifying what part or aspect of something is currently of interest. This is particularly true where quantification, abstraction, combination, negation or time are involved. This chapter describes how these two media of human-human interaction — natural language and gesture — can be integrated in a human-computer interface. It describes how these have been applied in a variety of different practical applications to combine the strengths of both modes of interaction.
Karl-Heinz Hanne

Chapter 12. Interface Design for Empowerment: a Case Study from Music

It is very seldom that psychological theory is applied to human - computer interface design — because very few theories have yet been formulated which are applicable. For the most part designers have to be content to use guidelines and models, which have less applicability. So, the work described in this chapter is unusual, because it describes an interface to a program which teaches about musical harmony, based on psychological theories. The success of that approach is borne out by the fact that the theories suggest the use of a specific style of interface, based on a two-dimensional spatial representation of harmony relationships. This in turn has been shown to be very successful in teaching novice users about harmony.
Simon Holland

Chapter 13. Notes While Waiting for the View to Clear

Integrating Multiple Media in Educational Computer Interfaces. Such was the announced topic of this Advanced Research Workshop — part of the Nato Science Committee’s Special Programme in Advanced Educational Technology. Its stated objective: to permit us to share our ‘interests in the use of different sensory modalities in human-computer interfaces’, while exploring ‘how the different modalities should be integrated in educational computer systems.’
R. Sterling Beckwith


Weitere Informationen