Skip to main content
Erschienen in: Universal Access in the Information Society 4/2013

01.11.2013 | Long paper

Equivalent representations of multimodal user interfaces

Runtime Reification of Abstract User Interface Descriptions

verfasst von: Kris Van Hees, Jan Engelen

Erschienen in: Universal Access in the Information Society | Ausgabe 4/2013

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

While providing non-visual access to graphical user interfaces has been a topic of research for over 20 years, blind users still face many obstacles when using computer systems. Furthermore, daily life has become more and more infused with devices that feature some kind of visual interface. Existing solutions for providing multimodal user interfaces that ensure accessibility are largely based on either graphical toolkit hooks, queries to the application and environment, scripting, model-driven user interface development or runtime adaptation. Parallel User Interface Rendering (PUIR) is a novel approach based on past and current research into accessibility, promoting the use of abstract user interface descriptions. Based on a single consistent conceptual model, PUIR provides a mechanism to render a user interface simultaneously in multiple modalities. Each representation of the UI provides equivalent semantics to ensure that collaboration between users of different modalities is facilitated. The possible application of this novel technique goes well beyond the realm of accessibility, as multimodal interfaces become more popular and even needed. The design presented here has been implemented as a prototype for testing and further research in this fascinating area of HCI.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
Either through direct manipulation (“direct access”) or indirectly (“assisted access”) by means of some form of assistive technology solution.
 
2
Edwards et al. [20] list only three layers in, but they limited themselves to a description of the layers of modelling, where the conceptual layer forms the basis for those three layers of modelling.
 
3
Also known as the CAMELEON Reference Framework.
 
4
The Unifying Reference Framework comprises more elements than are presented here. The discussion of the state of the art does not require all elements of the framework, and the scope has therefore been limited to what is sufficient to describe, understand, and compare the various approaches.
 
5
The initial version of the Unifying Reference Framework [11]. defined the adaptation operation as a transformation between representations at the same level of abstraction. Revisions made in support of plasticity of user interfaces (being able to adapt to context changes without affecting usability) introduced adaptation as a cross-level operation.
 
6
This is an follow-up study using an updated list of UIDLs. It confirms the findings of Souchon and Vanderdonckt [63].
 
7
Each host where application code may be executed must have a session manager running.
 
8
It is possible for all components to run on a single system.
 
9
Even though some of the data capturing may take place between the application and the graphical toolkit (e.g. by means of an interposing library), and therefore prior to the graphical rendering, the data can still be considered visual because the application usually either tailors the data in function of the chosen representation or it passes it to specific functions based on a chosen visualisation.
 
10
This not only involves defining objects and activities (functionality of objects), but also developing strategies to introduce the new model to the anticipated user population.
 
11
“Computer reality” is defined by Gaver as [22, p. 85] “the domain in which computer events are described, either by reference to the physical hardware of the system or its operations expressed in some programming language.”
 
12
Sighted users may often not even realise that a mental model is involved due to the fact that a GUI is generally presented visually using iconic elements that are closely related to the underlying conceptual (mental) model.
 
13
Objects and the manipulations that are possible on and with those objects.
 
14
It is safe to assume for the purpose of this discussion that the element provides user interaction functionality only. Conveying information is an aspect of abstract UI semantics that can be represented in various ways—it is not purely related to visualisation.
 
15
It is important to note that although information may be lost, accessibility may not be impacted because often only perceptual information is affected.
 
16
Either between blind users or between a sighted and a blind user. Sighted users often tend to depend on a visual focal point when collaborating about the interaction with a system or an application.
 
17
The main requirement for the communication channel is that it provides for sufficient bandwidth to enable efficient and specific exchange of information. A phone connection is often much more constructive to collaboration than, for example, an online chat session.
 
18
Or at least, not possible in an equivalent and/or efficient manner.
 
19
Kochanek [34] provides a detailed description of the construction-process for an off-screen model for a GUI.
 
20
Specifically, the needs of the target group as those relate to UI interaction.
 
21
A pluggable component is one that can easily be replaced by an equivalent component. The term is commonly used in UI contexts to indicate exchangeable presentation components. It is a derivative of the “plug-n-play” hardware concept.
 
22
This AUI description effectively defines the user interface at the conceptual and semantic level. It is a formal description of the conceptual mode.
 
23
Note that this does not necessarily imply that the result of user interaction is immediate, although it has become common practice to provide near-immediate results in support of the WYSIWYG design principle.
 
24
It is important to note that it is not a requirement that all representations are generated at runtime, although development time construction of any representations could imply that dynamic updates to the UI structure are not possible.
 
25
The physical devices that the user employs to perform operations of user interaction with the application.
 
26
Device drivers at the OS level are meant to serve all possible consumers. The events they generate are most commonly very low-level events.
 
27
This is known as a contraction in English Braille, American Edition.
 
28
It is obvious that even amongst the blind or the sighted, not necessarily everyone will prefer everything the same way. This has been a driving force behind the efforts to provide user customisations for UIs.
 
29
Generically, this type of control is known as a “valuator.”
 
30
This is a common problem in any circumstance where essentially the same information is presented in tow different locations.
 
31
Although “appearance” is commonly interpreted as an aspect of visual perception, it actually carries a much broader meaning, across multiple modalities of perception.
 
32
This is fundamentally different from other approaches (e.g. the HOMER UIMS [59, 60]) where UI objects are described multiple times: once in abstract form, and once or more in modality-specific forms.
 
33
This is commonly known as a “call back” feature.
 
34
In this context, being “visible” means that the user can note the existence of the widget. Being part of the AUI, the widget obviously has no perceptual characteristics.
 
35
More specific research into the impact of mental context switching and related topics is outside the scope of this work.
 
36
A common problem would be that the modality in which the user interaction was initiated might render the feedback prior to the application logic responding to the operation, whereas all other renderings would render feedback afterwards. This is also commonly observed in assistive technology solutions such as screen readers that are implemented as a derivative to the graphical representation.
 
37
And in order to work towards a possible future adoption as an AT support solution.
 
Literatur
1.
Zurück zum Zitat Ali, M.F.: A transformation-based approach to building multi-platform user interfaces using a task model and the user interface markup language. Ph.D. thesis, Virginia Polytechnic Institute and State University (2004) Ali, M.F.: A transformation-based approach to building multi-platform user interfaces using a task model and the user interface markup language. Ph.D. thesis, Virginia Polytechnic Institute and State University (2004)
2.
Zurück zum Zitat Barnicle, K.: Usability testing with screen reading technology in a Windows environment. In: Proceedings of the 2000 Conference on Universal Usability, CUU ’00, pp. 102–109. ACM (2000) Barnicle, K.: Usability testing with screen reading technology in a Windows environment. In: Proceedings of the 2000 Conference on Universal Usability, CUU ’00, pp. 102–109. ACM (2000)
3.
Zurück zum Zitat Bergman, E., Johnson, E.: Toward accessible human-computer interaction. In: J. Nielsen (ed.) Advances in Human-Computer Interaction, vol. 5, pp. 87–113. Ablex Publishing Corp., New York (1995) Bergman, E., Johnson, E.: Toward accessible human-computer interaction. In: J. Nielsen (ed.) Advances in Human-Computer Interaction, vol. 5, pp. 87–113. Ablex Publishing Corp., New York (1995)
4.
Zurück zum Zitat Bishop, J., Horspool, N.: Developing principles of GUI programming using views. In: Proceedings of the 35th SIGCSE technical symposium on Computer science education, SIGCSE ’04, pp. 373–377. ACM (2004) Bishop, J., Horspool, N.: Developing principles of GUI programming using views. In: Proceedings of the 35th SIGCSE technical symposium on Computer science education, SIGCSE ’04, pp. 373–377. ACM (2004)
5.
Zurück zum Zitat Blattner, M., Glinert, E., Jorge, J., Ormsby, G.: Metawidgets: towards a theory of multimodal interface design. In: Computer Software and Applications Conference, 1992. COMPSAC ’92. Proceedings., Sixteenth Annual International, pp. 115–120. IEEE Computer Society Press (1992) Blattner, M., Glinert, E., Jorge, J., Ormsby, G.: Metawidgets: towards a theory of multimodal interface design. In: Computer Software and Applications Conference, 1992. COMPSAC ’92. Proceedings., Sixteenth Annual International, pp. 115–120. IEEE Computer Society Press (1992)
6.
Zurück zum Zitat Bouillon, L., Vanderdonckt, J., Chow, K.C.: Flexible re-engineering of web sites. In: Proceedings of the 9th International Conference on Intelligent user interfaces, IUI ’04, pp. 132–139. ACM (2004) Bouillon, L., Vanderdonckt, J., Chow, K.C.: Flexible re-engineering of web sites. In: Proceedings of the 9th International Conference on Intelligent user interfaces, IUI ’04, pp. 132–139. ACM (2004)
7.
Zurück zum Zitat Bouraoui, A., Soufi, M.: Improving computer access for blind users. In: K. Elleithy (ed.) Advances and Innovations in Systems, Computing Sciences and Software Engineering, pp. 29–34. Springer, The Netherland (2007)CrossRef Bouraoui, A., Soufi, M.: Improving computer access for blind users. In: K. Elleithy (ed.) Advances and Innovations in Systems, Computing Sciences and Software Engineering, pp. 29–34. Springer, The Netherland (2007)CrossRef
8.
Zurück zum Zitat Braille Authority of North America: Braille Formats: Principles of Print to Braille Transcription 1997. American Printing House for the Blind (1998) Braille Authority of North America: Braille Formats: Principles of Print to Braille Transcription 1997. American Printing House for the Blind (1998)
9.
Zurück zum Zitat Brunet, P., Feigenbaum, B.A., Harris, K., Laws, C., Schwerdtfeger, R., Weiss, L.: Accessibility requirements for systems design to accommodate users with vision impairments. IBM Syst. J. 44(3), 445–466 (2005)CrossRef Brunet, P., Feigenbaum, B.A., Harris, K., Laws, C., Schwerdtfeger, R., Weiss, L.: Accessibility requirements for systems design to accommodate users with vision impairments. IBM Syst. J. 44(3), 445–466 (2005)CrossRef
10.
Zurück zum Zitat Buxton, W., Gaver, W., Bly, S.: Auditory interfaces: The use of non-speech audio at the interface (1994). Draft manuscript Buxton, W., Gaver, W., Bly, S.: Auditory interfaces: The use of non-speech audio at the interface (1994). Draft manuscript
11.
Zurück zum Zitat Calvary, G., Coutaz, J., Thevenin, D.: A unifying reference framework for the development of plastic user interfaces. In: Proceedings of the 8th IFIP International Conference on Engineering for Human-Computer Interaction, EHCI ’01, pp. 173–192. Springer, Berlin (2001) Calvary, G., Coutaz, J., Thevenin, D.: A unifying reference framework for the development of plastic user interfaces. In: Proceedings of the 8th IFIP International Conference on Engineering for Human-Computer Interaction, EHCI ’01, pp. 173–192. Springer, Berlin (2001)
12.
Zurück zum Zitat Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L., Vanderdonckt, J.: A unifying reference framework for multi-target user interfaces. Interact. Comput. 15(3), 289–308 (2003)CrossRef Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L., Vanderdonckt, J.: A unifying reference framework for multi-target user interfaces. Interact. Comput. 15(3), 289–308 (2003)CrossRef
13.
Zurück zum Zitat Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Souchon, N., Bouillon, L., Florins, M., Vanderdonckt, J.: Plasticity of user interfaces: a revised reference framework. In: Proceedings of the First International Workshop on Task Models and Diagrams for User Interface Design, pp. 127–134. INFOREC Publishing House Bucharest (2002) Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Souchon, N., Bouillon, L., Florins, M., Vanderdonckt, J.: Plasticity of user interfaces: a revised reference framework. In: Proceedings of the First International Workshop on Task Models and Diagrams for User Interface Design, pp. 127–134. INFOREC Publishing House Bucharest (2002)
14.
Zurück zum Zitat Chapman, M.: Create user interfaces with glade. Linux J. 87, 90–92,94 (2001) Chapman, M.: Create user interfaces with glade. Linux J. 87, 90–92,94 (2001)
15.
Zurück zum Zitat Congress of the United States of America: 42 U.S.C.—The Public Health and Welfare, Section 1382(a)2). GPO (1997) Congress of the United States of America: 42 U.S.C.—The Public Health and Welfare, Section 1382(a)2). GPO (1997)
16.
Zurück zum Zitat Coutaz, J., Nigay, L., Salber, D.: Multimodality from the user and system perspectives. In: Proceedings of the ERCIM’95 Workshop on Multimedia Multimodal User Interfaces (1995) Coutaz, J., Nigay, L., Salber, D.: Multimodality from the user and system perspectives. In: Proceedings of the ERCIM’95 Workshop on Multimedia Multimodal User Interfaces (1995)
17.
Zurück zum Zitat Draheim, D., Lutteroth, C., Weber, G.: Graphical user interfaces as documents. In: Proceedings of the 7th ACM SIGCHI New Zealand chapter’s International Conference on Computer-human interaction: design centered HCI, CHINZ ’06, pp. 67–74. ACM (2006) Draheim, D., Lutteroth, C., Weber, G.: Graphical user interfaces as documents. In: Proceedings of the 7th ACM SIGCHI New Zealand chapter’s International Conference on Computer-human interaction: design centered HCI, CHINZ ’06, pp. 67–74. ACM (2006)
18.
Zurück zum Zitat Edwards, A.D.N.: The difference between a blind computer user and a sighted one is that the blind one cannot see (1994). Interactionally Rich Systems Network, Working Paper No. ISS/WP2 Edwards, A.D.N.: The difference between a blind computer user and a sighted one is that the blind one cannot see (1994). Interactionally Rich Systems Network, Working Paper No. ISS/WP2
19.
Zurück zum Zitat Edwards, A.D.N., Mitsopoulos, E.: A principled methodology for the specification and design of nonvisual widgets. ACM Trans. Appl. Percept. 2(4), 442–449 (2005)CrossRef Edwards, A.D.N., Mitsopoulos, E.: A principled methodology for the specification and design of nonvisual widgets. ACM Trans. Appl. Percept. 2(4), 442–449 (2005)CrossRef
20.
Zurück zum Zitat Edwards, W.K., Mynatt, E.D., Stockton, K.: Providing access to graphical user interfaces—not graphical screens. In: Proceedings of the First Annual ACM Conference on Assistive Technologies, Assets ’94, pp. 47–54. ACM (1994) Edwards, W.K., Mynatt, E.D., Stockton, K.: Providing access to graphical user interfaces—not graphical screens. In: Proceedings of the First Annual ACM Conference on Assistive Technologies, Assets ’94, pp. 47–54. ACM (1994)
21.
Zurück zum Zitat Gajos, K., Weld, D.S.: SUPPLE: automatically generating user interfaces. In: IUI ’04: Proceedings of the 9th International Conference on Intelligent User Interfaces, pp. 93–100. ACM Press (2004) Gajos, K., Weld, D.S.: SUPPLE: automatically generating user interfaces. In: IUI ’04: Proceedings of the 9th International Conference on Intelligent User Interfaces, pp. 93–100. ACM Press (2004)
22.
Zurück zum Zitat Gaver, W.W.: The sonicfinder: an interface that uses auditory icons. Hum.-Comput. Interact. 4(1), 67–94 (1989)MathSciNetCrossRef Gaver, W.W.: The sonicfinder: an interface that uses auditory icons. Hum.-Comput. Interact. 4(1), 67–94 (1989)MathSciNetCrossRef
23.
Zurück zum Zitat Guerrero-Garcia, J., González-Calleros, J.M., Vanderdonckt, J., Muñoz-Arteaga, J.: A theoretical survey of user interface description languages: preliminary results. In: Latin American Web Congress, 2009. LA-WEB ’09, pp. 36–43 (2009) Guerrero-Garcia, J., González-Calleros, J.M., Vanderdonckt, J., Muñoz-Arteaga, J.: A theoretical survey of user interface description languages: preliminary results. In: Latin American Web Congress, 2009. LA-WEB ’09, pp. 36–43 (2009)
24.
Zurück zum Zitat Gunzenhäuser, R., Weber, G.: Graphical user interfaces for blind people. In: K. Brunnstein, E. Raubold (eds.) 13th World Computer Congress 94, vol. 2, pp. 450–457. Elsevier, Amsterdam (1994) Gunzenhäuser, R., Weber, G.: Graphical user interfaces for blind people. In: K. Brunnstein, E. Raubold (eds.) 13th World Computer Congress 94, vol. 2, pp. 450–457. Elsevier, Amsterdam (1994)
25.
Zurück zum Zitat Haneman, B., Mulcahy, M.: The GNOME accessibility architecture in detail (2002). Presented at the CSUN Conference on Technology and Disabilities Haneman, B., Mulcahy, M.: The GNOME accessibility architecture in detail (2002). Presented at the CSUN Conference on Technology and Disabilities
26.
Zurück zum Zitat Harness, S., Pugh, K., Sherkat, N., Whitrow, R.: Fast icon and character recognition for universal access to WIMP interfaces for the blind and partially sighted. In: Ballabio, E., Placencia-Porrero , I., Bellcasa, R.P.D.L. (eds.) Rehabilitation Technology: Strategies for the European Union (Proceedings of the First Tide Congress), pp. 19–23. IOS Press, Brussels (1993) Harness, S., Pugh, K., Sherkat, N., Whitrow, R.: Fast icon and character recognition for universal access to WIMP interfaces for the blind and partially sighted. In: Ballabio, E., Placencia-Porrero , I., Bellcasa, R.P.D.L. (eds.) Rehabilitation Technology: Strategies for the European Union (Proceedings of the First Tide Congress), pp. 19–23. IOS Press, Brussels (1993)
27.
Zurück zum Zitat Hollins, M.: Understanding Blindness: An Integrative Approach. Lawrence Erlbaum Associates, Boca Raton (1989) Hollins, M.: Understanding Blindness: An Integrative Approach. Lawrence Erlbaum Associates, Boca Raton (1989)
28.
Zurück zum Zitat Institute of Electrical and Electronics Engineers: 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology. IEEE, Los Alamos, CA (1990) Institute of Electrical and Electronics Engineers: 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology. IEEE, Los Alamos, CA (1990)
29.
Zurück zum Zitat International Organization for Standardization: ISO/IEC 9126, Information Technology, Software Product Evaluation, Quality Characteristics and Guidelines for their Use. ISO, Geneva (1991) International Organization for Standardization: ISO/IEC 9126, Information Technology, Software Product Evaluation, Quality Characteristics and Guidelines for their Use. ISO, Geneva (1991)
30.
Zurück zum Zitat International Organization for Standardization: ISO/IEC 9241-11, Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs), Part 11: Guidance on Usability. ISO, Geneva (1998) International Organization for Standardization: ISO/IEC 9241-11, Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs), Part 11: Guidance on Usability. ISO, Geneva (1998)
31.
Zurück zum Zitat Jacob, R.J.K.: User interfaces. In: Ralston, A., Reilly, E.D., Hemmendinger, D. (eds.) Encyclopedia of Computer Science, 4th Edition. Grove Dictionaries, Inc., Williston (2000) Jacob, R.J.K.: User interfaces. In: Ralston, A., Reilly, E.D., Hemmendinger, D. (eds.) Encyclopedia of Computer Science, 4th Edition. Grove Dictionaries, Inc., Williston (2000)
32.
Zurück zum Zitat Kawai, S., Aida, H., Saito, T.: Designing interface toolkit with dynamic selectable modality. In: Proceedings of the second annual ACM conference on Assistive technologies, Assets ’96, pp. 72–79. ACM (1996) Kawai, S., Aida, H., Saito, T.: Designing interface toolkit with dynamic selectable modality. In: Proceedings of the second annual ACM conference on Assistive technologies, Assets ’96, pp. 72–79. ACM (1996)
33.
Zurück zum Zitat Kay, A.C.: User interface: A personal view. In: Laurel, B. (ed.) The Art of Human-Computer Interface Design, pp. 191–207. Addison-Wesley Publishing Co, Boston (1990) Kay, A.C.: User interface: A personal view. In: Laurel, B. (ed.) The Art of Human-Computer Interface Design, pp. 191–207. Addison-Wesley Publishing Co, Boston (1990)
34.
Zurück zum Zitat Kochanek, D.: Designing an offscreen model for a gui. In: Zagler, W., Busby, G., Wagner, R. (eds.) Computers for Handicapped Persons, Lecture Notes in Computer Science, vol. 860, pp. 89–95. Springer, Berlin/Heidelberg (1994) Kochanek, D.: Designing an offscreen model for a gui. In: Zagler, W., Busby, G., Wagner, R. (eds.) Computers for Handicapped Persons, Lecture Notes in Computer Science, vol. 860, pp. 89–95. Springer, Berlin/Heidelberg (1994)
35.
Zurück zum Zitat Kost, S.: Dynamically generated multi-modal application interfaces. Ph.D. thesis, Technische Universität Dresden, Dresden, Germany (2006) Kost, S.: Dynamically generated multi-modal application interfaces. Ph.D. thesis, Technische Universität Dresden, Dresden, Germany (2006)
36.
Zurück zum Zitat Kraus, M., Völkel, T., Weber, G.: An off-screen model for tactile graphical user interfaces. In: K. Miesenberger, J. Klaus, W. Zagler, A. Karshmer (eds.) Computers Helping People with Special Needs, Lecture Notes in Computer Science, vol. 5105, pp. 865–872. Springer, Berlin/Heidelberg (2008) Kraus, M., Völkel, T., Weber, G.: An off-screen model for tactile graphical user interfaces. In: K. Miesenberger, J. Klaus, W. Zagler, A. Karshmer (eds.) Computers Helping People with Special Needs, Lecture Notes in Computer Science, vol. 5105, pp. 865–872. Springer, Berlin/Heidelberg (2008)
37.
Zurück zum Zitat Kurniawan, S.H., Sutcliffe, A.G.: Mental models of blind users in the Windows environment. In: K. Miesenberger, J. Klaus, W. Zagler (eds.) Computers Helping People with Special Needs, Lecture Notes in Computer Science, vol. 2398, pp. 373–386. Springer, Berlin/Heidelberg (2002) Kurniawan, S.H., Sutcliffe, A.G.: Mental models of blind users in the Windows environment. In: K. Miesenberger, J. Klaus, W. Zagler (eds.) Computers Helping People with Special Needs, Lecture Notes in Computer Science, vol. 2398, pp. 373–386. Springer, Berlin/Heidelberg (2002)
38.
Zurück zum Zitat Kurniawan, S.H., Sutcliffe, A.G., Blenkhorn, P.L.: How blind users’ mental models affect their perceived usability of an unfamiliar screen reader. In: M. Rauterberg, M. Menozzi, J. Wesson (eds.) Human-Computer Interaction INTERACT ’03, pp. 631–638. IOS Press, Brussels (2003) Kurniawan, S.H., Sutcliffe, A.G., Blenkhorn, P.L.: How blind users’ mental models affect their perceived usability of an unfamiliar screen reader. In: M. Rauterberg, M. Menozzi, J. Wesson (eds.) Human-Computer Interaction INTERACT ’03, pp. 631–638. IOS Press, Brussels (2003)
39.
Zurück zum Zitat Laberge-Nadeau, C.: Wireless telephones and the risk of road crashes. Accid. Anal. Prev. 35(5), 649–660 (2003)CrossRef Laberge-Nadeau, C.: Wireless telephones and the risk of road crashes. Accid. Anal. Prev. 35(5), 649–660 (2003)CrossRef
40.
Zurück zum Zitat Lauridsen, O.: Abstract specification of user interfaces. In: Conference companion on Human factors in computing systems, CHI ’95, pp. 147–148. ACM (1995) Lauridsen, O.: Abstract specification of user interfaces. In: Conference companion on Human factors in computing systems, CHI ’95, pp. 147–148. ACM (1995)
41.
Zurück zum Zitat Lévesque, V.: Blindness, technology and haptics. Technical Report TR-CIM-05.08, McGill University, Centre for Intelligent Machines, Haptics Laboratory (2008) Lévesque, V.: Blindness, technology and haptics. Technical Report TR-CIM-05.08, McGill University, Centre for Intelligent Machines, Haptics Laboratory (2008)
42.
Zurück zum Zitat Limbourg, Q., Vanderdonckt, J.: UsiXML: A user interface description language supporting multiple levels of independence. In: Matera M., Comai S. (eds.) Engineering Advanced Web Applications, pp. 325–338. Rinton Press, Paramus (2004) Limbourg, Q., Vanderdonckt, J.: UsiXML: A user interface description language supporting multiple levels of independence. In: Matera M., Comai S. (eds.) Engineering Advanced Web Applications, pp. 325–338. Rinton Press, Paramus (2004)
43.
Zurück zum Zitat Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon, L., Florins, M., Trevisan, D.: UsiXML: A user interface description language for context-sensitive user interfaces. In: K. Luyten, M. Abrams, J. Vanderdonckt, Q. Limbourg (eds.) Proceedings of the ACM AVI’2004 Workshop ”Developing User Interfaces with XML: Advances on User Interface Description Languages”, pp. 55–62 (2004) Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon, L., Florins, M., Trevisan, D.: UsiXML: A user interface description language for context-sensitive user interfaces. In: K. Luyten, M. Abrams, J. Vanderdonckt, Q. Limbourg (eds.) Proceedings of the ACM AVI’2004 Workshop ”Developing User Interfaces with XML: Advances on User Interface Description Languages”, pp. 55–62 (2004)
44.
Zurück zum Zitat Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon, L., López-Jaquero, V.: UsiXML: A language supporting multi-path development of user interfaces. In: R. Bastide, P. Palanque, J. Roth (eds.) Engineering Human Computer Interaction and Interactive Systems, Lecture Notes in Computer Science, vol. 3425, pp. 134–135. Springer, Berlin/Heidelberg (2005) Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon, L., López-Jaquero, V.: UsiXML: A language supporting multi-path development of user interfaces. In: R. Bastide, P. Palanque, J. Roth (eds.) Engineering Human Computer Interaction and Interactive Systems, Lecture Notes in Computer Science, vol. 3425, pp. 134–135. Springer, Berlin/Heidelberg (2005)
45.
Zurück zum Zitat de Melo, G., Honold, F., Weber, M., Poguntke, M., Berton, A.: Towards a flexible ui model for automotive human-machine interaction. In: Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’09, pp. 47–50. ACM (2009) de Melo, G., Honold, F., Weber, M., Poguntke, M., Berton, A.: Towards a flexible ui model for automotive human-machine interaction. In: Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’09, pp. 47–50. ACM (2009)
46.
Zurück zum Zitat Morley, S.: Window Concepts: An Introductory Guide for Visually Disabled Users. Royal National Institute for the Blind, London (1995) Morley, S.: Window Concepts: An Introductory Guide for Visually Disabled Users. Royal National Institute for the Blind, London (1995)
47.
Zurück zum Zitat Mynatt, E.D.: Transforming graphical interfaces into auditory interfaces for blind users. Hum.-Comput. Interact. 12(1), 7–45 (1997)CrossRef Mynatt, E.D.: Transforming graphical interfaces into auditory interfaces for blind users. Hum.-Comput. Interact. 12(1), 7–45 (1997)CrossRef
48.
Zurück zum Zitat Mynatt, E.D., Edwards, W.K.: Mapping guis to auditory interfaces. In: Proceedings of the 5th Annual ACM Symposium on User Interface Software and Technology, UIST ’92, pp. 61–70. ACM (1992) Mynatt, E.D., Edwards, W.K.: Mapping guis to auditory interfaces. In: Proceedings of the 5th Annual ACM Symposium on User Interface Software and Technology, UIST ’92, pp. 61–70. ACM (1992)
49.
Zurück zum Zitat Mynatt, E.D., Weber, G.: Nonvisual presentation of graphical user interfaces: contrasting two approaches. In: Proceedings of the SIGCHI Conference on Human factors in Computing Systems: Celebrating Interdependence, CHI ’94, pp. 166–172. ACM (1994) Mynatt, E.D., Weber, G.: Nonvisual presentation of graphical user interfaces: contrasting two approaches. In: Proceedings of the SIGCHI Conference on Human factors in Computing Systems: Celebrating Interdependence, CHI ’94, pp. 166–172. ACM (1994)
50.
Zurück zum Zitat Newell, A.F.: CHI for everyone. Interfaces 35, 4–5 (1997) Newell, A.F.: CHI for everyone. Interfaces 35, 4–5 (1997)
51.
Zurück zum Zitat Nigay, L., Coutaz, J.: A design space for multimodal systems: concurrent processing and data fusion. In: Proceedings of the INTERACT ’93 and CHI ’93 Conference on Human Factors in Computing Systems, CHI ’93, pp. 172–178. ACM (1993) Nigay, L., Coutaz, J.: A design space for multimodal systems: concurrent processing and data fusion. In: Proceedings of the INTERACT ’93 and CHI ’93 Conference on Human Factors in Computing Systems, CHI ’93, pp. 172–178. ACM (1993)
52.
Zurück zum Zitat Parnas, D.L.: On the criteria to be used in decomposing systems into modules. Commun. ACM 15(12), 1053–1058 (1972)CrossRef Parnas, D.L.: On the criteria to be used in decomposing systems into modules. Commun. ACM 15(12), 1053–1058 (1972)CrossRef
53.
Zurück zum Zitat Pontelli, E., Gillan, D., Xiong, W., Saad, E., Gupta, G., Karshmer, A.I.: Navigation of HTML tables, frames, and XML fragments. In: Proceedings of the Fifth International ACM Conference on Assistive Technologies, Assets ’02, pp. 25–32. ACM (2002) Pontelli, E., Gillan, D., Xiong, W., Saad, E., Gupta, G., Karshmer, A.I.: Navigation of HTML tables, frames, and XML fragments. In: Proceedings of the Fifth International ACM Conference on Assistive Technologies, Assets ’02, pp. 25–32. ACM (2002)
54.
Zurück zum Zitat Rose, D., Stegmaier, S., Reina, G., Weiskopf, D., Ertl, T.: Non-invasive adaptation of black-box user interfaces. In: Proceedings of the Fourth Australasian User Interface Conference on User Interfaces 2003—vol. 18, AUIC ’03, pp. 19–24. Australian Computer Society, Inc. (2003) Rose, D., Stegmaier, S., Reina, G., Weiskopf, D., Ertl, T.: Non-invasive adaptation of black-box user interfaces. In: Proceedings of the Fourth Australasian User Interface Conference on User Interfaces 2003—vol. 18, AUIC ’03, pp. 19–24. Australian Computer Society, Inc. (2003)
55.
Zurück zum Zitat Sacks, O.: The Mind’s Eye: What The Blind See. The New Yorker, New York, pp. 48–59 (2003) Sacks, O.: The Mind’s Eye: What The Blind See. The New Yorker, New York, pp. 48–59 (2003)
56.
Zurück zum Zitat Sadato, N., Pascual-Leone, A., Grafman, J., Deiber, M.P., Ibañez, V., Hallett, M.: Neural networks for braille reading by the blind. Brain 121, 1213–1229 (1998)CrossRef Sadato, N., Pascual-Leone, A., Grafman, J., Deiber, M.P., Ibañez, V., Hallett, M.: Neural networks for braille reading by the blind. Brain 121, 1213–1229 (1998)CrossRef
57.
Zurück zum Zitat Sadato, N., Pascual-Leone, A., Grafman, J., Ibañez, V., Deiber, M.P., Dold, G., Hallett, M.: Activation of the primary visual cortex by braille reading in blind subjects. Nature 380(6574), 526–528 (1996)CrossRef Sadato, N., Pascual-Leone, A., Grafman, J., Ibañez, V., Deiber, M.P., Dold, G., Hallett, M.: Activation of the primary visual cortex by braille reading in blind subjects. Nature 380(6574), 526–528 (1996)CrossRef
58.
Zurück zum Zitat Savidis, A., Stephanidis, C.: Building non-visual interaction through the development of the rooms metaphor. In: Conference Companion on Human Factors in Computing Systems, CHI ’95, pp. 244–245. ACM (1995) Savidis, A., Stephanidis, C.: Building non-visual interaction through the development of the rooms metaphor. In: Conference Companion on Human Factors in Computing Systems, CHI ’95, pp. 244–245. ACM (1995)
59.
Zurück zum Zitat Savidis, A., Stephanidis, C.: Developing dual user interfaces for integrating blind and sighted users: the HOMER UIMS. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’95, pp. 106–113. ACM Press/Addison-Wesley Publishing Co. (1995) Savidis, A., Stephanidis, C.: Developing dual user interfaces for integrating blind and sighted users: the HOMER UIMS. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’95, pp. 106–113. ACM Press/Addison-Wesley Publishing Co. (1995)
60.
Zurück zum Zitat Savidis, A., Stephanidis, C.: The HOMER UIMS for dual user interface development: fusing visual and non-visual interactions. Interact. Comput. 11(2), 173–209 (1998)CrossRef Savidis, A., Stephanidis, C.: The HOMER UIMS for dual user interface development: fusing visual and non-visual interactions. Interact. Comput. 11(2), 173–209 (1998)CrossRef
61.
Zurück zum Zitat Seybold, J.: Xerox’s ”star”. The Seybold Report 10(16) (1981) Seybold, J.: Xerox’s ”star”. The Seybold Report 10(16) (1981)
62.
Zurück zum Zitat Smith, D.C., Harslem, E.F., Irby, C.H., Kimball, E.B., Verplank, W.L.: Designing the Star User Interface. BYTE, pp. 242–282 (1982) Smith, D.C., Harslem, E.F., Irby, C.H., Kimball, E.B., Verplank, W.L.: Designing the Star User Interface. BYTE, pp. 242–282 (1982)
63.
Zurück zum Zitat Souchon, N., Venderdonckt, J.: A review of XML-compliant user interface description languages. In: Jorge, J.A., Jardim Nunes, N., Falcāo e Cunha, J. (eds.) Interactive Systems. Design, Specification, and Verification, Lecture Notes in Computer Science, vol. 2844, pp. 391–401. Springer Berlin / Heidelberg (2003) Souchon, N., Venderdonckt, J.: A review of XML-compliant user interface description languages. In: Jorge, J.A., Jardim Nunes, N., Falcāo e Cunha, J. (eds.) Interactive Systems. Design, Specification, and Verification, Lecture Notes in Computer Science, vol. 2844, pp. 391–401. Springer Berlin / Heidelberg (2003)
64.
Zurück zum Zitat Stephanidis, C., Savidis, A.: Universal access in the information society: Methods, tools, and interaction technologies. Univ. Access Inf. Soc. 1(1), 40–55 (2001) Stephanidis, C., Savidis, A.: Universal access in the information society: Methods, tools, and interaction technologies. Univ. Access Inf. Soc. 1(1), 40–55 (2001)
65.
Zurück zum Zitat Sun Microsystems: GNOME 2.0 desktop: Developing with the accessibility framework. Technical report, Sun Microsystems (2003) Sun Microsystems: GNOME 2.0 desktop: Developing with the accessibility framework. Technical report, Sun Microsystems (2003)
66.
Zurück zum Zitat Theofanos, M.F., Redish, J.G.: Bridging the gap: between accessibility and usability. Interactions 10(6), 36–51 (2003)CrossRef Theofanos, M.F., Redish, J.G.: Bridging the gap: between accessibility and usability. Interactions 10(6), 36–51 (2003)CrossRef
67.
Zurück zum Zitat Trewin, S., Zimmermann, G., Vanderheiden, G.: Abstract user interface representations: how well do they support universal access? In: Proceedings of the 2003 conference on Universal usability, CUU ’03, pp. 77–84. ACM (2003) Trewin, S., Zimmermann, G., Vanderheiden, G.: Abstract user interface representations: how well do they support universal access? In: Proceedings of the 2003 conference on Universal usability, CUU ’03, pp. 77–84. ACM (2003)
68.
Zurück zum Zitat Trewin, S., Zimmermann, G., Vanderheiden, G.: Abstract representations as a basis for usable user interfaces. Interact. Comput. 16(3), 477–506 (2004)CrossRef Trewin, S., Zimmermann, G., Vanderheiden, G.: Abstract representations as a basis for usable user interfaces. Interact. Comput. 16(3), 477–506 (2004)CrossRef
69.
Zurück zum Zitat Vanderdonckt, J., Limbourg, Q., Michotte, B., Bouillon, L., Trevisan, D., Florins, M.: UsiXML: a user interface description language for specifying multimodal user interfaces. In: WMI ’04: Proceedings of the W3C Workshop on Multimodal Interaction (2004) Vanderdonckt, J., Limbourg, Q., Michotte, B., Bouillon, L., Trevisan, D., Florins, M.: UsiXML: a user interface description language for specifying multimodal user interfaces. In: WMI ’04: Proceedings of the W3C Workshop on Multimodal Interaction (2004)
70.
Zurück zum Zitat Weber, G.: Programming for usability in nonvisual user interfaces. In: Proceedings of the third international ACM conference on Assistive technologies, Assets ’98, pp. 46–48. ACM (1998) Weber, G.: Programming for usability in nonvisual user interfaces. In: Proceedings of the third international ACM conference on Assistive technologies, Assets ’98, pp. 46–48. ACM (1998)
71.
Zurück zum Zitat Weber, G., Mager, R.: Non-visual user interfaces for X Windows. In: Proceedings of the 5th international conference on Computers helping people with special needs. Part II, pp. 459–468. R. Oldenbourg Verlag GmbH (1996) Weber, G., Mager, R.: Non-visual user interfaces for X Windows. In: Proceedings of the 5th international conference on Computers helping people with special needs. Part II, pp. 459–468. R. Oldenbourg Verlag GmbH (1996)
72.
Zurück zum Zitat Weber, G., Petrie, H., Kochanek, D., Morley, S.: Training blind people in the use of graphical user interfaces. In: W. Zagler, G. Busby, R. Wagner (eds.) Computers for Handicapped Persons, Lecture Notes in Computer Science, vol. 860, pp. 25–31. Springer, Berlin/Heidelberg (1994) Weber, G., Petrie, H., Kochanek, D., Morley, S.: Training blind people in the use of graphical user interfaces. In: W. Zagler, G. Busby, R. Wagner (eds.) Computers for Handicapped Persons, Lecture Notes in Computer Science, vol. 860, pp. 25–31. Springer, Berlin/Heidelberg (1994)
Metadaten
Titel
Equivalent representations of multimodal user interfaces
Runtime Reification of Abstract User Interface Descriptions
verfasst von
Kris Van Hees
Jan Engelen
Publikationsdatum
01.11.2013
Verlag
Springer Berlin Heidelberg
Erschienen in
Universal Access in the Information Society / Ausgabe 4/2013
Print ISSN: 1615-5289
Elektronische ISSN: 1615-5297
DOI
https://doi.org/10.1007/s10209-012-0282-z

Weitere Artikel der Ausgabe 4/2013

Universal Access in the Information Society 4/2013 Zur Ausgabe

Acknowledgement to Reviewers

Acknowledgement to reviewers for 2012

Premium Partner