Skip to main content



Chapter 1. Introduction

Visual communications have existed as long as the history of mankind. People communicated through symbols and drawings long before spoken languages have been developed. Carrying specific meanings, or semantics, those symbols and drawings may be considered visual languages1, that serve effective communication purposes. In a broader sense, visual lan- guages refer to any kinds non-textual but visible human communication medias, including art, images, sign languages, maps, and charts, to name a few.

Chapter 2. The Foundation — Graph Grammars

In the implementation of textual languages, formal grammars are commonly used to facilitate the language understanding and the parser creation. When implementing a diagrammatic visual programming language (in the rest of the chapter, diagrammatic visual programming languages will simply be referred to as visual languages), this is not usually the case. A visual language requires a formal syntactic definition, which is indispensable for automatic analysis, transformation, and non-ambiguous expression. Graph grammars with their well-established theoretical background may be used as a natural and powerful syntax-definition formalism (Rozenberg 1997) and the parsing algorithm based on a graph grammar may be used to check the syntactical correctness and to interpret the language semantics.

Chapter 3. Spatial Specification

Rather than expressing sentences in sequences of characters, visual programming languages allow programs to be expressed in visual sentences in a multi-dimensional fashion. As noted by Reker and Schuir (1996), the physical layout and the meaning of a diagram are two important aspects of a visual sentence. A spatial relations graph (SRG) specifies spatial relationships between pictorial objects while an abstract syntax graph (ASG) provides structural information in a succinct form. Consequently in their approach, a visual sentence is specified through two intermediate graphs. One is geared toward visualization, and the other toward interpretation (Rekerand Schuir 1996).

Chapter 4. Multimedia Authoring and Presentation

With the rapid advance of the Internet and Web technology, an increasing amount of graphs and media contents are delivered on the Web. On-line multimedia presentations, such as news, need to be constantly updated. The content and the presentation structure of an on-line multimedia presentation may also be frequently updated. At the client side, there are various kinds of viewing conditions, such as varying screen size, style preference, and different device capabilities. For example, consider a diagram representing an organizational structure on the Web that may be of considerable complexity occupying a large screen space, and thus may be unsuitable for small displays (Marriott et al. 2002). Thus, if the diagram is to be viewed on the screen of a mobile device, such as a PDA (Personal Digital Assistant), the original diagram layout may not be appropriate. Another example is a news Web site, which generally needs to be constantly updated with the incoming news items. Such a site may have to adapt itself frequently to the changing space and style requirements for different news categories. The ability of dynamically adapting its layout would be highly desirable. There are also increasing demands for accessing on-line multimedia documents from mobile devices such as PDas.

Chapter 5. Data Interoperation

As more research disciplines and social sectors are becoming computerized, an increasing amount of observational data and documentation are digitized. To support interchange of these digital materials, encoding standards such as XML (extended Markup Language) (W3C 2004a) have been proposed for digital document markup. A document structure is realized by a set of element tags that can be used to delimit data items in a document of the specific domain. If all the delimiting tags are properly placed in a document, the document is said to be well-formed. However, whether an XML document is valid is determined by the Document Type Definition (DTD), a formal grammar for specifying the structure and permissible values of XML documents. The content of the document elements and their markups can be specified and validated by a schema language, such as the XML Schema (Thompson et al. 2000), RELAX (Makoto 2000), or SOX (Davidson 1999). The tag set of a specific domain is called the “vocabulary” of that domain. People of the same domain could use the same basic syntax, parsers, and assisting tools of the vocabulary. This opens a way for different types of document structures to be created to facilitate communications for various professional domains. Digital data is thus not only represented but also defined in different languages.

Chapter 6. Software Architecture Design

Software architecture and design (Shaw and Garlan 1995) are usually modeled and represented by informal diagrams, such as architecture diagrams and UML diagrams. While these graphic notations are easy to understand and are convenient to use, they are not amendable to automated verification and transformation. The developer has to rely on personal experience to discover errors and inconsistencies in the architecture/design diagrams. She also has to manually transform an architecture/design diagram while needed. These processes are tedious and error-prone. This chapter presents an approach that abstracts UML class diagrams and architecture styles into graph grammars. These grammars enable a high level of abstraction for the general organization of a class of software architectures, and form a basis for various analysis and transformations, in this approach, software verification is performed through a syntax analyzer. Architecture transformation is achieved by applying predefined transformation rules. In general, the presented approach facilitates the following aspects:
  • Graphs are used to specify software by distinguishing individual components and their relationships. Using graph grammars as design policies, the presented approach provides a powerful mechanism for syntactic checking and verification, which are not supported by most current tools.
  • In addition to software design and vérification, the presented approach facilitates a high level of software reuse by supporting the composition of design patterns, and uses graph rewriting techniques in assisting the transformation of software architectures and in reusing the existing products.

Chapter 7. Visual Web Engineering

The development of Web sites with complex interconnections of large number of Web pages so far has been largely an ad hoc process. There has been no commonly accepted methodology, which supports ease of design, navigation, and maintenance of sophisticated Web sites. As the number of Web sites is increasing in an exponential order, with the huge information space provided by the Web, users become increasingly confused when they navigate a growing number of Web sites; finding the right information also takes longer time. The problems are partially due to the unstructured nature of the current organization of Web sites. For example, in most of the existing Web browsers, the process of jumping from one location to another could easily confuse the user. The main reason for this is that the user does not know the current context of space with respect to the overall information space.

Chapter 8. Visual Language Generation

Implementing a visual programming language (VPL) is much harder than implementing a textual programming language (Myers 1990). VPLs are usually embedded and tightly integrated within visual environments. Consequently, they are often characterized by the attributes of the environments (Goldberg et al. 1994). The VPL implementation involves the implementation of a whole programming environment with a user interface which supports developing programs using a visual language. Notice that VPL interfaces are not the same as graphical user interfaces (GUIs) nor are they just for visualization. Traditional GUI development toolkits are inadequate for the creation of VPLs because they do not support syntactic and semantic specifications of visual programming. The graphical user interface of a visual language relates to the language’s syntax and semantics. The interaction (dialogue) between the interface, the syntax and the semantics must be maintained. Implementing a VPL interface and its support for syntactic and semantic specifications of visual programming suffers from a problem common to all large, complex software systems, i.e. the generation is difficult and time consuming. The remaining part of this section will address the typical problems of implementing VPLs.

Chapter 9. Conclusions and Future Perspective

The book has described visual programming languages and their applications in several important domains. We started with a general introduction to the concepts of visual languages and presented a theory behind such languages, i.e. a context-sensitive graph grammar formalism, known as the reserved graph grammar (RGG). The RGG was based but improved on the LGG (Rekers and Schürr 1997). By keeping the layer decomposition mechanism of the LGG to terminate parsing in finite steps, the RGG uses a marking technique with an embedding rule to solve the embedding problem. The rule ensures that the application of a production in the graph rewriting process would not create dangling edges. By ensuring selectionfree productions in the RGG, sometimes called confluent graph grammars elsewhere, a selection-free parsing algorithm (SFPA) attempts only one parsing path and thus achieves a polynomial time complexity. It is difficult to estimate how many types of diagrams could be specified by selectionfree productions. Having applied RGGs to many different application domains, we have not come across a diagram formalism that cannot be specified by selection-free productions.


Weitere Informationen

Premium Partner