Virtual Reality

Traditional simplification algorithms operate either in the main memory or in the disk space. In this paper, we propose a novel storage independent polygonal mesh simplification system (SIPMSS). The new system offers an advanced memory utilization scheme and efficient storage-independent primitives that achieve constant main memory footprint and excellent runtime efficiency. It not only provides a very flexible platform for both in-core and out-of-core simplification algorithms but also permits very efficient integration of various types of simplification algorithms. The results presented in the section of experiments further approve its effectiveness and usefulness for both in-core and out-of-core of simplification.

In order to get better rendering effect, we proposed a wavelet transform based image enhancement algorithm for real time multi-resolution texture mapping in this paper. In this algorithm, we proposed a concept of weighted pyramid. Texture images to be synthesized are weighted enhanced by wavelet transform. Comparing with original ones, the edges in this image is much clearer and the contrast of this image is more evident. And when dealing with the maps with their particular characteristics, we also proposed some method for algorithm improvement. By using this method, current multi-resolution texture mapping method will be much more improved. It can give us a more realistic and efficient rendering effect by using lower resolution image to replace higher resolution one.

In this paper, stereoscopic image visualization for teleoperated robots is studied. The use of stereo vision systems shows some difficulties: image disparity is the main problem. It affects the correct image fusion process by the brain. Disparity curve calculations are shown and a method to calibrate stereoscopic images for telerobotics is explained. Experiments have been carried out to obtain the limits for a proper image fusion. The use of stereoscopic image has been assessed to execute teleoperated robot guidance tasks. The performance of stereoscopic and monoscopic image is compared. Stereoscopic images have shown a guaranteed depth perception.

This paper describes various techniques of accessing objects hidden behind obstacles in virtual environments. We show some of well known interaction techniques that are suitable for usage in this case. Next, we introduce new Head Manipulator method and several aids that allow a user to change the visibility of objects. We propose several combinations of interaction techniques and discuss their suitability for the given task. Finally, we describe our experiments when testing these methods and then we conclude with the results from the experiments and from users’ observations.

This paper illustrates a novel registration method based on robust estimation of trifocal tensors using point and line correspondences synthetically. The proposed method distinguishes itself in following ways: Firstly, besides feature points, line segments are also used to calculate the needed trifocal tensors. The registration process can still be achieved even under poorly textured scenes. Secondly, to estimate trifocal tensors precisely, we use PROSAC instead of RANSAC algorithm to remove outliers. While improving the accuracy of our system, PROSAC also reduces the computation complexity to a large degree.

This paper investigates methods of describing two-dimensional and three-dimensional scenes using eXtensible Mark-up Language (XML). It also investigates the initial development of a non-commercial object-based scene description language, for the effective capture of two- and three-dimensional scenes. Design science research techniques were applied to determine the ideal factors involved in effective scene capture. The solution facilitates the description of 2D and 3D environments via interpretation of object relationships; the implementation of the inheritance, functionality, interactions and behaviour.

High-quality mesh provides good rendering effects of virtual reality systems. In this paper a solid technique for generating triangle meshes optimized for real-time rendering in virtual environments is proposed. Modified weighted Voronoi barycenter relaxation is used to improve the mesh geometry. A simulated annealing algorithm is presented for optimizing vertex connec tivity. The shape deviations are effectively prevented, and the boundaries and features are well preserved. The advantage and robustness of this technique are verified by many examples.

Terrain rendering is essential to represent virtual environment in interactive applications. We have explored voxel-based terrain rendering using ray casting method. Real-time applications such as computer games and simulations require interactive fly-through or walk-through on the terrain. However, since previous approaches usually require a tremendous amount of CPU computation for view-frustum culling and level-of-detail selection, they are not suitable for those applications. We propose an accelerated terrain rendering method using GPU, which exploits a novel data structure, named

Cube Mesh

, for the acceleration of the voxel-based terrain rendering. In addition, we expanded the conventional PARC (Polygon Assisted Ray Casing) algorithm by applying our method to accelerate rendering performance of GPU- based terrain rendering method.

In this paper we present an illumination model for rendering realistic stained glass which simulates the phenomenon of stained glass in real world. The optics for stained glass involves three basic physical mechanisms: diffuse light and specular highlight, Fresnel refraction, volume absorption; these dominate the appearance of stained glass. In this paper, we combine them to render realistic stained glass. Our approach avoids performing any expensive geometric tests, and involves no pre-computation; both of which are common features in previous work. The rendered stained glass images achieve excellent realism.

Visual hull is intersection of cones made by back-projections of reference images. We introduce a real-time rendering method of jaggy-free visual hull on programmable graphics hardware. Visual hull can be rendered quickly by using texture mapping approach. However, jagged artifacts are present near the edges of visual hull faces. In this paper, we solve the problem by using silhouette information. Our implementation demonstrates high-quality rendering results in real-time. Time complexity of our algorithm is

$\mathcal{O}$

(N)

, where

N

is the number of reference images. Thus, the examples in this paper are rendered over one hundred of frames per second without jaggies.

Terrain texture plays a great role on visualization in many 3D applications. Sometimes terrain texture is difficult to obtain in a desired area, thus a new approach to terrain texture generation based on satellite images is proposed to resolve this problem. We prepare a silhouette map and a satellite image for the desired terrain texture and colorize the two pictures with the same pattern according to the terrain features. Then a method of texture synthesis is adopted to generate the final terrain texture.

A navigation system for virtual environments using low-quality HMD(head mounted display) must quantize images when the system presents true-color image with restricted number of colors. Such navigation system quantizes an image by using fixed palette. If the system represents an image by using a variable palette which is made considering a region around the viewpoint then user can perceive a virtual environments more vividly because human visual system is sensitive to the colors variation in the region around the viewpoint. In this paper we propose a color quantization algorithm that quantize a region around the viewpoint more finely than other regions at each variation of viewpoint for virtual environments navigation system and compose virtual environments navigation system using proposed algorithm. The system quantizes an image at each variation of viewpoint and shows a quantized image to user through HMD. We tested user preferences for our proposed system and the results show that users preferred our system.

Emerging applications in the area of Emergency Response and Disaster Management are increasingly demanding interactive capabilities to allow for the quick understanding of a critical situation, in particular in urban environments. A key component of these interactive simulations is how to recreate the behavior of a crowd in real- time while supporting individual behaviors. Crowds can often be unpredictable and present mixed behaviors such as panic or aggression, that can very rapidly change based on unexpected new elements introduced into the environment. We present preliminary research specifically oriented towards the simulation of large crowds for emergency response and rescue planning situations. Our approach uses a highly scalable architecture integrated with an efficient rendering architecture and an immersive visualization environment for interaction. In this environment, users can specify complex scenarios, “plug-in” crowd behavior algorithms, and interactively steer the simulation to analyze and evaluate multiple “what if” situations.

To preserve appearance attributes in a 3D model, a novel mesh simplification is proposed in this work based on the Quadric Error Metric (QEM). We make use of half-edge collapse method for mesh simplification and modify the QEM to solve the break between different texture coordinates. After analyzing several cases of the collapsing operation, a new formula on the cost of edge contraction is obtained to eliminate aberrances. Experimental results demonstrate that compared with other state-of-the-art techniques our algorithm can achieve satisfactory efficiency with desirable geometry and feature-preserving.

In direct-projected augmented reality, the visual patterns for compensation may distract users despite users would not be interested in the compensation process. The distraction becomes more serious for dynamic projection surface in which compensation and display should be done simultaneously. Recently, a complementary pattern-based method of efficiently hiding the compensation process from users’ view has been proposed. However, the method faced the tradeoff between the pattern imperceptibility and compensation accuracy. In this paper, we embed locally different strength of pattern images into different channels of the projector input images (AR images) after analyzing their spatial variation and color distribution. It is demonstrated that our content adaptive approach can significantly improve the imperceptibility of the patterns and produce better compensation results by comparing it with the previous approach through a variety of experiments and subjective evaluation.

The development of lower cost modular spatially immersive visualization systems based on commodity components and faceted display surfaces is described. Commodity computers are networked to form graphics computing clusters. Commodity digital projectors are used to form surrounding rear projected faceted display surfaces based on polyhedral shapes. The use of these systems in the design and evaluation of human environments is discussed.

In some collaborative manipulation activities – for example, medical experts making a diagnosis based on a 3D reconstruction of a human organ – remote participants may tailor their views on the shared object to their particular needs and task. This implies that each user has her viewpoint on the object. Awareness of viewpoint is then necessary both to coordinate each other and to understand remote users’ activities. This work investigates how to provide the remote viewpoint awareness in a 3D collaborative desktop in which multiple shared objects can be independently positioned and manipulated to accomplish a common single activity. Preliminary results of ergonomic evaluations of the proposed metaphors are also provided.

We propose a hybrid algorithm – (Parallel Search Algorithm) between PSO and simplex methods to approximate optimal solution for the Geometric Constraint problems. Locally, simplex is extended to reduce the number of infeasible solutions while solution quality is improved with an operation order search. Globally, PSO is employed to gain parallelization while solution diversity is maintained. Performance results on Geometric Constraint problems show that Parallel Search Algorithm outperforms existing techniques.

Realism is a key goal of most VR applications. As graphics computing power increases, new techniques are being developed to simulate important aspects of the human visual system, increasing the sense of ‘immersion‘ of a participant in a virtual environment. One aspect of the human visual system, depth cueing, requires accurate scene depth information in order to simulate. Yet many of the techniques for producing these effects require a trade-off between accuracy and performance, often resulting in specialized implementations that do not consider the need to integrate with other techniques or existing visualization systems. Our objective is to develop a new technique for generating depth based effects in real time as a post processing step performed on the GPU, and to provide a generic solution for integrating multiple depth dependent effects to enhance realism of the synthesized scenes. Using layered fog as an example, our new technique performs per pixel scene depth reconstruction accurately for the evaluation of fog integrals along line-of-sight. Requiring only the depth buffer from the rendering processing as input, our technique makes it easy to integrate into existing applications and uses the full power of the GPU to achieve real time frame rates.

To evaluate the effects of visually-induced motion sickness that induces symptoms related to the autonomic nervous activity, we proposed a new method for obtaining the physiological index

ρ

max

, which represents the maximum cross-correlation coefficient between blood pressure and heart rate, with measurement of neither continuous blood pressure nor ECG but using finger photoplethysmography only. In this study, a blood pressure-related parameter was obtained using the independent component analysis of finger photoplethysmography. Two experimental trials in which subjects performed the Valsalva maneuver and then they watched a swaying video image were carried out to evaluate the adequacy of the proposed method. The experimental results have shown that the proposed method worked successfully as well as the conventional method.

The paper introduces a novel methodology for the computer simulation of gaze behavior in net-based interactions. 38 female participants interacted using a special avatar platform, allowing for the real-time transmission of nonverbal behavior (head and body movement, gestures, gaze) as captured by motion trackers, eye tracking devices and data gloves. During interaction eye movement and gaze direction of one partner was substituted by computer simulated data. Simulated duration of directed gaze (looking into the face of the vis-a-vis) was varied lasting 2 seconds in one condition and 4 seconds in the other. Mutual person perception (impression ratings) and individual experience of social presence (short questionnaire) were measured as dependent variables. The results underline the validity of the computer animation approach. Consistent with the literature the longer gaze duration was found to cause significantly better evaluations of the interaction partners and higher levels of co-presence.

There are many visual display devices and input devices available to designers of immersive virtual environment (VE) applications. Most 3D interaction techniques, however, were designed using a particular combination of devices. The effects of migrating these techniques to different displays and input devices are not known. In this paper, we report on a series of studies designed to determine these effects. The studies show that while 3D interaction techniques are quite robust under some conditions, migration to different displays and input devices can cause serious usability problems in others. This implies that display-specific and/or device-specific versions of these techniques are necessary. In addition to the studies, we describe our display- and device-specific designs for two common 3D manipulation techniques.

Human-Computer Interaction (HCI) has mostly developed along two competing methodologies: direct manipulation and intelligent agents. Other possible but complementary methodologies are those of augmented cognition and affective computing and their adaptive combination. Augmented cognition harnesses computation to exploit explicit or implicit knowledge about context, mental state, and motivation for the user, while affective computing provides the means to recognize emotional intelligence and affects human-computer interfaces and interactions people are engaged with. Most HCI studies elicit emotions in relatively simple settings, whereas augmented cognition and affective computing include bodily (physical) embedded within mental (cognitive) and emotional events. Recognition of affective states currently focuses on their physical form (e.g., blinking or face distortions underlying human emotions) rather than implicit behavior and function (their impact on how the user employs the interface or communicates with others). Augmented cognition and affective computing are examined throughout this paper regarding design, implementation, and benefits. Towards that end we have designed an HCII interface that diagnoses and predicts whether the user was fatigued, confused, frustrated, momentarily distracted, or even alive through non-verbal information, namely paralanguage, in a virtual reality (VR) learning environment.

We present an efficient algorithm for haptic rendering of deformable bodies with highly detailed surface geometry using a fast contact handling algorithm. We exploit a layered deformable representation to augment the physically based deformation simulation with efficient collision detection, contact handling and interactive haptic force feedback.

Many Collaborative Virtual Environments (CVEs) have been developed to date. However, when focusing our attention on the way users perform their task in these systems, there is still little research on understanding the impact of different platforms on the collaboration experience. This paper describes not only a CVE, in this case, one that reproduces a block building game; but also an experiment that was carried out to evaluate both the interaction using different input and output technologies and the impact of collaboration on task performance and overall experience.

In this study, two types of marker-based tracking systems for Augmented Reality were developed: a tracking system using line markers and a tracking system using circular markers. Both markers were designed for use inside buildings such as nuclear power plants, which are very complicated and in which it is difficult to paste many large markers. To enlarge the area in which tracking is available using only a limited number of markers, a hybrid tracking method and a two-layer tracking method were also developed. The experimental evaluation shows that both methods can increase the available area compared to legacy methods such as single-camera tracking and simple square markers.

Typical shortest-path search algorithm, e.g. Dijkstra algorithm, is difficult to implement in VRML/X3D world directly due to the simplicity of VRML/X3D programming. By using JavaScript with good cross-platform and compatibility with VRML, this paper proposed an efficient back-traceable climbing (BTC) based navigation algorithm, by improving Hill Climbing search algorithm with the destination oriented guidance and loop removal, and amplifying it with simple data structure and flexible interfaces. The BTC based navigation algorithm performs greatly better than Dijkstra algorithm in terms of efficiency, consumed memory and the number of accessed nodes. It also possesses the merits of simplicity, easy implementation and reliability. Experimental results also show that it can provide real-time virtual navigation services with enough precision for large scale VRML/X3D environment.

This paper describes a design and implementation of a new handheld user interface framework called HIVE. HIVE provides the familiar 2D user interface on a mobile handheld computer to support user interactions in a virtual environment. It provides a scripting language based on XML and Lua to ease the development of handheld user interfaces and to increase the reusability of the interface components. This paper also discusses the use of the HIVE framework to develop a couple of handheld user interface applications for virtual environments to show usability.

We have investigated cybersickness in terms of image motion vectors, visual characteristics, and the autonomic nervous regulation. We obtained the RR interval, respiration, and blood pressure time-series and estimated the low-frequency (LF) and high-frequency (HF) power components to determine the some sensation intervals. Then, we traced the time-series of the LF component backwards to find out the local minimum as the onset. An experiment consisted of five consecutive exposure sessions of the same first-person-view video image. In the unpleasant group from fifteen healthy young subjects, the LF/HF increased with respect to the number of trials and a significant difference was confined between two groups. The trigger points concentrated around the specific segments. Within the unpleasant group, eyes did not follow the camera motion around the trigger points. Accordingly, it recommends to monitor image motion vectors as a trigger factor and autonomic nervous regulation as an accumulation factor for studying cybersickness.

The paper introduces an audio rendering system based on the binaural approach, which allows a real-time simulation of spatially distributed sound sources and, in addition to that, near-to-head sources in room-mounted virtual environments. We have been developing a dynamic crosstalk cancellation, allowing the listener to freely move around without wearing any headphones. The paper gives a comprehensive description of the system, concentrating on the dual dynamic crosstalk cancellation and aspects of the integration of a real-time room acoustical simulation. Finally, two applications are described to show the wide applicability of the system.

Hybrid display systems are those that combine different types of displays to exploit the complementary characteristics of the constituent display systems. In this paper, we introduce a hybrid system that combines a stereoscopic optical see-through head-mounted display (HMD) and a large projection display for an application in a multi-user ship painting training scenario. The proposed hybrid system’s projection display provides a large FOV and a physical metaphor to the ship surface with natural depth perception, while the HMD provides personal and unoccluded display of the motion training guides. To quantify its effectiveness, we conducted a human subject experiment, comparing the subject’s motion following task performance among three different display systems: large projection display, head-mounted display, and hybrid. The preliminary results obtained from the experiment has shown that given the same FOV, the hybrid system performed, despite problems with registration between the real and virtual worlds, up to par with the head-mounted display, and better than the projection display. Thus, it is expected that the hybrid display will result in higher task performance with the larger FOV factor available.

Subjects in the experiment reported observed the same spatial layout in the rectangular room and the cylindrical room from the exocentric ( 45° ) perspective first and then the egocentric ( 0° ) perspective. The mental representations of space were testified by the judgment of relative direction between objects. The results showed that subjects represented the spatial horizontal relation more accurately along the imagined direction that paralleled to the wall in the rectangular room but along the imagined direction that was ever faced in the cylindrical room. The rectangular room better facilitated the coding of spatial vertical information than the cylindrical room. Subjects could respond faster when retrieving the spatial relations in the direction faced during the observation. The data indicated that the orientation-specific representation was constructed and the environmental geometry could influence the accuracy of spatial direction in mind.

It is proposed that the concept of transparent telepresence can be closely approached through high fidelity technological mediation. It is argued that the matching of the system capabilities to those of the human user will yield a strong sense of immersion and presence at a remote site. Some applications of such a system are noted. The concept is explained and critical system elements are described together with an overview of some of the necessary system specifications.

This paper highlights some of the current problems in CVE development. These problems are mainly due to a lack of a good interaction model guiding the behaviour of the environments. This paper introduces the definition of a model based on the idea of the

interaction views

. The key concepts for understanding

interaction views

are also given during the explanation of the model. The paper also describes a reference architecture based on this model which can be found useful for the design of modelling tools, and a prototype application that helps understanding both the architecture and the model.

This paper proposed a new visual tag system for enhancing real world media interaction using handheld camera devices. This paper also described performance evaluations of the prototype, and its applications. C-Band is based on a ring with a color pattern code. A C-Band tag can provide a self-contained URL, and is flexible enough to allow various aesthetic designs for the tag’s surface. Furthermore, the tag’s structure is useful for building interactive techniques. Taken together, these features suggest that C-Band is an effective method to build various attractive camera-based media interactions.

This paper proposes a framework for VR application based on 3 different kinds of hierarchical structures for spatial, temporal, and semantic relationship. To achieve this, we incorporate scene graph, time graph, and ontology into the framework. This approach will enable to integrate synthetic space, real space, and knowledge seamlessly as a new way of developing immersive tangible space.

In this paper we describe our experiences and lessons learned from two user studies about the effects of short-term memory on data analysis tasks with graphical user interface and virtual environment. Visualization is a powerful tool for representing scientific data for analysis purpose. Current research progress enables us to create a high fidelity visual representation of scientific data. However, the value of traditional graphical user interface and Virtual Reality (VR) as human computer interface in data analysis via visualization tool is questioned by the domain scientists. We carried out two user studies that asked users to perform data analysis tasks. The first user study compared the user performance on selecting and manipulating transfer function (TF) with different graphical user interfaces in volume visualization to search certain structures within a 3D data set. The second user study investigated the performance difference of immersive VR, fish tank VR and haptic-based fish tank VR systems on four generic visualization tasks. The two user studies indicates that short-term memory plays an important role in 3D user interaction during data analysis task. The pitfall of 3D user interface design for scientific visualization is that too many efforts have been put into the interface itself or technology, rather than designing user interface to match or amplify human capabilities, for instance the limit amount of short term memory and its importance in data analysis process.

Navigation is a fundamental task in virtual environments. Some of the metaphors that are used to determine where users want to be placed at each moment are based on physical locomotion. Usually, these techniques require the use of several sensors and complex algorithms. In this paper, we present Super-Feet, a wireless navigation system controlled by natural movements without requiring invasive methods or devices. It uses a low-cost infra-red system to detect the instantaneous position of both feet and this information is translated into a walking and rotation velocity in the environment. Preliminary results show that Super-Feet navigation is more accurate but slower than joypad based navigation systems.

In stereo vision, the measurement of depth perception remains an enigmatic question. On the basis of geometric optics, numerous hypothesis and mathematical formulations have been proposed for explaining the mechanism of depth perception responsible of vertical exaggeration, although none of them have shown to be entirely satisfactory. We believe that stereo vision is not merely determined by geometry elements but also by mental factors that interact through a psychophysical process. Our initial observations refer to stereo-drawings that suggest a logarithmic correlation between real depth (stimulus) and perceptual depth (sensation), according to the psychophysical Fechner’s law. On this new approach, experimental data is being obtained in order to determine the Fechner’s constant for the sensorial modality of stereo vision, and to establish the logarithmic equation that rules the stereoscopic relationships between real and sensorial space. This equation might involve possibilities for technological developments, and it is the next step of our current research.

In the context of the optimization of the mechanical platform of a virtual reality system involving a haptic device, this paper introduces two tools in order to help the designer for obtaining the best positioning of the device respect to the application workspace. With this purpose we have defined a measure called Average Volumetric Manipulability, of how the application workspace fits in with the volume where haptic device provides its best performance. Also, we have defined other measure called Useful Manipulability which takes in account the frequency with which each zone of the application workspace is visited during the simulation process. The practical use of these measures is demonstrated using them during the design and development of a real application.

In this paper we present evaluation results of an innovative application designed to make collaborative design review in the architectural and automotive domain more effective. Within IMPROVE, a European research project in the area of advanced displays, we are combining high resolution multi-tile displays, TabletPCs and head-mounted displays with innovative 2D and 3D Interaction Paradigms to better support collaborative mobile mixed reality design reviews. Our research and development is motivated by application scenarios in the automotive domain involving FIAT Elasis from Naples, Italy and in the architectural domain involving Page/Park architects from Glasgow, Scotland. User evaluation took place at Glasgow (UK), Naples (ITA) and Darmstadt (GER), where we tested the integrated IMPROVE prototype application. The tests were conducted based on several heuristics such as ergonomics and psychomotorial factors and they were conducted based on guidelines recommended by ISO 9241 to verify whether the developed interfaces were suitable for the applications scenarios. Evaluation results show that there is a strong demand for more interactive design review systems, allowing users greater flexibility and greater choice of input and visualization modalities as well as their combination.

This paper reports an experiment conducted to study the effects of changing scene color inside a virtual environment on the rated levels of nausea among sixty-four viewers. Current theory on visually induced motion sickness suggests that changing the color of dynamically moving visual stimuli, while keeping everything equals, will not affect the rated sickness symptoms of the viewers. Interestingly, a recent study by another authors reported that color do affect levels of visually induced motion sickness. Preliminary results of this experiment suggest that while exposure duration to the visual stimuli significantly increased the rated levels of nausea and simulator sickness questionnaire scores (

p

<

0.001, ANOVAs)

, changes of color did not affect the levels of sickness. Reasons for the conflicting results are discussed in the paper.

The present study examined the effect of global motion combinations on visually induced motion sickness, presenting global motion analyzed from the video movies that induced the VIMS incident in Japan in 2003. The stimulus movie was presented on LC displays, whose size was either 20 or 37 inch with viewing distance of one meter. The results showed that when the extent of pitch and yaw motion was reduced to one third of from the original motion of the movie had larger effect than when other types of motion was reduced the same ratio of extent. This may be related to the fact that the original movie included larger amount of yaw motion. Moreover, we found that the lager display pro-duced larger total scores, and also larger “Disorientation” sub-socres, of SSQ than the smaller display. This may suggest the discomfort for the larger and smaller displays are produced by mainly different mechanisms.

This paper sums up the previous research on Head-Related Transfer Function (HRTF) individualization for three-dimensional (3D) virtual auditory display. Papers which involve experiment research, theoretical computation research, and applications of 3D virtual auditory display based on HRTFs have been reviewed. The necessity of HRTF individualization is discussed based on the review of previous experimental comparison between non-individual and individual HRTFs. At last the seven potential methods of individualization of HRTFs are summarized.

Most of facial expression recognition methods generally use single feature extraction method currently. These methods can not extract effective features for each feature area. A method of facial expression recognition based on hybrid features and fusing discrete HMMs is presented in this paper. In this method, texture feature for the eye area is extracted by using Gabor wavelet transformation, and shape variety feature for the mouth area is extracted by using AAM. In the process of recognition, discrete HMM is adopted for expression recognition in each expression area respectively. The recognition results are fused by means of integrating the probability of each expression in each area with its weight obtained by contribution analysis algorithm, and the final expression is determined as which with the maximal probability. Experiments show that our method can get high recognition rate.

In the design process of the external form of a product, sketching is applied for creating and embodying idea. Sketches are able to reflect the aesthetic sense of designers the most easily, making them the most efficient design tool. Sketches can be classified into thumbnail sketches, rough sketches, and rendering sketches according to the embodied idea. Rough sketches are used for developing the idea. In this paper, a system to support the rough sketching process and evaluate the designed product shape using 3D models constructed from the rough sketch is described. This system has a function for extracting and generating sketch lines drawn by a designer, a function for interactively modifying sketch lines, and a function for constructing 3D models from rough sketches. In order to support rough sketching, algorithms which automatically extract/generate the desired sketch-lines are proposed and implemented as a system. A method to construct 3D models form sketch lines is also proposed and implemented as a system.

The field of computer graphics is greatly increasing its overall performance enabling consequently the implementation of most of the product design process phases into virtual environments. The deriving benefits of using virtual practices in product development have been proved intrinsically highly valuable, since they foster the reduction of time to market, process uncertainty, and the translation of most prototyping activities into the virtual environment. In this paper we present the developed platform in mixed environment for ergonomic validation. Specifically we defined a methodology for testing both aspects related to design and ergonomic validation by allowing the tester to interact visually and physically with the car dashboard control devices and related interface by the mean of a rotatory haptic device. By experimental session it has been highlighted that it is possible gathering qualitative data about the design, and finding typical occlusion problems, but also quantitative data can be collected by testing the infotainment interface and the consequent users’ distraction during the device use.

This paper presents some results of a research project aiming at developing haptic tools for virtual shape modeling resembling the real tools like rakes and sandpaper used by modelers and designers in the real workshop. The developed system consists of a CAS (Computer Aided Styling) system enhanced with intuitive designer-oriented interaction tools and modalities. The system requirements have been defined on the basis of the observation of designers during their daily work and translating the way they model shapes using hands and craft tools into specifications for the modeling system based on haptic tools. The haptic tool and the interaction modality developed for exploring and sanding a surface is presented in the paper.

In emergency operations centers, operators are challenged with managing and analyzing massive amounts of information in a timely manner. With the advent of motion tracking systems and low-cost, high-performance graphics workstations, novel spatial augmented reality (AR) interfaces are becoming technologically feasible. AR interfaces have very unique characteristics as compared to other media and computer interfaces: users interact with the computer system through body motion in a volumetric space, instead of via a two-dimensional surface. We have created three prototype immersive AR computing interfaces for emergency operation center. User interacts with the information using a pair of motion tracked pinch gloves, and information is displayed through a head-tracked optical see-though head mounted display. The emergency operator uses intuitive grab and release gesture to move and manipulate the digital information in the environment analogous to interaction of everyday life objects.

Virtual and Augmented Reality have been widely used in many scientific fields for the last two decades in order to visualize complex data and information. Although both techniques are oriented to show users complex 3D environments by means of an intuitive and easy mechanism, they use to become useless to manipulate the information in an intuitive and realistic way. In this paper, we present SOROLLA, a new concept of workbench designed for virtual and augmented reality purposes and specially oriented to the fields of tele-education and engineering. Unlike other proposals, SOROLLA not only allows an easy utilization and configuration, but also shows a cost-effective immersive visualization system based on off-the-shelf elements. The initial results using our workbench and wall-system show that both efficiency and user satisfaction are higher than the ones obtained using conventional devices.

Test plays a very important role in manufacturing, but the general practice so far cannot fully meet the demands of flexibility and efficiency required by a fast-changing market. To supply a new and flexible approach to the solution of various complex engineering testing problems, the concept of “VR-based virtual test technology” was put forward in the authors’ previous works, which can be regarded as the combination of these three technologies: virtual reality (VR), test technology and computer simulation. This paper is the deepening of the authors’ previous works. After elucidating the theoretical model of VR-based virtual test technology, its application in instrument development is discussed, and finally a prototype VR-based instrument development system is given to show the initial effects of the approach. The conclusion is that VR-based virtual test technology can play a very important and active role in testing and instrument development.

Executing user-test of information appliances costs much due to fabrication of physical functional mockups, the securing of human subjects for the test, and preparation of testing facilities. Results of the user test have to be manually analyzed by usability engineers, and the analysis takes a couple of weeks. In this paper, we propose an integrated software and hardware environment for prototyping and assessing usability of information appliances where the users can operate a user-interface either of a 2D digital mockup or 3D digital mockup or a physical-mockup of the appliance under the test. This function enables engineers to test and evaluate the usability of the appliances both from the cognitive aspect and from the physical aspects.

We simulate the application of facial makeup in 3D by means of multi-sensory interaction which integrates a haptic interface with stereo visual perception. We aim to allow a user to simulate the facial makeup within an immersive virtual environment. To allow interactive 3D makeup process, we start by acquiring surface elements of a face which is enhanced by adjusting its texture and skin tone. Then the user can begin a virtual makeup session using 2D pixel-based paint model by grasping a haptic handle and looking at an auto-stereoscopic display. The use of this model avoids the texture distortion seen in existing 3D painting systems.

Visiting the outer space freely was the human imagination until recently. Virtual environments of the space will be the way for people to “visit” the space to study their motion and interrelationships. This paper assembles a scalable, immersive virtual environment system called the Virtual Space Environment Simulation System (VSESS) that allows users to visualize objects and physical phenomena in near-earth space and provides a brief sketch of the design thinking of such system. The software architecture of the VSESS is designed incorporate the various simulation, analysis and visualization elements of the system into a single, integrated environment. The system employs a VR system with double projector to build immense 3-D environment and applied to some actual space projects and they demonstrate the efficacy of the concept and the processing.

Simple and rapid prototyping is important for collaborative design where end users are involved in the design process from the early phase. This paper presents a collaborative design prototyping tool for hardware software integrated information appliances, such as mobile phones and digital cameras. It supports the collaboration between designers and users for identification of user needs and evaluation of design. The tool consists of STCtools (State Transition Chart tools) software, hardware modeling materials, physical user interface toolkits for software hardware integration, and Augmented Reality desk (ARdesk) for interactive simulation. STCtools allow users to prototype the behavioral aspect of information appliances using states and events. Using the hardware modeling materials and the physical user interface toolkit, designers and end-users explore hardware aspects together. ARdesk is a workbench where the interactive sketch display is projected onto an invisible paper marker of the hardware model using the projection based augmented reality technique. Three collaborative design workshops of designing a portable device for theme park users were accomplished with the prototyping tool for qualitative evaluation. The results suggested that the tool supported collaborative design by iterative concept development and active interactions between designers and end users.

This paper explores how a ‘learning‘ algorithm can be added to UGV’s by giving it the ability to test the terrain through ‘feeling‘ using incorporated sensors, which would in turn increase its situational awareness. Once the conditions are measured the system will log the results and a database can be built up of terrain types and their properties (terrain classification), therefore when it comes to operating autonomously in an unknown, unpredictable environment, the vehicle will be able to cope by identifying the terrain and situation and then decide on the best and most efficient way to travel over it by making adjustments, which would greatly improve the vehicles ability to operate autonomously.

Virtual reality (VR) based skill training simulators are highly interactive computer systems that allow for training of e.g. manipulation skills. In the most complex domains, such as in surgery, teaching correct manipulation requires expert’s judgement and instructions. Video annotation has enabled presentation of explicit principles of manipulation that are highlighted on a recorded demonstration of an expert. In VR, similar annotation methods have been applied even though interactivity of VR allows for novel kinds of representations of annotated data. This paper presents an annotation model and an interface that uses interactivity of VR as an advantage on reading the annotations, which has potential to convey the principles of manipulation through multiple modalities to novices. The design of the model is based on behavioural parameters of the simulation – features which cannot be accessed in the real world. This approach has potential to enhance efficiency of VR simulator based skill training.

The Aircraft maintenance and inspection is a complex system wherein humans play a key role in ensuring the worthiness of the aircraft. Traditional visual inspection training consisted mainly of on-the-job training (OJT). While OJT provides novice inspectors with the hands-on experience critical to effective transfer, it lacks the ability to provide real-time feedback and exposure to various scenarios in which to inspect.

With advances in technology, computer simulators have been developed to train the novice inspector and reduce the learning curve inherent with transitioning from the classroom to the workforce. Advances in graphics and virtual reality (VR) technology have allowed for an increase the sense of involvement in using these simulators. Though these simulators are effective, their deployment in aircraft maintenance training schools is limited by the high cost of VR equipment. This research investigates the effectiveness of different interaction devices for providing projector based simulated aircraft maintenance inspection training.

In this paper we introduce an innovative application designed to make collaborative design review in the architectural and automotive domain more effective. For this purpose we present a system architecture which combines variety of visualization displays such as high resolution multi-tile displays, TabletPCs and head-mounted displays with innovative 2D and 3D Interaction Paradigms to better support collabo rative mobile mixed reality design reviews. Our research and development is motivated by two use scenarios: automotive and architectural design review involving real users from Page\Park architects and FIAT Elasis. Our activities are supported by the EU IST project IMPROVE aimed at developing advanced display techniques, fostering activities in the areas of: optical see-through HMD development using unique OLED technology, marker-less optical tracking, mixed reality rendering, image calibration for large tiled displays, collaborative tablet-based and projection wall oriented interaction and stereoscopic video streaming for mobile users. The paper gives an overview of the hardware and software developments within IMPROVE and concludes with results from first user tests.

In the VTT PLAMOS (Plant Model Services for Mobile Process Maintenance Engineer) project new tools were developed for modern maintenance work carried out in industrial plants by either the plant personnel or personnel of an industrial service provider. To formulate the requirements for new tools the work of a maintenance man was studied with a particular method, the Core-Task Analysis which has its roots in study and development of work in complex settings. The aim was to develop and create concepts for novel tools that would support the development of good work practices in a situation where the work is concurrently undergoing several transformations. Hence, the new tools should have potential to enable and affect new ways of working.

In development of a handy information device, for example, a smart phone, the repetition of prototyping, evaluating and correcting are indispensable for ensuring stability of grasp, operationality and non-fragility. Because those properties are not able to be evaluated by common digital product development systems. In addition, operationality of these products greatly changes by shape of product, arrangement of buttons, output of implemented programmes etc. However, there is no system that makes the user evaluate efficiently and totally such a product. Therefore, the goal of this research is to develop the augmented reality system with the tangible interface that can tactually present the outline shape of the product model for efficiently product development.

Emerging augmented reality and tangible user interface techniques offer great opportunities towards delivering rich, interactive prototypes in product development. However, as most of these are evaluated outside the complexity of. Design practice, little is known about the impact of these prototypes on the resulting product or the process. As a part of a larger multiple-case study approach, this study attempts to explore cues to characterize and improve the design practice of information appliances by performing a retrospective case study. The development of a handheld digital oscilloscope was chosen as an exemplar, embodying complexity in both form giving, interaction and engineering aspects. Although some of the employed techniques have grown obsolete, reflection on this development project still forecasts interesting and useful issues that should be considered while developing new design support methods and techniques.

Driving simulator is an important facility for driving safety study. This paper introduced three well-know large-scale moving-base driving simulators and two fixed-base simulators. Driving simulator has a broad range of applications of driving safety study, from driver’s behavior study to vehicle device and technology study, the paper reviewed seven main research aspects including behavior study, driver education and training, transportation infrastructure, medicine and therapy, ergonomics, Intelligent Transportation System, and administrative method.

In this paper, an integrated virtual assembly simulation experiment environment of interactive workplace planning, assembly process simulation and ergonomics evaluation is built and it provides a new ways and means for interactive and visualized workplace design and process planning. A simple and effective method of measuring the movement manipulating data of operator is presented and can provide data source for ergonomics evaluation of the operator. In order to obtain these data in VE, movement data of the operator in real environment is transformed into VE and is related with virtual objects by the corresponding relation between VH and real hand (RH). Based on these data, the operator’s fatigue index and energy consumption can be calculated. The conformability and fatigue of users for repetitive assembly operations are quantitatively evaluated to reduce the health problems of operators because of unreasonable layout and repetitive work.

Massively Multiplayer Online Games (MMOGs) are no longer just games. “Metaverses,” a variant of these games, which include Second Life and Active Worlds, represent some of the most immersive, interactive possibilities for learning, simulation, and digital design in use today. They also increasingly blur distinctions between work and play, as well as user and designer, prompting questions about the nature and practice of virtual design, that is, design that is practiced inside virtual reality by and on 3D avatars. This paper describes results from a qualitative study of collaborative design practices within Second Life, a popular metaverse and design environment. We analyze design processes, including the artifacts that avatar-designer create and use in the design activities.

This paper introduces a concept of augmented reality haptics close to its vision equivalent, namely the blending of information between direct input, measured in the real world, with virtual ones, computed from a VR simulation, related to the explored real world. Hence, we propose blending functions enabling the transition of haptic features, applied for forces and for position switch. Finally, to demonstrate our concept, a tele-operative system dedicated to the tele-touch-diagnosis, is presented and used for scenarios of comparison between real and virtual objects as well as experimental results.

This paper presents a method for cultural heritage exhibition. A design scheme of multi-projection screens based digital storytelling system for the Grand Canal Museum in China is introduced. The task of this system is to exhibit a famous Chinese painting on a large display wall dynamically. In order to display this painting in a large display wall, the painting have to be repainted with very high resolution and segmented into many smaller tiles for being projected onto the large wall. Additionally, in order to make this static painting attractive and charming, the technology of digital storytelling is used. So a script called

water transportation story

is composed elaborately by us to show the magnificent scene along the Grand Canal.

Videodope

is an interactive digital environment that is meant to show the short-, medium- and long-run effects of psychoactive substances on the human organism. Its purpose is to increase the users’ knowledge of these effects, which is often based on misconceptions and legends, and to provide a feedback on their level of information, with preventive purposes. These kind of educational games have the advantage of being more persistent, ubiquitous, informative, anonymous and multi-medial than a normal lecture. However, they have to be perceived as credible and plausible to the user in order to be persuasive, especially since the target is constituted of young people familiar both with the technology and with the subject. In this paper, we will describe the process of designing and evaluating the videogame in the field, as well as the insights provided by the participants who took part to the evaluation.

In this paper, we apply augmented reality to provide pottery design experiences to the user. Augmented reality offers natural 3D interaction, a tangible interface, and integration into the real environment. In addition, certain modeling techniques impossible in the real world can be realized as well. Using our AR system, the user can create a pottery model by deforming a virtual pottery displayed on a marker with another marked held by the user’s hand. This interaction style allows the user to experience the traditional way of pottery making. Also provided are six interaction modes to facilitate the design process and an intuitive switching technique using occlusion-based interface. The AR pottery system can be used for virtual pottery prototyping and education.

This paper presents an optical see-through (OST) Augmented Reality (AR) system for the treatment of phobia to small animals. Our group has already presented two AR systems for this treatment. The first system was a video see-through (VST) AR system. The second one was a markerless AR system. And now, we present a third version using an OST approach. To check whether the OST AR system is more or less effective than the OST AR system, we have carried out an experiment comparing both systems. In this paper we focus on the anxiety level of participants during the experiment. Results show that both systems provoke anxiety in participants even if they do not have fear to cockroaches/spiders. OST AR system provokes slightly more anxiety than the VST AR system.

We summarize three evaluations of an educational augmented reality application for geometry education, which have been conducted in 2000, 2003 and 2005 respectively. Repeated formative evaluations with more than 100 students guided the redesign of the application and its user interface throughout the years. We present and discuss the results regarding usability and simulator sickness providing guidelines on how to design augmented reality applications utilizing head-mounted displays.

Virtual reality (VR) based system has an advantage to provide an experience which never be realized in the real world. In these years, medical training simulators are developed by VR-based system. The aim of medical VR training system is to teach the actual manipulation in surgery. At the same time, the most important feature is that the system can teach anatomical and physiological behaviors. Interactive learning by VR system is expected to motivate active learning effectively. Hence, this study aims to provide a game system which does not require completely accurate simulation for a teaching material. However, modeling of the whole body relation between anatomy and physiology is not simple. So far, conventional brain study defines the functional map on the surface of brain. Therefore, the authors propose self surgery game for learning the basis of brain by stimulation and reaction. The self surgery game provides an experience to treat oneself by oneself. The game system would support the subject to imagine the relation between actual behavior and brain function map by touching one’s own head.

Historic buildings are important to many aspects of human, but most buildings are not protected well. The natural and man-made calamities are major destruction. The destruction of natural and some man-made destruction are unable to avoid. The laws about the preservation of historic building are too passive, because they need a lot of time to establish identity. But before the historic buildings establish identity and between the official documents make a round trip, the buildings are destroyed constantly. To face such problem, we are necessary to suggest new preservation concepts to assist preservation of historic building. Due to development of computer, we can protect the data of historic building by digitization. In this case, I will use virtools to do an interface to help people to build the buildings in the virtual reality. Making everyone can protect the historic buildings by it.

Interactive 3D computer graphics technology is now extremely popular, seen in the increasing interest in and use of 3D digitization of cultural heritage contents. This paper introduces a digital interactive cultural heritage system which embeds Virtual-Reality (VR) technology within the Peranakan culture, the Peranakan culture being a Singaporean unique ethnic culture. A prototype has undergone the full developmental process of being implemented, tested, and evaluated. This paper will also discuss the results of the usability test.

Tangible Moyangsung is a collaborative virtual heritage environment that employs tangible interfaces to encourage group social interaction and cooperation. It is designed for a group of users to play fortification games for reconstructing a Korean war-defensive castle in virtual reality. This paper describes a historical background and cultural meanings of Moyangsung, and details in design, implementation and a user study of the Tangible Moyangsung system. The study results showed that collaborative work using tangible interfaces helped users to be more engaged in the task and learn cooperation to produce better performance.

In recent years, there has been an increasing interest in clinical and research applications of virtual reality (VR). However, the adoption of this approach is still limited by the high costs of software development, lack of technical expertise among end-users, and the difficulty of adapting the contents of the virtual environments (VEs). To address these needs, we have designed and developed NeuroVR, (http://www.neurovr.org), a cost-free virtual reality platform based on open-source software components. NeuroVR allows non-expert users to easily customize a VE by using a set of pre-designed virtual scenes, and to run them in an immersive or non-immersive modality. In this paper, we provide a description of the key functional features of the platform.

Showing pictures of famous artists to students is a fundamental task to teach fine art history. Images found on the Internet are a major resource of such pictures that can replace older albums of fine art collections. The colours reproduced in Internet images are, however, in many instances not correct. Images have been collected both from museums’ publications and from the Internet, and some critical areas of the images have been colorimetrically evaluated. Investigations have shown that in some databases the colours of the images are highly distorted. Before using pictures from one or the other database, one should compare the reproductions downloadable from different sources, and select – according to the aesthetic feeling of the user – the artefact that one can assume to come nearest to the original. It is a great pity that the artefacts in the databases are not accompanied with the necessary background information (metadata) that would provide some information on the taking illumination, the spectral sensitivity and gamma of the taking apparatus, and of any transformations the provider has performed. Without this information that is quite common in other areas of image databases, the user is unable to set the reproduction parameters of his or her equipment to get to a reasonable reproduction.

This paper discusses methods of implementing 3D simulation technology of the cultural relics, brings forward 3D simulation technology to implement high-quality, high-precision and vivid texture of the ancient cultural relics by adopting range laser 3D scanner and CCD cameras; suggests the automatic registry methods of 3D laser scanning data, which can make 3D laser scanning data with different viewpoints as well as different resolutions into the same coordinate, so as to achieve the high-precision 3D cultural relic models; proposes to take the reflectivity map of 3D laser scanner as middle reference to realize the accurate mapping of planar images and 3D laser scanning data, so as to obtain simulated cultural relics with vivid texture.

This paper describes a pilot study to investigate participants’ perceptions of usefulness and usability of our developed Plant Mixed Reality System (PMRS), designed for primary school children (11-12 years old). Preliminary results seemed to indicate participants’ intention to use PMRS for learning. The paper concludes with a discussion on how the findings were used to formulate a second study based on the Technology Acceptance Model, and discuss implications on intention to use and acceptance of mixed reality systems for education.

In this paper, we present a new immersive first-person shooting (FPS) game. Our system provides an intuitive way for the users to interact with the virtual world by physically moving around the real world and aiming freely with tangible objects. This encourages physical interaction between the players as they compete or collaborate with each other.

This paper presents a 3D plant morpha modeling method based on projection image, and analyzes the influence calibration error on the precision of reconstruction. Mini-section algorithm and Ball B-spline are used to extract the skeleton of the reconstructed image and to fit the meshes respectively. The experiment shows that this method is high-speed and robust, and the reconstructed root, branch, haulm that are columnar is very vivid,. So it indicates that we can achieve a simple and practical plant modeling method that transforms image of entities to surface meshes of plant. And it is helpful for plant morpha digitization and visualization in agriculture digital museum.