Skip to main content

About this book

This book constitutes the refereed proceedings of the 37th Computer Graphics International Conference, CGI 2020, held in Geneva, Switzerland, in October 2020. The conference was held virtually.

The 43 full papers presented together with 3 short papers were carefully reviewed and selected from 189 submissions. The papers address topics such as: virtual reality; rendering and textures; augmented and mixed reality; video processing; image processing; fluid simulation and control; meshes and topology; visual simulation and aesthetics; human computer interaction; computer animation; geometric computing; robotics and vision; scientific visualization; and machine learning for graphics.

Table of Contents


CGI’20 Full Papers


Comparing Physical and Immersive VR Prototypes for Evaluation of an Industrial System User Interface

Since immersive VR devices have become commodities, immersive environments appear as a new tool in the development of high-fidelity prototypes of systems in which the user interaction relies on expensive or unusual hardware, e.g., industrial systems. However, there is not enough evidence that the interface of a complex system and its VR counterpart have equal usability and user experience qualities. Our main objective is to assess the feasibility of carrying out studies on user-based evaluation in industrial interactive systems through immersive VR simulation. To achieve this, we compared user assessment with a conventional prototype of an industrial system with its immersive VR simulation. We performed within-subjects user testing in both the physical and the VR setups, and collected (i) experimenters’ observations on usability issues and (ii) subjective and objective measures of 16 participants. Subjective measures were taken using standardized questionnaires and objective measures by logging the elapsed time to fulfill task scenarios. Our results indicate that the perceived quality of the immersive VR system is indistinguishable from the physical counterpart regarding User Experience, usability, and cybersickness. On the other hand, the users’ performance on VR simulation was significantly slower in immersive VR. Finally, the same usability issues could be detected with either of the conditions.

Jean F. P. Cheiran, Laura A. Torres, Antonio A. S. da Silva, Gabrielle A. de Souza, Luciana P. Nedel, Anderson Maciel, Dante A. C. Barone

Gaze-Contingent Rendering in Virtual Reality

Virtual reality (VR) is a technology that relies on a computer graphics system and other external display and control interfaces, to create an immersive experience by generating an interactive three-dimensional environment on a computer. Currently, however, most virtual reality scenes are far behind the real world in naturalism. One of the limitations is the insufficient graphics computing performance of the computer. It is difficult for mainstream consumer GPUs to meet the requirements of high picture quality and high fluency at the same time when running VR scenes, resulting in a reduction in the game’s visual experience and even human discomfort. In order to balance the quality and fluency of the picture, the areas within and outside the focus range of the user’s sight can be rendered hierarchically, so as to efficiently use computing resources. In order to achieve this goal, the following article proposes a model that combines the saliency information of the virtual scene and the head motion information to predict the focus of the field of view in real time. The model can assign different rendering priorities to objects in the field of view according to the prediction results, and give different priorities, use different rendering algorithms to provide a flexible VR scene rendering optimization solution.

Fang Zhu, Ping Lu, Pin Li, Bin Sheng, Lijuan Mao

Hierarchical Rendering System Based on Viewpoint Prediction in Virtual Reality

Virtual reality (VR) systems use multi-modal interfaces to explore three-dimensional virtual worlds. During exploration, the user may look at different objects of interest or in different directions. The field of view of human vision is 135 $$^{\circ }\times 160^{\circ }$$ ∘ × 160 ∘ , but the one requiring the highest resolution is only in 1.5 $$^{\circ }\times 2^{\circ }$$ ∘ × 2 ∘ . It is estimated that in modern VR, only 4 $$\%$$ % of the pixel resources of the head-mounted display are mapped to the visual center. Therefore, allocating more computing resources to the visual center and allocating fewer viewpoint prediction rendering techniques elsewhere can greatly speed up the rendering of the scene, especially for VR devices equipped with eye trackers. However, eye trackers as additional equipment may be relatively expensive and be harder to use, at the same time, there is considerable work to be done in the development of eye trackers and their integration with commercial head-mounted equipment. Therefore, this article uses an eye-head coordination model combined with the saliencey of the scene to predict the gaze position, and then uses a hybrid method of Level of Detail (LOD) and grid degeneration to reduce rendering time as much as possible without losing the perceived details and required calculations.

Ping Lu, Fang Zhu, Pin Li, Jinman Kim, Bin Sheng, Lijuan Mao

Reinforcement Learning-Based Redirection Controller for Efficient Redirected Walking in Virtual Maze Environment

Redirected walking (RDW) is a locomotion technique used in virtual reality (VR) that enables users to explore large virtual environments in a limited physical space. Existing RDW techniques mainly work on the obstacle-free physical spaces larger than a square of four-meter sides. To improve usability, RDW techniques that work on comparatively smaller physical spaces with obstacles need to be developed. In RDW, users are restricted to the physical space by redirection techniques (RETs) that control the view of the head-mounted display. Reinforcement learning, a branch of machine learning techniques, is advantageous in designing efficient redirection controllers compared to manual design. In this paper, we propose a reinforcement learning-based redirection controller (RLRC) that aims to realize an efficient RDW in small physical spaces. The controller is trained using the simulator and is expected to select an appropriate redirection technique from the current state and route information of the virtual environment. We evaluate the RLRC with simulator and user tests in a virtual maze in several physical spaces, including a square physical space of four-meter sides with an obstacle, and a square physical space of two-meter sides. The simulator test shows that the proposed RLRC can reduce the number of undesirable redirection techniques performed compared with existing methods. The proposed RLRC is found to be effective in the square physical space of two-meter sides in the user test.

Wataru Shibayama, Shinichi Shirakawa

Locality-Aware Skinning Decomposition Using Model-Dependent Mesh Clustering

Skinning decomposition is a popular technique to approximate a vertex animation for memory-efficient playback of complex shape deformation in real-time applications. However, conventional methods have several practical limitations related to computational performance and animation quality. We propose a skinning decomposition method that takes the locality of the skin weight into consideration. Our main idea is to decompose a skin mesh into multiple clusters and estimate the skin weight in each cluster to achieve local weight distribution with compact support. Our framework provides two types of mesh clustering algorithms for enhancing approximation accuracy. We also propose a frame-reduction algorithm for efficient computation. The experimental results indicate that our locality-aware approach produces a highly accurate approximation while significantly reducing the computation time.

Fumiya Narita, Tomohiko Mukai

A New Volume-Based Convexity Measure for 3D Shapes

Convexity, as a global and learning-free shape descriptor, has been widely applied to shape classification, retrieval and decomposition. Unlike its extensively addressed 2D counterpart, 3D shape convexity measurement attracting insufficient attention has yet to be studied. In this paper, we put forward a new volume-based convexity measure for 3D shapes, which builds on a conventional volume-based convexity measure but excels it by resolving its problems. By turning the convexity measurement into a problem of influence evaluation through Distance-weighted Volume Integration, the new convexity measure can resolve the major problems of the existing ones and accelerate the overall computational time.

Xiayan Shi, Rui Li, Yun Sheng

Deep Inverse Rendering for Practical Object Appearance Scan with Uncalibrated Illumination

In this paper, we propose a practical method to estimate object appearance from an arbitrary number of images. We use a moving flashlight as light source, and encode surface reflectance properties in a pre-learned embedded latent space. Such lighting and appearance model combination enables our method to effectively narrow the solution space. Uncalibrated illumination requirement extremely simplifies our setup and affords it unnecessary to accurately locate light positions in advance. Moreover, our method automatically selects key frames before appearance estimation, which largely reduces calculation cost. Both synthetic and real experiments demonstrate that our method can recover object appearance accurately and conveniently.

Jianzhao Zhang, Guojun Chen, Yue Dong, Jian Shi, Bob Zhang, Enhua Wu

Application of the Transfer Matrix Method to Anti-reflective Coating Rendering

Thin-film coating is a common practice to modify the appearance of materials. In optics for example, coating is often used on mirrors or lenses to modify their reflectance and transmittance properties. To achieve high transmittance optics or wavelength selective filters for sensors, multilayer coatings are required. Thin-film coating is an active area of research. In this paper we introduce to the rendering community the transfer matrix method to calculate the Fresnel coefficients for multilayer thin-film coating. This method, commonly used in optics, provides an easy way to calculate reflectance and transmittance coefficients for an arbitrary number of thin-film layers. Unlike previous methods [10], which relied on the infinite Airy summation, this method is based on the multiplication of $$2\times 2$$ 2 × 2 matrices which allows handling more general cases. We apply this method to simulate physically based anti-reflective coating where a single layer of thin-film coating is often not enough to obtain a good performance over the full visible spectrum.

Alexis Benamira, Sumanta Pattanaik

Dynamic Shadow Rendering with Shadow Volume Optimization

The shadow volume is utilized extensively for real-time rendering applications which includes updating volumes and calculating silhouette edges. Existing shadow volume methods are CPU intensive and complex occluders result in poor rendering efficiency. In this paper, we propose a hash-culling shadow volume algorithm that uses hash-based acceleration for the silhouette edge determination which is the most time-consuming processing in the traditional shadow volume algorithm. Our proposed method uses a hash table to store silhouette edge index information and thus reduces the time taken for redundant edge detection. The method significantly reduces CPU usage and improves algorithm time efficiency. Furthermore, for low hardware-level systems, especially embedded systems, it is still difficult to render dynamic shadows due to their high demand on the fill-rate capacity of graphics hardware. Our method has low hardware requirements and is easy to implement on PCs and embedded systems with real-time rendering performance with visual-pleasing shadow effects.

Zhibo Fu, Han Zhang, Ran Wang, Zhen Li, Po Yang, Bin Sheng, Lijuan Mao

Adaptive Illumination Sampling for Direct Volume Rendering

Direct volume rendering is used to visualize data from sources such as tomographic imaging devices. The perception of certain structures depends very much on visual cues such as lighting and shadowing. According illumination techniques have been proposed for both surface rendering and volume rendering. However, in the case of direct volume rendering, some form of precomputation is typically required for real-time rendering. This however limits the application of the visualization. In this work we present adaptive volumetric illumination sampling, a ray-casting-based direct volume rendering method that strongly reduces the amount of necessary illumination computations without introducing any noise. By combining it with voxel cone tracing, realistic lighting including ambient occlusion and image-based lighting is facilitated in real-time. The method only requires minimal precomputation and allows for interactive transfer function updates and clipping of the visualized data.

Valentin Kraft, Florian Link, Andrea Schenk, Christian Schumann

Musical Brush: Exploring Creativity Through an AR-Based Tool for Sketching Music and Drawings

The economic growth and social transformation in the 21st century are hardly based on creativity. To help with the development of this skill, the concept of Creativity Support Tools (CST) was proposed. In this paper, we introduce Musical Brush (MB), an artistic mobile application whose main focus is to allow novices to improvise music while creating drawings. We investigated different types of interactions and audiovisual feedbacks in the context of a mobile application that combines music with drawings in a natural way, measuring their impact on creativity support. In this study, we tested different user interactions with real-time sound generation, including 2D drawings, three-dimensional device movements, and visual representations on Augmented Reality (AR). A user study was conducted to explore the support for creativity of each setup. Results showed the suitability of the association of Musical Brush with augmented reality for creating sounds and drawings as a tool that supports the exploration and expressiveness.

Rafael Valer, Rodrigo Schramm, Luciana Nedel

MR Environments Constructed for a Large Indoor Physical Space

To resolve the problem that existing mixed reality (MR) apparatus are unable to scan and model a large and complex indoor scene at once, we present a powerful toolkit for constructing MR environment easily. Our toolkit establishes and maintains accurate mapping between the virtual and physical space, and sets the occlusion relationships of the walls. Additionally, we design spatial anchor deployment strategy supporting deviation correction between the real and virtual spaces, so that the spatial anchors can maintain a virtual object’s location and orientation in the real world. Our experiments and applications show that the toolkit is convenient for constructing MR apps targeting large physical spaces in which users can roam in real time.

Huan Xing, Chenglei Yang, Xiyu Bao, Sheng Li, Wei Gai, Meng Qi, Juan Liu, Yuliang Shi, Gerard De Melo, Fan Zhang, Xiangxu Meng

FIOU Tracker: An Improved Algorithm of IOU Tracker in Video with a Lot of Background Inferences

Multiple object tracking(MOT) is a fundamental problem in video analysis application. Associating unreliable detection in a complex environment is a challenging task. The accuracy of multiple object tracking algorithms is dependent on the accuracy of the first stage object detection algorithm. In this paper, we propose an improved algorithm of IOU Tracker–FIOU Tracker. Our proposal algorithm can overcome the shortcoming of IOU Tracker with a small amount of computing cost that heavily relies on the precision and recall of object detection accuracy. The algorithm we propose is based on the assumption that the motion of background inference is not obvious. We use the average light flux value of the track and the change rate of the light flux value of the center point of the adjacent object as the conditions to determine whether the trajectory is to be retained. The tracking accuracy is higher than the primary IOU Tracker and another well-known variant VIOU Tracker. Our proposal method can also significantly reduce the ID switch value and fragmentation value which are both important metrics in MOT task.

Zhihua Chen, Guhao Qiu, Han Zhang, Bin Sheng, Ping Li

An Approach of Short Advertising Video Generation Using Mobile Phone Assisted by Robotic Arm

Recently, Short Advertising Video has become an increasingly dominant form of advertisement on social media. However, making Short Advertising Video is a challenging task for micro and small businesses, since it requires professional skills and years of experience. In this paper, we present a novel approach of Short Advertising Video generation assisted by robotic arms. We analyzed the professional composition and imaging of advertising videos, and transformed them into an automatic shooting process during the production of Short Advertising Video, assisted by a robotic arm. Practically, we applied our approach in two kinds of robotic arms and the results showed that robotic arm assist solution can highly enhance the efficiency and effect of making Short Advertising Video. In addition, our video generation approach can save time and money for novice users from micro and small business who has very limit resources and budget. And, we believe that our approach might overturn the existing production model of the Short Advertising Video propagated in the online business and social media.

Jiefeng Li, Yingying She, Lin Lin, Yalan Luo, Hao He, Weiyue Lin, Shengjing Hou

“Forget” the Forget Gate: Estimating Anomalies in Videos Using Self-contained Long Short-Term Memory Networks

Abnormal event detection is a challenging task that requires effectively handling intricate features of appearance and motion. In this paper, we present an approach of detecting anomalies in videos by learning a novel LSTM based self-contained network on normal dense optical flow. Due to their sigmoid implementations, standard LSTM’s forget gate is susceptible to overlooking and dismissing relevant content in long sequence tasks. The forget gate mitigates participation of previous hidden state for computation of cell state prioritizing current input. Besides, the hyperbolic tangent activation of standard LSTMs sacrifices performance when a network gets deeper. To tackle these two limitations, we introduce a bi-gated, light LSTM cell by discarding the forget gate and introducing sigmoid activation. Specifically, the proposed LSTM architecture fully sustains content from previous hidden state thereby enabling the trained model to be robust and make context-independent decision during evaluation. Removing the forget gate results in a simplified and undemanding LSTM cell with improved performance and computational efficiency. Empirical evaluations show that the proposed bi-gated LSTM based network outperforms various LSTM based models for abnormality detection and generalization tasks on CUHK Avenue and UCSD datasets.

Habtamu Fanta, Zhiwen Shao, Lizhuang Ma

An Improved Image Stitching Method Based on Seed Region Growth and Poisson Fusion

In order to address the problem of ghosting and gap in the field of image stitching, we proposed a novel image stitching method based on seed region growth algorithm and Poisson fusion. Firstly, PSO (Particle Swarm Optimization) is used to improve the accuracy of image registration. Then we combine the improved seed region growth algorithm with Poisson fusion to implement image stitching. Experimental results illustrate outstanding performance of our method by comparing with one famous image stitching tool: PTGui and one existing image stitching method. Our method can effectively solve the problem of ghosting and gap in the process of image stitching, and avoid objects distortion.

Yewen Pang, Aimin Li, Jianwen Wang

Illumination Harmonization with Gray Mean Scale

Illumination harmonization is an important problem for high-quality image composite. Given the source image and the target background, it aims to transform the foreground appearance that it looks in the same lighting condition as the target background. Because the ground-truth composite image is difficult to get, previous works can use only synthetic datasets, which however, provide with only artificially adjusted and limited inputs. In this paper we contribute to this problem in two aspects: 1) We introduce a semi-automatic approach to capture the ground-truth composite in real scenes, and then create a dataset that enables faithful evaluation of image harmonization methods. 2) We propose a simple yet effective harmonization method, namely the Gray Mean Scale (GMS), which models the foreground appearance transformation as channel-wise scales, and estimates the scales based on gray pixels of the source and the target background images. In experiments we evaluated the proposed method and compared it with previous methods, using both our dataset and previous synthetic datasets. A new benchmark thus is established for illumination harmonization in real environments.

Shuangbing Song, Fan Zhong, Xueying Qin, Changhe Tu

An Unsupervised Approach for 3D Face Reconstruction from a Single Depth Image

In this paper, we propose a convolutional encoder network to learn a mapping function from a noisy depth image to a 3D expressive facial model. We formulate the task as an embedding problem and train the network in an unsupervised manner by exploiting the consistent fitting of the 3D mesh and the depth image. We use the 3DMM-based representation and embed depth images to code vectors concerning facial identities, expressions, and poses. Without semantic textural cues from RGB images, we exploit geometric and contextual constraints in both the depth image and the 3D surface for reliable mapping. We combine the multi-level filtered point cloud pyramid and semantic adaptive weighting for fitting. The proposed system enables the 3D expressive face completion and reconstruction in poor illuminations by leveraging a single noisy depth image. The system realizes a full correspondence between the depth image and the 3D statistical deformable mesh, facilitating landmark location and feature segmentation of depth images.

Peixin Li, Yuru Pei, Yicheng Zhong, Yuke Guo, Gengyu Ma, Meng Liu, Wei Bai, Wenhai Wu, Hongbin Zha

Fusing IMU Data into SfM for Image-Based 3D Reconstruction

Structure-from-Motion (SfM), one of the most extensively used image-based 3D reconstruction methods in Computer Graphics and Vision, suffers from mismatching of image feature points when input images are sparse, thus leading to miscalculation of camera rotation matrices as well as subsequent reconstruction errors. To address these problems, this paper reports an improved SfM reconstruction system and proposes to suppress the miscalculation of camera rotation matrices during Bundle Adjustment (BA) by forming a rotation constraint with Internal Measurement Unit (IMU) data collected from a smartphone, consequently improving the reconstruction precision and efficiency. More specifically, being refined by the Kalman filter, the IMU data are adopted to estimate the camera rotation matrix of each image captured by the smartphone. As the camera rotation matrix calculated by the IMU data is based on the sensor coordinate system, this paper employs the Lie Algebra theory to transform the reference frame of the rotation matrix from the sensor coordinate system to the camera coordinate system.

Hua Yuan, Yifan Ma, Yun Sheng

Physics-Guided Sound Synthesis for Rotating Blades

This paper focuses on sound synthesis for rotating blades such as fans, helicopters and wind turbines, which is common in both real world and computer games though has received little attention until now. In this paper, we propose a novel physics-guided sound synthesis method for rotating blades. First, we propose an efficient rotating blade sound solver for Ffowcs Williams-Hawkings (FW-H) equation, which can greatly reduce the computational complexity. Then, inspired by the good expression of Mel-scale Frequency Cepstral Coefficients (MFCC) in speech recognition, we design a new sound parameter A_MFCC to enrich the sound. Specifically, while ensuring the sensitivity of MFCC to formants, we improve MFCC to make it well show the properties of sound timbre and loudness, so that it can be well applied in sound synthesis. Finally, based on the observation that rotating blade sound has similar qualities with noise, we specially devise a method to further enrich the sounding result by combining noise and A_MFCC. Experimental results demonstrated that our method can achieve great sounding results for various rotating blades.

Siqi Xu, Shiguang Liu

Elimination of Incorrect Depth Points for Depth Completion

Commodity-level scan cameras generally capture RGB-D image with depth missing or incorrect depth points if the surface of the object is transparent, bright, or black. These incorrect depth points are generated randomly and limit the downstream applications of raw RGB-D images. In this paper, we propose a coarse-to-fine method to detect and eliminate the incorrect depth points via RGB semantics. In our flowchart, deep learning-based networks are applied to predict the potential regions with incorrect depth points and the normals of the point cloud. Then we develop a three-step elimination method to remove the incorrect depth points in the regions. Experimental results show that our method leads to great improvements for downstream applications of RGB-D images, especially in depth completion application.

Chuhua Xian, Kun Qian, Guoliang Luo, Guiqing Li, Jianming Lv

Pose Transfer of 2D Human Cartoon Characters

Pose transfer between two 2D cartoon characters provides a fast way to copy pose without complex deformation operations on the 2D shape. This paper proposes an effective method for transferring the pose of 2D human cartoon characters while preserving the character’s geometric features. We compare our method with other similar works and discuss the convergence of the results under geometric constraints. The results show that our method can effectively achieve smooth pose transfer between cartoon characters with good convergence.

Tiezeng Mao, Wenbo Dong, Aihua Mao, Guiqing Li, Jie Luo

Broad-Classifier for Remote Sensing Scene Classification with Spatial and Channel-Wise Attention

Remote sensing scene classification is an important technology, which is widely used in military and civil applications. However, it is still a challenging problem due to the complexity of scene images. Recently, the development of remote sensing satellite and sensor devices has greatly improved the spatial resolution and semantic information of remote sensing images. Therefore, we propose a novel remote sensing scene classification approach to enhance the performance of scene classification. First, a spatial and channel-wise attention module is proposed to adequately utilize the spatial and feature information. Compare with other methods, channel-wise module works on the feature maps with diverse levels and pays more attention to semantic-level features. On the other hand, spatial attention module promotes correlation between foreground and classification result. Second, a novel classifier named broad-classifier is designed to enhance the discriminability. It greatly reduces the cost of computing in the meantime by broad learning system. The experimental results have show that our classification method can effectively improve the average accuracies on remote sensing scene classification data sets.

Zhihua Chen, Yunna Liu, Han Zhang, Bin Sheng, Ping Li, Guangtao Xue

GARNet: Graph Attention Residual Networks Based on Adversarial Learning for 3D Human Pose Estimation

Recent studies have shown that, with the help of complex network architecture, great progress has been made in estimating the pose and shape of a 3D human from a single image. However, existing methods fail to produce accurate and natural results for different environments. In this paper, we proposed a novel adversarial learning approach and studied the problem of learning graph attention network for regression. Graph Attention Residual Networks (GARNet), which processes regression tasks with graphic-structured data, learns to capture semantic information, such as local and global node relationships, through end-to-end training without additional supervision. The adversarial learning module is implemented by a novel multi-source discriminator network to learn the mapping from 2D pose distribution to 3D pose distribution. We conducted a comprehensive study to verify the effectiveness of our method. Experiments show that the performance of our method is superior to that of most existing techniques.

Zhihua Chen, Xiaoli Liu, Bing Sheng, Ping Li

GPU-based Grass Simulation with Accurate Blade Reconstruction

Grass is a very important element of nature and it could almost be found in every natural scene. Thus grass modeling, rendering as well as simulation becomes an important task for virtual scene creation. Existing manual grass modeling and reconstruction methods have researched on generate or reconstructing plants. However, these methods do not achieve a good result for grass blades for their extremely thin shape and almost invariant surface color. Besides, current simulation and rendering methods for grasses suffer from efficiency and computation complexity problems. This paper introduces a framework that reconstructs the grass blade model from the color-enhanced depth map, simplifies the grass blade model and achieves extremely large scale grassland simulation with individual grass blade response. Our method starts with reconstructing the grass blade model. We use color information to guide the refinement of captured depth maps from cameras based on an autoregressive model. After refinement, a high-quality depth map is used to reconstruct thin blade models, which cannot be well handled by multi-view stereo methods. Then we introduce a blade simplification method according to each vertex’s movement similarity. This method takes both geometry and movement characteristics of grass into account when simplifying blade mesh. In addition, we introduce a simulation technique for extremely large grassland that achieve tile management on GPU and allow individual response for each grass blade. Our method excels at reconstructing slender grass blades as well as other similar plants, and realistic dynamic simulation for large scale grassland.

Sheng Wang, Saba Ghazanfar Ali, Ping Lu, Zhen Li, Po Yang, Bin Sheng, Lijuan Mao

Flow Visualization with Density Control

In flow visualization, it remains challenging to flexibly explore local regions in 3D fields and uniformly display the structures of flow fields while preserving key features. To this end, this paper presents a novel method for streamline generation and selection for 2D and 3D flow fields via density control. Several levels of streamlines are divided by flow density. The lowest level is produced using an entropy-based seeding strategy and a grid-based filling procedure. It can generate uniform streamlines without loss of important structural information. Other levels are then generated by a streamline selection algorithm based on the average distance among streamlines. It could help users understand flow fields in a more flexible manner. For 3D fields, we further provide local density control and density control along any axis for users, which are helpful to explore the fields both globally and locally. Various experimental results validate our method.

Shiguang Liu, Hange Song

DbNet: Double-Ball Model for Processing Point Clouds

Learning and understanding 3D point clouds with convolutional networks is challenging due to the irregular and unordered data format. Reviewing existing network models based on PointNet [13] and PointNet++ [14], they resample in different regions and explore not enough due to the irregularity and sparsity of the geometric structures. In this paper, we proposed a double-ball model embedded in the hierarchical network(DbNet) that directly extracts the features from the point clouds. This method avoids overlapping and better captures the local neighborhood geometry of each point. Double-ball model has two key steps: double-ball query and building features graph. Double-ball query avoids the resampling problem caused by the simple ball query. Building features graph takes angular features and edge features of point clouds into consideration. This method has no requirements for translation and rotation with the object. We apply it to 3D shapes classification and segmentation. And experiments on two benchmarks show that the suggested network outperforms the models based on PointNet/PointNet++ and is able to provide state of the art results.

Meisheng Shen, Yan Gao, Jingjun Qiu

Evolving L-Systems in a Competitive Environment

Lindenmayer systems (L-systems) have been developed to model plant growth by repeatedly applying production rules to an initial axiom, and serve as a model for genetically driven growth processes found in nature. A simulation method is proposed to evolve their phenotypic representations through competition in a heterogeneous environment to further expand on this biological analogy. The resulting simulations demonstrate evolution driven by competition, resulting in agents employing strategies similar to those found in nature.

Job Talle, Jiří Kosinka

ParaGlyder: Probe-driven Interactive Visual Analysis for Multiparametric Medical Imaging Data

Multiparametric imaging in cancer has been shown to be useful for tumor detection and may also depict functional tumor characteristics relevant for clinical phenotypes. However, when confronted with datasets consisting of multiple values per voxel, traditional reading of the imaging series fails to capture complicated patterns. These patterns of potentially important imaging properties of the parameter space may be critical for the analysis, but standard approaches do not deliver sufficient details. Therefore, in this paper, we present an approach that aims to enable the exploration and analysis of such multiparametric studies using an interactive visual analysis application to remedy the trade-offs between details in the value domain and in spatial resolution. This may aid in the discrimination between healthy and cancerous tissue and potentially highlight metastases that evolved from the primary tumor. We conducted an evaluation with eleven domain experts from different fields of research to confirm the utility of our approach.

Eric Mörth, Ingfrid S. Haldorsen, Stefan Bruckner, Noeska N. Smit

3D Geology Scene Exploring Base on Hand-Track Somatic Interaction

Terrain analysis is the basis of geological research. However, due to factors such as distance and range, it is often difficult to study the terrain environment in the field. Therefore, researchers can observe and study the terrain by making a three-dimensional terrain model. The 3D terrain model can reduce the terrain range, eliminate the limitation on distance, and control the scene through program interface, to achieve human-computer interaction to meet different research needs. The usual human-computer interaction methods are implemented through traditional peripherals such as the mouse and keyboard. With the rapid development of computer network technology and the continuous improvement of intelligent software and hardware, people have greater requirements for interactive manipulation and immersion. This article proposes a method for displaying terrain models based on real-sensing technology by using Intel’s RealSense camera to control the scene of the desert model through gestures. The user can observe the model from two different perspectives, and use different gestures to zoom in, zoom out, move, and rotate the scene, as well as choose some options. The traditional method of controlling by mouse is also applicable. The entire project is designed as a game, with a realistic and complete model, an exquisite interface, and strong interactivity.

Wei Zhang, Fang Zhu, Ping Lu, Pin Li, Bin Sheng, Lijuan Mao

GHand: A Graph Convolution Network for 3D Hand Pose Estimation

Vision-based 3D hand pose estimation plays an important role in the field of human-computer interaction. In recent years, with the development of convolutional neural networks (CNN), the field of 3D hand pose estimation has made a great progress, but there is still a long way to go before the problem is solved. Although recent studies based on CNN networks have greatly improved the recognition accuracy, they usually only pay attention on the regression ability of the network itself, and ignore the structural information of the hands, thus leads to a low accuracy in contrast. In this paper we proposed a new hand pose estimation network, which can fully learn the structural information of hands through an adaptive graph convolutional neural network. The experiment on the public dataset shows the accuracy of our graph convolution network exceeds the SOTA methods in 3D hand pose estimation.

Pengsheng Wang, Guangtao Xue, Pin Li, Jinman Kim, Bin Sheng, Lijuan Mao

Bézier Curve as a Generalization of the Easing Function in Computer Animation

The description of movement has always been one of the basic problems in traditional and computer animation. A person watching an animated film may accept simplifications in the appearance of the characters, but will not accept unnatural, unexpected motion. The transition problem is an example of movement description which is successfully solved by Penner’s easing functions. But, the scope of an easing function is limited to known examples of specific functions proposed by Penner or other developers. However, there is often a need to describe the transition problem by a function resulting from the interpolation of points that approximate the trajectory. A convenient way to describe the shape, in such a situation, would be a Bézier curve. The article is an attempt to generalize the problem of interpolation of transition trajectories using the Bézier curve. By analyzing various cases of the cubic polynomial equations in the context of transition problems in animation, we can limit the solution to several families of the transcendental functions.

Łukasz Izdebski, Ryszard Kopiecki, Dariusz Sawicki

Generating Orthogonal Voronoi Treemap for Visualization of Hierarchical Data

A novel space partitioning strategy is presented for implicit hierarchy visualization. The proposed orthogonal Voronoi treemap (OVT) partitions an empty canvas into nested orthogonal rectangles, thus the generated layout is not only flexible to diversified data value, but also much tidier than the Voronoi treemap with nested polygons. To achieve this, we first introduce a new distance calculation strategy in order to generate axis-aligned segmentation among the sites. To cope with the new segmentation strategy, we then design a sweepline + skyline heuristic algorithm to partition the canvas to generate an orthogonal Voronoi treemap. Comparative analyses on the computational experiment results in terms of aspect ratio is discussed.

Yan-Chao Wang, Jigang Liu, Feng Lin, Hock-Soon Seah

CGI’20 Short Papers


Preserving Temporal Consistency in Videos Through Adaptive SLIC

The application of image processing techniques to individual frames of video often results in temporal inconsistency. Conventional approaches used for preserving the temporal consistency in videos have shortcomings as they are used for only particular jobs. Our work presents a multipurpose video temporal consistency preservation method that utilizes an adaptive simple linear iterative clustering (SLIC) algorithm. First, we locate the inter-frame correspondent pixels through the SIFT Flow and use them to find the respective regions. Then, we apply a multiframe matching statistical method to get the spatially or temporally correspondent frames. Besides, we devise a least-squares energy-based flickering-removing objective function by taking into account the inter-frame temporal consistency and inter-region spatial consistency jointly. The obtained results demonstrate the potential of the proposed method.

Han Zhang, Riaz Ali, Bin Sheng, Ping Li, Jinman Kim, Jihong Wang

Efficient Non-fused Winograd on GPUs

This paper presents an optimized implementation for Winograd non-fused convolution. Our optimizations comprise application-independent grouped producer-consumer chains and a set of Winograd-specific software techniques, including specialized interface-kernels data format which enhances memory access efficiency; warp specialization and double buffer prefetching which effectively exploit computational resources and memory bandwidth; utilizing “shuffle” instruction which conserves hardware resources. The paper also provides supplementary explanation of Winograds’ tile extraction, which saves memory and computing resources.The proposed techniques has been evaluated head to head by kernel level in GTX 980 GPU, CUDA 9.2 with a wide range of parameters which meet CNN layers benchmark. Compared with the state-of-the-art Winograd Non-fused convolution in CuDnn 7.6.4 (released in Sept, 2019), our implementation achieves a total speedup of 1.64x.

Hui Wei, Enjie Liu, Youbing Zhao, Hongqing Yu

ENGAGE Full Papers


Surface Fitting Using Dual Quaternion Control Points with Applications in Human Respiratory Modelling

In this paper we present a method for representing surfaces using a set of dual quaternion control points, with the goal of fitting to point clouds. Each control point is defined by a position and radius, which specify the area of the surface it affects, and a dual quaternion defining the transformation it applies. A point is mapped using the surface by a weighted sum of the control points, in a similar method to dual quaternion skinning. A surface is then represented as the transformation of an original surface, such as a unit square plane, using the control points. We demonstrate how we may fit surfaces to point clouds using a modified iterative gradient descent algorithm, adding control points to regions of the surface that are most poorly modelled at the current step. These methods are applied to the problem of representing human breathing by fitting surfaces to a subject’s chest as recorded by an RGB-D (image plus depth) camera and parameterizing the breathing using each control point’s parameters. Variations in the breathing pattern are shown before and after exercise.

Alex Grafton, Joan Lasenby

Deform, Cut and Tear a Skinned Model Using Conformal Geometric Algebra

In this work, we present a novel, integrated rigged character simulation framework in Conformal Geometric Algebra (CGA) that supports, for the first time, real-time cuts and tears, before and/or after the animation, while maintaining deformation topology. The purpose of using CGA is to lift several restrictions posed by current state-of-the-art character animation & deformation methods. Previous implementations originally required weighted matrices to perform deformations, whereas, in the current state-of-the-art, dual-quaternions handle both rotations and translations, but cannot handle dilations. CGA is a suitable extension of dual-quaternion algebra that amends these two major previous shortcomings: the need to constantly transmute between matrices and dual-quaternions as well as the inability to properly dilate a model during animation. Our CGA algorithm also provides easy interpolation and application of all deformations in each intermediate steps, all within the same geometric framework. Furthermore we also present two novel algorithms that enable cutting and tearing of the input rigged, animated model, while the output model can be further re-deformed. These interactive, real-time cut and tear operations can enable a new suite of applications, especially under the scope of a medical surgical simulation.

Manos Kamarianakis, George Papagiannakis

The Forward and Inverse Kinematics of a Delta Robot

The Delta robot is one of the most popular parallel robots in industrial use today. In this paper we analyse the forward and inverse kinematics of the robot from a geometric perspective using Conformal Geometric Algebra. We calculate explicit formulae for all joints in both the forward and inverse kinematic problems as well as explicit forward and inverse Jacobians to allow for velocity and force control. Finally we verify the kinematics in Python and simulate a physical model in the Unity3D game engine to act as a test-bed for future development of control algorithms.

Hugo Hadfield, Lai Wei, Joan Lasenby

Constrained Dynamics in Conformal and Projective Geometric Algebra

In this paper we tackle the problem of constrained rigid body dynamics in the Conformal and Projective Geometric Algebras (CGA, PGA). First we construct a screw-theory based formulation of dynamics in CGA and note the equivalence between this and the PGA dynamics presented by Gunn in [1]. After verifying the formulation via simulation, we move on to the challenge of adding constraints. First we apply the standard mechanical engineering technique of virtual power to the constraint problem in our Geometric Algebra (GA) framework. We then discuss a novel technique for ‘pinning’ dynamic rigid bodies to geometric primitives, a technique that relies on the invariance of certain multivectors and functions of multivectors to specific rotor transformations.

Hugo Hadfield, Joan Lasenby

Application of 2D PGA as an Subalgebra of CRA in Robotics

We present a concept of 2D Projective Geometric Algebra (PGA) as a subalgebra of Compass Ruler Algebra (CRA) to handle problems in computer graphics and engineering efficiently in terms of an algebra with minimal dimension. In this case, we can benefit from both CRA and PGA simultaneously. When we deal with complex problems, we can use CRA objects such as circles but at the same time we can switch to PGA as a subalgebra of CRA to handle operations with flat-objects more efficiently without the change of structure of any further implementation. We demonstrate this approach on example of inverse kinematics of a planar 3-link manipulator.

Radek Tichý

Outline of Tube Elbow Detection Based on GAC

We outline an algorithm for camera–based tube elbow detection provided that a moving camera is placed in the tube axis together with a light source. We claim that once an elbow is approached, circular contour is replaced by an elliptical one and the displacement of respective circle and ellipse centres is proportional to the elbow angle. We provide Python code for computation in Geometric Algebra for Conics (GAC), parameter extraction and an example of proportionality.

Roman Byrtus, Anna Derevianko, Petr Vašík

Optimal Parenthesizing of Geometric Algebra Products

Manipulating objects using geometric algebra may involve several associative products in a single expression. For example, an object can be constructed by the outer product of multiple points. This number of products can be small for some conformal algebra and high for higher dimensional algebras such as quadric conformal geometric algebras. In these situations, the order of products (i.e. the choice of the parenthesis in the expression) should not change the final result but may change the overall computational cost, according to the grade of the intermediate multivectors. Indeed, the usual left to right way to evaluate the expression may not be most computationally efficient. Studies on the number of arithmetic operations of geometric algebra expressions have been limited to products of only two homogeneous multivectors. This paper shows that there exists an optimal order in the evaluation of an expression involving geometric and outer products, and presents a dynamic programming framework to find it.

Stéphane Breuils, Vincent Nozick, Akihiro Sugimoto

Geometric Algebra-Based Multilevel Declassification Method for Geographical Field Data

The diversity of GIS application patterns leads to the demand for multilevel GIS data declassification. For example, Publicly used data must be declassified to hide confidential spatial information. The reversion process is not a common data permutation like the conventional encryption method does. The reverted data should also keep the general geospatial features. Furthermore, when facing different levels of confidentiality, different levels of reversion were needed. In this paper, A declassification and reversion method with controllable accuracy is realized using geometric algebra (GA). The geographical field is expressed as a GA object and the unified representation of the field is further realized. By introducing the rotor operator and perturbation matrix, the declassification methods are proposed for geographic field data, which can progressively revert the features of the field. A geometric algebraic declassification operator is also constructed to realize the unification operations of field features and spatial coordinate. By exploring the space error and space structure characterization of the results, a quantitative performance evaluation is provided. Experiments have shown that the method can carry out effective precision control and has good randomness and a high degree of freedom characteristics. The experimental data show a correlation coefficient of 0.945, 0.923 and 0.725 for the longitude-oriented field data during the low level, medium level and high level declassification, respectively. The algorithm characteristics meet the application needs of geographic field data in data disclosure, secure transmission, encapsulation storage, and other aspects.

Wen Luo, Dongshuang Li, Zhaoyuan Yu, Yun Wang, Zhengjun Yan, Linwang Yuan

Homomorphic Data Concealment Powered by Clifford Geometric Algebra

We propose general-purpose methods for data representation and data concealment via multivector decompositions and a small subset of functions in the three dimensional Clifford geometric algebra. We demonstrate mechanisms that can be explored for purposes from plain data manipulation to homomorphic data processing with multivectors. The wide variety of algebraic representations in Clifford geometric algebra allow us to explore concepts from integer, complex, vector and matrix arithmetic within a single, compact, flexible and yet powerful algebraic structure in order to propose novel homomorphisms. Our constructions can be incorporated into existing applications as add-ons as well as used to provide standalone data-centric algorithms. We implement our representation and concealment mechanisms in the Ruby programming language to demonstrate the ideas discussed in this work.

David W. H. A. da Silva, Marcelo A. Xavier, Philip N. Brown, Edward Chow, Carlos Paz de Araujo

An Online Calculator for Qubits Based on Geometric Algebra

We use Geometric Algebra for quantum computing motivated by the fact that qubits and gates can be handled as elements of the same algebra. Additionally, Geometric Algebra allows us to describe gate operations very easily, based on the geometrically intuitive description of transformations in Geometric Algebra. As the main contribution of this paper, we make the calculations with the specific QBA (quantum bit algebra) accessible via an online tool based on GAALOPWeb, which can be handled on many computing devices.

D. Hildenbrand, C. Steinmetz, R. Alves, J. Hrdina, C. Lavor

ENGAGE Short Papers


On Basis-Free Solution to Sylvester Equation in Geometric Algebra

The Sylvester equation and its particular case, the Lyapunov equation, are widely used in image processing, control theory, stability analysis, signal processing, model reduction, and many more. We present the basis-free solution to the Sylvester equation in geometric algebra of arbitrary dimension. The basis-free solutions involve only the operations of geometric product, summation, and the operations of conjugation. The results can be used in symbolic computation.

Dmitry Shirokov


The direct construction of geometric elements in an N dimensional geometric algebra by taking the outer product between $$N-1$$ N - 1 primitive points is one of the cornerstone tools. It is used to construct a variety of objects, from spheres in CGA [6], up to quadric [2] and even cubic surfaces [9] in much higher dimensional algebras. Initial implementations of the latter however revealed that this is not without numerical issues. Naively taking the outer product between $$N-1$$ N - 1 vectors in these high dimensional algebras is not practically possible within the limits of IEEE 64 bit floating point. In this paper we show how established techniques from linear algebra can be used to solve this problem and compute a fast hyperwedge. We demonstrate superior precision and speed, even for low dimensional algebras like 3D CGA.

Steven De Keninck, Leo Dorst


Additional information

Premium Partner

    Image Credits