Introducing quantitative analysis methods into virtual environments for real-time and continuous ergonomic evaluations

doi:10.1016/j.compind.2005.12.005

Computers in Industry

Volume 57, Issue 3, April 2006, Pages 283-296

https://doi.org/10.1016/j.compind.2005.12.005 Get rights and content

Abstract

This paper presents our work on methods to link virtual environments (VE) and quantitative ergonomic analysis tools in real time for occupational ergonomic studies. We pursued two distinct approaches: (a) create methods to integrate the VE with commercially available ergonomic analysis tools for a synergistic use of functionalities and capabilities; (b) create a built-in ergonomic analysis module in the VE. The first approach provides the use of established, off-the shelf tools integrated with the VE to create a hybrid application. This integration is performed through the use of APIs provided by the software vendor and existing Internet and communications technologies. The commercial ergonomics tool and the VE run concurrently and integrate their capabilities. The second approach provides the capability to do ergonomic evaluations in a self-contained VE application. In this method, the required ergonomics calculations are built into the VE. Each approach has its own distinct advantages. The use of a commercially available ergonomics tool integrated with a VE provides significant more capability and should be used where detailed and complex ergonomics evaluations are required. However, the process of integration in this approach is more difficult and time consuming. The self-contained VE application is more suited for simple ergonomic evaluations or in cases where the ergonomics algorithms are readily accessible and easily implemented. The two integration strategies are methodically explained and demonstrated using case studies conducted with industry partners. This integrated capability facilitates integration of ergonomic issues early in the design and planning phases of workplace layouts. It provides functionality beyond the capabilities of current commercial off-the-shelf (COTS) solutions. In addition, it contributes to a new trend in the integration of different technology fields for synergistic use in industry.

Section snippets

Introduction and motivation

In the past, workplace ergonomic considerations have often been reactive, time-consuming, incomplete, sporadic, and difficult. Ergonomic experts who were consulted after problems occurred in the workplace examined data from injuries that had been observed and reported. There are now emerging technologies supporting simulation-based engineering to address this in a proactive manner. These allow the workplaces and the tasks to be simulated even before the facilities are physically in place.

Related work

The importance of applying ergonomics to workplace design is illustrated by the Injuries, Illnesses, and Fatalities (IIF) program of the U.S. Department of Labor, Bureau of Labor Statistics [4]. According to the report there were 5.2 million occupational injuries and illnesses among U.S. workers and approximately 5.7 of every 100 workers experienced a job-related injury or illness. Workplace related injuries and illnesses increase workers’ compensation and retraining costs, absenteeism, and

Technologies and tools used

The virtual environment chosen was VADE. The commercial ergonomics analysis tool chosen was Jack. The ergonomic evaluation capability to be integrated with the immersive environment was RULA. The three will be described in some detail now since that is essential to understanding the integration strategy.

Creating a human model in the VE

VADE supported the display and use of just the right hand (wrist, palm and fingers). However, for complete ergonomics analysis, a full human model was required. A parametric, hierarchical human model was created for the VE. The 3D models of different body parts were generated using a parametric CAD system (Pro/Engineer). Thirteen parameters were defined to allow flexibility in adopting the human model for people of different sizes and shapes. The parametric relations were defined for head

Approach 1: formulation of HYBRID system

This approach formulates a method to integrate the VE with the commercial ergonomics tool JACK. Fig. 9 demonstrates the integration of the ergonomic analysis functionality as a plug-in functionality for the virtual assembly design environment. The goal of the integration is to make the human model in Jack follow the motions of the human model in the immersive environment, which in turn is controlled by the data obtained from the tracking device attached to the real human. Another process

Approach 2: single self-contained VE that includes ergonomics

This approach formulates a strategy to create a simple ergonomics analysis capability within a VE to create a self-contained application. The RULA algorithm was integrated into VADE. The overall methodology is described in Fig. 11. The application monitors each frame and the factors used to determine the RULA scores are calculated in real-time. A warning is generated if the posture is not acceptable.

Results

The proposed integration was implemented on an Onyx2 system with six processors. A V-8 helmet [31], CyberGlove and Flock of Birds [32] hardware devices were used for the VE application.

Discussion and conclusions

Both approaches presented in this paper enhanced the functionality of the VE by allowing more rigorous ergonomic analysis functionality and eliminating manual and paper/pencil data capture process. The first approach that used the COTS application Jack had the advantages of decreased effort and risk of development and higher reuse of software as far as the basic ergonomic analysis functionality. However, this method required greater effort in terms of configuration, communication, and

Acknowledgements

The work presented in this paper was supported by the Virtual Assembly Technology Consortium (VATC). Founding members of the consortium: National Institute of Standards and Technology (NIST), PACCAR Inc., Caterpillar Inc., Deere & Company, Komatsu Ltd. and Washington State University (WSU).

Uma Jayaram is an associate professor at Washington State University. Her research interests are in CAD/CAM and VR for engineering. She received her PhD in mechanical engineering from Virginia Tech in 1991. She is also the vice president of Integrated Engineering Solutions.

References (32)

J. Nomura et al.
Virtual reality technology and its application on aircraft assembling
Annual Reviews in Control
(2001)
J.R. Li et al.
Desktop virtual reality for maintenance training: an object oriented prototype system (V-REALISM)
Journal of Computers in Industry
(2003)
L. McAtamney et al.
RULA: a survey method for the investigation of work-related upper limb disorders
Applied Ergonomics
(1993)
J.R. Wilson
Virtual environments applications and applied ergonomics
Applied Ergonomics
(1999)
F. Wang et al.
Virtual reality simulation and its application on aircraft assembling
Journal of Beijing University of Aeronautics and Astronautics
(2001)
U.S. Department of Labor (Bureau of Labor Statistics) Workplace Injuries and Illnesses in 2001. Available on-line via...
B. Das et al.
Participative versus assigned production standard setting in a repetitive industrial task: a strategy for improving worker productivity
International Journal of Occupational Safety and Ergonomics
(1999)
M.W. Riley et al.
Design cost savings and ergonomics
S. Montreuil et al.
From training in ergonomic diagnosis to finding solutions: Assessment of ergo groups that used participatory ergonomics
P.G. Dempsey, Usability of the revised NIOSH lifting equation, Ergonomics 45 (12) (2002) 817–828, Ergonomic Design...

W.M. Keyserling, OWAS: An Observational Approach to Posture Analysis, 2004, available online via...

L. McAtamney, E.N. Corlett, RULA – A Rapid Upper Limb Assessment Tool, available on-line via...

L. Hanson

Computerized tools for human simulation and ergonomic evaluation of car interiors

UGS The PLM Company, E-Factory JACK, 2004, available on-line via http://www.ugs.com/products/ efactory/jack/>, accessed...

S.A. Gill et al.

Using virtual humans to solve real ergonomic design problems

E. Eynard et al.

Generation of virtual man models representative of different body proportions and application to ergonomic design of vehicles

Cited by (95)

A methodological framework to integrate motion capture system and virtual reality for assembly system 4.0 workplace design
2022, Safety Science
Citation Excerpt :
In fact, the ergonomic assessments are carried out on a virtual operator which does not represent the reality, since its movements and anthropometrical data differ from those of the operator for whom the AS workplace is being designed. Recently, the combined use of the mocap system and VR has emerged as a new AS workplace design procedure that can overcome these limitations (Jayaram et al., 2006; Di Pardo et al., 2008). Specifically, this AS workplace design procedure consists in creating first the virtual environment in which the operator is immersed through the use of VR.
Nowadays new products are required more and more often. Since a certain product is assembled only on a specific Assembly System (AS), a new AS has to be designed every time a new product is developed. Similarly, since an AS is constituted by one or more workplaces, new workplaces need to be designed every time a new product is developed. However, this is not feasible considering the time- and resource-consuming AS workplace design procedures currently used (e.g., physical mock-ups and computer-aided systems). New solutions have thus emerged to accelerate the AS workplace design procedures, especially since the advent of Industry 4.0 (I4.0) technologies. Specifically, the combined use of motion capture (mocap) systems and Virtual Reality (VR) has been considered very promising, with many researchers showing its potential. However, to the best of the authors’ knowledge, none of them suggests a clear methodology to follow when designing AS workplaces using the mocap system and VR. In this paper, we thus aim to fill this gap by developing a methodological framework that describes in detail the different steps to be followed. Moreover, the methodological framework has been developed in such a way that both productivity and Occupational Safety and Health (OSH) considerations are included. Furthermore, it encompasses the current ageing workforce scenario by explicitly including the ageing workforce’s main characteristics (reduced flexibility and strength and greater experience of older operators). A simple but representative case study has then been carried out to demonstrate how to use the methodological framework and to prove its validity.
Methodology to integrate ergonomics information in contextualized digital work instructions
2022, Procedia CIRP
Musculoskeletal disorders (MSDs), often caused by working in non-ergonomic circumstances, are the most important category of work-related diseases. Different methods exist for examining ergonomics through a variety of postural evaluation techniques (e.g. Rapid Entire Body Assessment (REBA)), which are mostly performed through manual survey-based observations. Consequently, measuring ergonomics of operators is time-consuming and happens only sporadically, often after problems occur. Furthermore, as an ergonomist must assign a score to the performed activity, the ergonomics evaluation is subjective. These problems can be resolved by continuous and automated ergonomic load monitoring methods that directly provide feedback to operators. In order to monitor the operator without interfering the task at hand, a vision-based ergonomics monitoring system is developed. Here, ergonomic features (e.g. joint angles) are estimated based on multiple video streams, which are used to calculate an objective ergonomic score using a standard ergonomics evaluation technique. Although cognitive operator support systems (i.e. digital work instructions) for manual assembly environments are widely used, they do typically not include ergonomics information. The goal of the methodology presented in this work is to a) formulate or update digital work instructions, b) provide the operator feedback on its ergonomic soundness via work instructions and c) generate ergonomic risk level reports in an automated way based on the outcomes of the vision-based ergonomics evaluator. Making the ergonomic guidelines context-aware ensures that interventions occur at the right time. The methodology is integrated in a framework for context-aware work instructions and is validated via a proof of concept, based on an actual industrial case. Results show that context-awareness enhances user acceptance and ergonomics of the operator.
Health and productivity impact of semi-automated work systems in construction
2020, Automation in Construction
Variability of construction sites and tasks make their automation prohibitively complex. Workers continue to carry out physically demanding tasks which adversely affect their health, safety, and productivity. The flexibility of semi-automated work systems, where operators work in conjunction with machines and robots, is an attractive alternative. It is critical to estimate the anticipated effectiveness of these interventions before integrating them into the current work processes. This study proposes a systematic and objective methodology to assess the value of a semi-automated work system in a construction context, as it pertains to reduced exposure to musculoskeletal disorder risks and productivity improvements. Additional assessments are also suggested for a complete analysis of efficacy. The proposed methodology was validated through an experimental evaluation of a force-assist self-leveling pallet in a masonry task. It provides an objective evaluation of impact on the task showing 40% reduction in joint loads and 10% increase in productivity.
Ergonomic risk assessment based on computer vision and machine learning
2020, Computers and Industrial Engineering
We develop a novel method that performs accurate ergonomic risk assessment, automatically computing Rapid Upper Limb Assessment (RULA) scores from snapshots or digital video using computer vision and machine learning techniques. Our method overcomes the limitations in recent developments based on computer vision or in wearable measurement sensors, being able to perform unsupervised assessment handling multiple workers simultaneously, even under sub-optimal viewing conditions (e.g., poor illumination, occlusions, and unstable camera views). The processing workflow uses open-source neural networks to detect the workers’ skeletons, after which their body-joint positions and angles are inferred, with which RULA scores are computed. The method was tested with computer-generated, controlled real-world image datasets, and with freely available videos taken in outdoor working scenarios. The computed RULA scores were in close agreement with the assessments of seven specialists in the field, achieving a Cohen’s $κ$ over 0.6 in most real-world experiments.
Interaction design for multi-user virtual reality systems: An automotive case study
2020, Procedia CIRP
Virtual reality (VR) technology have become ever matured today. Various research and practice have demonstrated the potential benefits of using VR in different application area of manufacturing, such as in factory layout planning, product design, training, etc. However, along with the new possibilities brought by VR, comes with the new ways for users to communicate with the computer system. The human computer interaction design for these VR systems becomes pivotal to the smooth integration. In this paper, it reports the study that investigates interaction design strategies for the multi-user VR system used in manufacturing context though an automotive case study.
Exploring the potential of Operator 4.0 interface and monitoring
2020, Computers and Industrial Engineering
Citation Excerpt :
VR technologies to easily support reachability and visibility analysis, as well as visual inspection by using virtual models in immersive modality to make users directly simulate task execution. Several VR set-ups have been created to simulate and assess different aspects of manual operations in manufacturing workplaces (Jayaram, Jayaram, Shaikh, Kim, & Palmer, 2006; Wang et al., 2007). Honglun, Shouqian, and Yunhe (2007) used a virtual human model to reproduce the real human characteristics in virtual environments for product ergonomics analysis and demonstrated that digital simulations allow to detect design problems in advance, with the reduction of time and cost in prototype making; Hu et al. (2011) demonstrated the effectiveness of VR simulations for industrial workplace simulation; Abidi, El-Tamimia, Al-Ahmaria, Darwisha, and Rasheeda (2013) performed ergonomics analyses of a sport utility vehicle in a semi-immersive virtual environment to make users aware about their own feeling inside the real car; Aromaa and Vaananen (2016) proved the importance of using virtual prototypes in human-centred design supported by virtual reality environment to assess visibility, climbing, postures, space, reach and use of tools.
In the context of smart factories, where intelligent machines share data and support enhanced functionalities at a factory level, workers are still seen as spectators rather than active players (Hermann, Pentek, & Otto, 2017). Instead, Industry 4.0 represents a great opportunity for workers to become part of the intelligent system; on one hand, operators can generate data to program machines and optimize the process flows, on the other hand they can receive useful information to support their work and cooperate with smart systems (Romero et al., 2016). Diversely from machines, humans are naturally smart, flexible and intelligent, so putting the operators in the digital loop can bring more powerful and efficient factories. The paper aims at defining a theoretical human-centered framework for Operator 4.0, and testing its feasibility and impact on companies, thanks to the integration of human factors in 4.0 computerized industrial contexts. The proposed framework is based on data collection about the workers’ performance, actions and reactions, with the final objective to improve the overall factory performance and organization. Data are used to assess the workers’ ergonomics performance and perceived comfort and to build a proper knowledge about the human asset of the factory, to be integrated with the knowledge derived from machine data collection. The framework is cased on the adoption of an Operator 4.0 monitoring system, which consists of an eye tracking and a wearable biosensor, combined to a proper protocol analysis to interpret data and create a solid knowledge. Virtual prototypes are used to make the workers interact with the digital factory to conveniently simulate the human–machine interaction (HMI) in order to avoid bottlenecks at the shop floor, to optimize the workflows, and to improve the workstations’ design and layout. The study represents a step toward the design of human-centred industrial systems, including human factors in the digital twin. The research approach has been successfully tested on an industrial case study, developed in collaboration with CNH Industrial, for the re-design of assembly workstations.

View all citing articles on Scopus

Sankar Jayaram is a professor at Washington State University. He received his PhD in mechanical engineering from Virginia Tech in 1989. His research interests are in applying VR and visualization to mechanical design and manufacturing. He was an associate editor for the ASME Journal JCISE and is a co-founder of Translation Technologies, Inc.

Imtiyaz Shaikh is a quality assurance engineer at Gehry Technologies Inc. He received his MS (2003) in mechanical engineering from Washington State University. His interests are in ergonomics, VR and CAD/CAM applications.

YoungJun Kim is a PhD student in mechanical engineering at Washington State University. He received his BS in mechanical engineering (1999) from the HongIk University and an MS degree from Washington State University (2002). His research interests include VR applications, geometric modeling, CAD/CAM, and computer graphics.

Craig Palmer is a project engineer leading virtual reality, virtual prototyping, and ergonomics simulation technology development at the PACCAR Technical Center. He received his BS (1995) and his MS (2000) in mechanical engineering from Washington State University.

View full text

Introducing quantitative analysis methods into virtual environments for real-time and continuous ergonomic evaluations

Abstract

Section snippets

Introduction and motivation

Related work

Technologies and tools used

Creating a human model in the VE

Approach 1: formulation of HYBRID system

Approach 2: single self-contained VE that includes ergonomics

Results

Discussion and conclusions

Acknowledgements

Annual Reviews in Control

Journal of Computers in Industry

Applied Ergonomics

Applied Ergonomics

Virtual reality simulation and its application on aircraft assembling

Journal of Beijing University of Aeronautics and Astronautics

Participative versus assigned production standard setting in a repetitive industrial task: a strategy for improving worker productivity

International Journal of Occupational Safety and Ergonomics

Design cost savings and ergonomics

From training in ergonomic diagnosis to finding solutions: Assessment of ergo groups that used participatory ergonomics

Computerized tools for human simulation and ergonomic evaluation of car interiors

Using virtual humans to solve real ergonomic design problems

Generation of virtual man models representative of different body proportions and application to ergonomic design of vehicles