An Interview-Based Method for Extracting Knowledge of Skilled Workers at Construction Sites Using Photographs and Deep Learning

In the Japanese construction industry, the number of skilled workers has been decreasing year by year, and a large number of skilled workers will retire in the near future. Furthermore, at construction sites, education is carried out on-the-job-training (OJT) basis in the local environment of the site. Therefore, it is necessary to immediately establish a mechanism to effectively transfer knowledge before skilled workers retire. The first problem is the large amount of data that skilled workers have personally over many years, and it is difficult for them to organize manually. In this research, a system was developed for automatically classifying and extracting a large number of photographs. In this system, object detection with transfer learning is used. As a result of applying it to the special equipment of a nuclear power plant, the F-measure achieved 89%, and the time required for searching photographs was significantly reduced. The second issue is tacit knowledge in the brain of the expert. In general, it is possible to extract knowledge by interviewing experts. In this research, we developed a support system to extract tacit knowledge efficiently and adopted a method for conducting interviews effectively, which called the functional approach (FA) and the semi-structured interview (SSI). By applying FA, we can replace work-related things with “functions” and make many hypotheses for conducting SSI. As a result, the new method improved the time efficiency of interviews by 77.1% and increased the exhaustiveness (the number of knowledge/work step) by 2.9 times.