Requirements for Measuring Inspection System Performance

A recurrent in Ergonomics/Human Factors (EHF) studies of the performance of quality inspection systems is: How to obtain valid measures of system performance. Such measures are needed to determine whether the whole system meets the enterprise’s needs, and also as a baseline for measuring the effectiveness of interventions, e.g., human/automation integration. The methodology used in the paper was to select, review and analyze the findings of over 50 years of inspection studies (e.g., Drury 2019) in a variety of domains starting in manufacturing (e.g., See 2012), but continuing into maintenance, security screening, and medical imaging. In all of these domains a similar need emerges to accurately measure inspection system performance This paper provides an analysis of the issues involved, alternatives for measurement and recommendations for a comprehensive approach. The major issues in performance measurement are presented. For most applications we recommend one type of study for human, automated and hybrid systems: the Test Sample method.