On the Methodology of Automated Processing of Expert Opinions in the Projects and Scientific Research Works Assessment

This article delves into the methodology of automating the processing of expert opinions in project and scientific research evaluations, particularly in competitive settings. It explores the challenges of traditional evaluation methods and proposes a structured approach to enhance objectivity and efficiency. The text discusses various decision theory methods for criteria weight calculation and project ranking, highlighting the advantages and disadvantages of each. It introduces lexicographic methods for reducing the initial set of projects and ranking them by preference. The article also presents a practical example to illustrate the application of the proposed methodology, demonstrating how to select winning projects based on expert assessments. The conclusion emphasizes the potential of recommender systems based on large language models for future enhancements in expert review processes.