Decision support for proposal grouping: A hybrid approach using knowledge rule and genetic algorithm

https://doi.org/10.1016/j.eswa.2007.11.011Get rights and content

Abstract

Proposal grouping is a special procedure in the sponsorship process for research projects. In practice, it is conducted to simplify the following procedure of reviewer assignment. As the proposals grow, this procedure becomes complex. Practical managers spend an increasing amount of time struggling for identifying valid proposals, classifying proposals and partitioning proposals into groups as well as maintaining some control over the quality and composition of the resulting groups. This paper proposes an approach for proposal grouping, in which knowledge rules are designed to deal with proposal identification and proposal classification, and the genetic algorithm is developed to search for the expected groupings. In addition, a corresponding system is designed and developed to support the proposed approach. Compared to the previous manual grouping, the proposed approach significantly reduces the time required for grouping, ensures more diverse group composition, and increases overall grouping quality.

Introduction

A large portion of current academic research is sponsored through various agencies and funds with specific interests in different areas of research. Typically, the sponsorship process begins with a call for proposals (CFP), which is distributed to the relevant communities, such as universities and research institutions. Proposals are then submitted to the body (e.g., funding agencies) that issued the CFP. These proposals are assigned to experts for the peer review. Experts normally review the proposals according to the instructions on the rules and criteria of the funding agency. The review results are collected, and ranked based on the aggregation methods (Cook, Golany, & Kress, 2005).

The sponsorship process in China is special for its additional procedure: Proposal grouping. Before peer review, proposals in similar research disciplines are firstly partitioned into groups, and then proposal groups are assigned to reviewers. The goal of proposal grouping is to reduce the assigning times. Undoubtedly, by proposal grouping, the times of assigning proposal groups are less than that of assigning proposals. In most cases, the managers (decision makers) need to take some considerations about the group composition. For example, the proposals in one group should belong to different affiliations. For years, the procedure of proposal grouping has been conducted mainly by hand. In recent years, with the significant increase of proposals, such manual grouping is time consuming and tedious. Moreover, it is very difficult for the managers to regulate the group structure in the manual way. In year 2007, NSFC received more than 70,000 proposals. It can imagine that grouping so many proposals by hand is a so challenging work. In such circumstances, an effective grouping approach is highly desired.

In the past four decades, topics related to the research and development project selection have been widely discussed. A variety of decision models and methods have been developed to support the project selection (Henriksen and Traynor, 1999, Hochbaum and Levin, 2006, Tian et al., 2002). They refer to many aspects of the selection process from different perspectives. Unfortunately, the problem of proposal grouping, to our knowledge, has not been stressed until now in this research area.

Therefore, we seek to some parallel problems in other research areas. According to our literature review, the most similar problem could be found in the area of student group formulation. This problem can be described as partitioning a given number of students into groups that are diverse in their attributes. To solve this problem, many heuristic procedures are proposed (Beheshtian-Adekani and Mahmood, 1986, Weitz and Jelassi, 1992, Weitz and Lakshminarayanan, 1998). A bibliography of applications of heuristic procedures is provided in Weitz and Lakshminarayanan (1998). At the same time, a variety of linear and integer models are also developed for this problem (Baker and Benn, 2001, Reeves and Hickman, 1992, Saber and Ghosh, 2001). Most of these models are solved utilizing traditional optimal technologies.

The above methods for the student grouping problem illuminate our research greatly. However, both the heuristic procedures and the mathematical models are failed to solve our problem directly for the following three reasons. Firstly, the heuristic procedure may provide a poor solution, or in fact, no solution at all. For example, Weitz and Lakshminarayanan apply the heuristic to a decision support system and they find that the grouping procedure may be blocked or only get the partial solutions (Weitz & Jelassi, 1992). Since our approach will be potentially applied to seven departments and more than hundreds of research disciplines, it requires high effectiveness for the grouping approach. Furthermore, given the integer nature of the mathematical models, there is no guarantee of arriving at a solution within a reasonable time limit especially for those large size problems (Dhar & Ranganathan, 1990). To the problem of proposal grouping, a distinct character is its huge number of proposals. So, the traditional optimal technologies can hardly guarantee the expected solutions. In addition, most of these models require rigorous assumptions and only can deal with well-structured problems. The procedure of proposal grouping usually involves not only well-structured problems, but also semi-structured or ill-structured problems. For example, identifying valid proposals is a key step in this procedure, but it cannot be solved well if we only depend on the models.

So, this paper attempts to propose an approach for proposal grouping that can make up the drawbacks of the above methods. This approach has the following two features. Firstly, knowledge rules are designed for the semi-structured or ill-structured problems in the grouping procedure like proposal identification and proposal classification. On the other hand, genetic algorithm (GA) is developed to search for the expected groupings in the rational time. The knowledge rules and genetic algorithm complement each other and provide powerful support to the grouping procedure. It must be mentioned that the original motivation of our study is to support the real-world managers conduct the proposal grouping efficiently and effectively. So, a system is developed to support the proposed approach. The approach and the system are proposed to fit the grouping procedure of Natural Science Foundation of China (NSFC). However, it can be easily modified to cover other similar situations.

The paper is organized as follows. Section 2 presents the proposal grouping problem that triggered our research interest. Section 3 proposes an approach for proposal grouping. Section 4 designs a decision support system to support the approach proposed in Section 3. Section 5 validates the proposed approach by a real life case from NSFC, and some conclusions are drawn in Section 6.

Section snippets

Background information of NSFC

In China, there are more than 30 government funding agencies. They are very similar in the project selection. NSFC (http://www.nsfc.gov.cn) is the largest and most reputable one. It provides financial support for the basic research projects that have great potential of scientific breakthrough or social impacts. In NSFC, there are four bureaus, one general office, three associated units and seven scientific departments. The bureaus, general office and associated units are mainly responsible for

The proposed approach

Based on the manual grouping procedure described in Section 2, this section will present the proposed grouping approach. It is composed of five steps, which are shown in Fig. 3. An explanation of each step follows:

System design and development

To support the grouping approach proposed in Section 3, the corresponding system is designed and developed. The underlying technology includes Microsoft Windows 2000 Server, Microsoft SQL Server, and Microsoft Internet Information Server. The system is built using Microsoft Active Server Page. The major components of the system include the database, the knowledge base, the model base, and user interfaces.

  • (a)

    Database. There are two major categories of data stored in the database: Human resource

Application and evaluation

In order to assist the project selection in NSFC, we developed the Internet-based Science Information System (ISIS, https://isis.nsfc.gov.cn). It has been used for NSFC’s electronic submission and on-line evaluation of proposals, and dissemination of project outputs since May 2000. However, proposal grouping has not been probed and incorporated into ISIS. So, the designed system to support the proposed grouping approach will be complemented to the ISIS.

To validate the proposed approach, the

Conclusions

This paper proposed an approach for the practical problem of proposal grouping. In this approach, two technologies, knowledge rule and genetic algorithm, are utilized to solve the different problems happened in the grouping procedure. A system to support the proposed approach is designed and developed. It is expected to be incorporated into the ISIS in the future. The proposed approach could be extended to other government funding agencies that have to deal with the same problem of proposal

Acknowledgements

This work was partly supported by the National Science Fund for Distinguished Young Scholars of China (Project No. 70525002), National Science Fund for Excellent Innovation Research Group of China (Project No. 70721001), the Competitive Earmarked Research Grant of Hong Kong SAR (CERG, Project No. CityU 118705, CityU 1237/03E) and Strategic Research Grant of City University of Hong Kong (Project No. 7002049).

References (32)

  • A.T.W. Chu et al.

    A comparison of two methods for determining the weights of belonging to fuzzy sets

    Journal of Optimization Theory and Application

    (1979)
  • W.D. Cook et al.

    Optimal allocation of proposals to reviewers to facilitate effective ranking

    Management Science

    (2005)
  • V. Dhar et al.

    Integer programming vs. expert systems: An experimental comparison

    Communications of the ACM

    (1990)
  • M. Gen et al.

    Genetic algorithms and engineering optimization

    (2000)
  • A.D. Henriksen et al.

    A practical R&D project-selection scoring tool

    IEEE Transactions on Engineering Management

    (1999)
  • D.S. Hochbaum et al.

    Methodologies and algorithms for group-ranking decision

    Management Science

    (2006)
  • Cited by (30)

    • A recommender system of reviewers and experts in reviewing problems

      2016, Knowledge-Based Systems
      Citation Excerpt :

      Regardless of which algorithm has been used and whatever type of data is being processed, we have found that the most mature and applicable solutions are those which are based on the idea of a decision support system. Tian et al. [39] and later Sun et al. [7], then Fan et al. [6] and Xu et al. [8] have proposed a complete methodology and systems based on that conception. The systems are designed to support granting processes used by the Natural Science Foundation of China.

    • An incremental genetic algorithm for classification and sensitivity analysis of its parameters

      2011, Expert Systems with Applications
      Citation Excerpt :

      Avcı (2009) implemented classification method by combining genetic algorithm and support vector machine techniques. Fan, Chen, Ma, and Zhu (2007) created an approach for proposal grouping, in which knowledge rules are designed to interact with proposal classification, and the genetic algorithm is developed to search for the expected groupings. Yuen, Wong, Qian, Chan, and Fung (2009) proposed a hybrid model which combines genetic algorithm and neural network for classifying garment defects.

    • Team formation based on group technology: A hybrid grouping genetic algorithm approach

      2011, Computers and Operations Research
      Citation Excerpt :

      Correct management of their human resources is one of the key factors in the success of many companies and organizations [1–22].

    View all citing articles on Scopus
    View full text