A Question-Answering System that Can Count

This paper proposes a conceptual architectural design of Question-Answering (QA) system that can solve “counting” problem. Counting problem is the inability of QA system to produce numerical answer based on retrieved rationale (in text passage) containing list of items. For example, consider “How many items are on sale?” as question and “Currently shampoo, soap and conditioner are on sale” as retrieved rationale from text passage. Normally, system will produce “shampoo, soap and conditioner” as an answer while the ground truth answer is “three”. In other words, system is simply unable to perform the counting process needed in order to correctly answer such questions. To solve this problem, QA system architecture with following components is proposed: (1) A classifier to determine if given question requires a counting answer, (2) A classifier to determine if current system’s answer is not numeric, and (3) A counting method to produce numerical answer based on given rationale. Despite looking like a whole system, the proposed architecture is actually a modular system whereby each component can operate independently (allowing each component to be separately implemented by other systems). In essence, this paper intents to demonstrate a general idea of how the defined problem can be solved using a modular system, that hopefully also opens up more flexible enhancements in the future.