Semantic Pattern-Based User Interactive Question Answering: User Interface Design and Evaluation

This paper presents two user interfaces for a pattern-based User Interactive Question Answering system, which are designed to encourage users to ask questions by using semantic patterns. One is a Guide-Based User Interface (GBUI), which can guide users with clear instructions. However, it involves many steps and the operation may become tedious. The other is a Recommendation-Based User Interface (RBUI), which recommends a few relevant patterns containing automatically suggested details for each free-text question. However, the recommended patterns may not always be satisfactory and sometimes the user’s revision is needed. In comparing these two user interfaces, we propose a new Complexity Evaluation Model (CEM) to evaluate the complexity on the basis of user log study and a realistic focused user study. The results of the study user logs, which cover a test set of 1605 users and 488 semantic patterns, show that RBUI can improve the complexity of GBUI by 39.8% on average. The improvement is also confirmed by the user study. It has thus become clear that RBUI can improve the usability of the UIQA system in terms of helping the system accumulate high quality pattern-based questions.