Automatic question generation for supporting argumentation

In a constructivist learning environment, students are usually asked to solve an authentic problem. To solve the problem, students need to find a solution by researching, experimenting, and posing as well as testing hypotheses. Jonassen ([1], p. 226) proposed that a constructivist learning environment needs to provide some cognitive tools (also referred to as knowledge construction tools) that are used to “visualize (represent), organize, automate, or supplant thinking skills”. One example class of cognitive tools are question generation systems which pose questions related to the problem being solved. The question generation support can potentially be helpful for students during the process of gathering information and building hypotheses: if a student is not able to come up with any idea to investigate the problem to be solved, the learning environment could generate semantics-related questions for the student. Our hypothesis is that automatic question generation may be useful in a constructivist learning environment to support students in solving problems.

LASAD is a constructivist learning environment in which students discuss about a given topic [2]. This web-based collaborative argumentation system provides tools for supporting collaborative argumentation. That is, given a discussion topic, students are required to develop their arguments by creating a diagram. Figure 1 illustrates an argumentation map created by several users collaboratively using the LASAD system. In the system, participants can use typed argument boxes (e.g., claim, fact, explanation) and typed links to represent relationship between the arguments (e.g., support, oppose).

In order to generate questions related to a discussion topic automatically, two question areas need to be addressed:In order to illustrate the semantics-based question generation approach in this paper, we will use the computer-supported collaborative argumentation system LASAD and the natural language English as a study case.

In order to generate questions related to a discussion topic, the topic statement is taken as the basis information. The topic statement is supposed to consist of one or several (grammatically correct) sentences which can serve question generation. Existing question generation approaches can be classified into three classes: syntax-based¹, template-based, and semantics-based approaches.

Syntax-based question generation systems work through three steps: (1) delete the identified target concept, (2) place a determined question key word on the first position of the question, and (3) convert the verb into a grammatically correct form considering auxiliary and model verbs. For example, the question generation system of Varga and Le [3] uses a set of transformation rules for question formation. For subject–verb–object clauses whose subject has been identified as a target concept, a “Which Verb Object” template is selected and matched against the clause. The question word “Which” then replaces the target concept in the selected clause. For key concepts that are in the object position of a subject–verb–object, the verb phrase is adjusted (i.e., an auxiliary verb is used). Varga and Le reported that generated questions achieved a score of 2.45 (2.85) with respect to relevance and a score of 2.85 (3.1) with respect to syntactic correctness and fluency given a scale between from 1 to 4, with 1 being the best score. Values outside and inside in the brackets indicate ratings of the 1st and 2nd human rater.

The second approach, which is also employed widely in several question generation systems, is template-based [4]. The template-based approach relies on the idea that a question template can capture a class of questions, which are context specific. For example, Chen and colleagues [4] developed the following templates: “What would happen if \(\langle \)X\(\rangle \)?” for conditional text, “When would \(\langle \)X\(\rangle \)?” and “what happens \(\langle \)temporal-expression\(\rangle \)?” for temporal context, and “Why \(\langle \)auxiliary-verb\(\rangle \langle \)X\(\rangle \)?” for linguistic modality, where the place-holder \(\langle \)X\(\rangle \) is mapped to semantic roles annotated by a semantic role labeler. These question templates can only be used for these specific entity relationships. For other kinds of entity relationships, new templates must be defined. Hence, the template-based question generation approach is mostly suitable for applications with a special purpose.

In addition to questions that can be generated using phrases in a statement, semantic information related to the issue in a statement can also be exploited to generate semantics-based questions. For example, Jouault and Seta [5, 6] proposed to query information to facilitate learners’ self-directed learning using Wikipedia. Using this system, students in self-directed learning are asked to build a timeline of events of a history period with causal relationships between these events given an initial document (that can be considered a problem statement). The student develops a concept map containing a chronology by selecting concepts and relationships between concepts from the given initial Wikipedia document to deepen his understanding. While the student creates a concept map, the system also integrates the concept to its map and generates its own concept map by referring to semantic information, i.e., DBpedia [7] and Freebase [8]. The authors used ontological engineering and linked open data (LOD) techniques [9] to generate semantics-based adaptive questions and to recommend documents according to Wikipedia to help students create concept maps for the domain of history. The system’s concept map is updated with every modification of the student’s one. In addition, the system extracts semantic information from DBpedia and Freebase to select and add related concepts into the existing map. Thus, the system’s concept map always contains more concepts than the student’s map. Using these related concepts and their relations, the system generates questions for the student to lead to a deeper understanding without forcing to follow a fixed path of learning. One of the great advantages of adopting semantic information rather than natural language resources expected is that the system can give adequate advice based on the machine-understandable domain models without worrying about ambiguity of natural language.

From a technical point of view, automatic question generation can be achieved using a variety of natural language processing techniques which have gained wide acceptance. However, successful deployment of question generation in educational systems is rarely found in literature. Using the semantic information available on the Internet to generate questions to support learning is a relative new research area which is the subject for investigation recently. How can semantic information available on the Internet be processed to generate questions to support students learning in a constructivist environment? The question generation approach proposed in this paper employs all three existing approaches (syntax-based, template-based, and semantics-based) and uses semantic information provided on WordNet to generate questions. The question generation system (QGS) which applies this approach is described in the following section.

The purpose of generating questions in the context of this paper is to give students ideas related to a discussion topic and guiding them how to expand the topic and continue their argumentation. As an input, the question generation system takes an English text from the discussion topic, which is provided by participants of an argumentation system. The text of a discussion topic can be an individual word, a list of words, a phrase, a sentence/question, or a paragraph. Therefore, recognizing, understanding the content of discussion topic clearly, and taking all types of text that are listed in Table 1 as input is the first step of the QGS system.

Semantics-based question generation approaches use a source of semantic information which is related to the topic being discussed. Since in this paper we focus on using information available on the Internet for generating questions, the source of “semantic information” we look for is on the semantic web. For example, Wikipedia provides definitions of words and descriptions of concepts. While Wikipedia might contain incorrect information due to its contribution mechanism, one of the advantages of Wikipedia is that the definition of many concepts is available in many different languages. If we want to develop a question generation for different languages, Wikipedia might thus be an appropriate source. Beside Wikipedia, WordNet also provides a source of semantic information which can be related to a discussion topic. WordNet [10] is an online lexical reference system for English. Each noun, verb, or adjective represents a lexical concept and has a relation link to hyponyms which represent related concepts. In addition, for most vocabulary WordNet provides example sentences which can be used for generating questions. For example, if we input the word “energy” into WordNet, an example sentence like “energy can take a wide variety of forms” for this word is available. If we look for some hyponyms for this word, there are a list of direct hyponyms and a list of full hyponyms. The list of direct hyponyms provides concepts which are directly related to the word being searched, for example, for “energy”, we can find the following direct hyponyms on WordNet: “activation energy”, “alternative energy”, “atomic energy”, “binding energy”, “chemical energy”, and more. The list of full hyponyms contains a hierarchy of hyponyms which represent direct and indirect related concepts of the word being searched. One of the advantages of WordNet is that it provides accurate information (e.g., hyponyms) and grammatically correct example sentences. For this reason, we use WordNet to generate questions which are relevant and related to a discussion topic.

Concerning the types of questions to be generated, Graesser and Person [11] proposed 16 question categories: verification, disjunctive, concept completion, example, feature specification, quantification, definition, comparison, interpretation, causal antecedent, causal consequence, goal orientation, instrumental/procedural, enablement, expectation, and judgmental. The first 4 categories were classified as simple/shallow, 5–8 as intermediate, and 9–16 as complex/deep questions. This question taxonomy can be used to define appropriate question templates for generating questions.

Questions can also be generated using just main concepts available in a discussion topic (Sect. 3.5). The question generation approach proposed in this paper will use hyponyms (Sect. 3.6) and example sentences (Sect. 3.7) provided on WordNet for generating semantics-based questions. The question generation approach consists of the following steps.

The question generation system which is connected with the argumentation system LASAD consists of the following components: the Stanford Parser, a Noun Extractor, a Data Storage, a pool of Question Templates, the ARK syntax-based question generation tool, and WordNet 2.1 as a source of lexical knowledge (Fig. 6).

For parsing English phrases, currently Link Grammar Parser [14] and Stanford Parser [15] (a lexicalized Probabilistic Context-Free Grammar (PCFG)) are two of the best semantic parsers. Link Grammar Parser is a rule-based analyzer, which is essential to obtain accurate results. However, a statistical analyzer parser like Stanford Parser, which is written in Java, is more tolerant with both words and constructions, which are not grammatically correct. Even if there are grammatical errors (e.g., “Parents always does loves their childs”.), a parse tree still can be created by the Stanford Parser. For this reason, we used Stanford Parser to analyze grammatical structure of input sentences.

The noun extractor has been developed to extract main concepts from a discussion topic (cf. Sect. 3.2). It takes a complex text, which can be a word, a phrase, a sentence, or a paragraph as input and returns a list of extracted nouns and noun phrases (\(L_{\mathrm{result}})\) as output. The algorithm starts taking the best parse tree (which has highest parse score) returned by Stanford Parser. The parse tree will be used to obtain a list of nouns. The algorithm for extracting nouns is illustrated as pseudo-code in Fig. 7.

For the purpose of optimizing the time for searching and extracting nouns on WordNet, the Data Storage component works as a history tracer. It stores all the nouns and noun phrases extracted from the given text of users, along with their generated question-lists in an XML file. If the nouns extracted from the discussion topic statement exists in the Data Storage, the QGS only needs to extract the matching questions-list for each noun and noun phrase from Data Storage and returns these lists to users. In this case, the system does not have to generate questions for each noun phrase. Thus, the performance of the system is optimized.

In order to retrieve semantic information, we use the latest version of the WordNet database 2.1 for Windows. The ARK question generation tool has been described in Sect. 3.7. The process of question generation consists of five steps:

In this section, we report on evaluation about the utility of the algorithm for extracting main concepts from a discussion topic, the quality of generated questions and the efficiency of the Plural-to-Singular transformation method.

In this paper we have proposed an approach to generating questions to help students during brainstorming activities in which they expand their arguments when participating in a discussion. The approach proposed in this paper combines three question generation approaches: syntax-based, template-based, and semantics-based. This approach generates not only questions based on the main concepts of a given discussion topic, but also questions based on semantic information available on WordNet.

The question generation method proposed in this paper may have drawbacks due to using question templates. That is, the question templates can be very specific for a special domain as discussed in Sect. 3. We were aware of this problem and tried to define question templates which should be general enough for several discussion domains. For example, the question templates in Table 3 can be used to generate questions for the discussion topic “charity” by replacing the placeholder: (1) What is charity? (2) What do you have in mind when you think about charity? (3) What does charity remind you of? These questions are appropriate for helping participants think about the topic when they have to discuss about charity. Whether all question templates are general enough for other discussion topics, this needs to be evaluated and is a part of our future work. Since the goal of our research is to support students during brainstorming argumentation activities, we also intend to conduct an empirical evaluation study for this purpose.