An Analytic Tableaux Model for Deductive Mastermind Empirically Tested with a Massively Used Online Learning System

The exact way in which the rating works is described in Klinkenberg et al. (2011).

We do not need the rule for negation because in our formulae only propositional atoms are negated.

The recursive algorithm predicts difficulty of an item that corresponds to the minimal tableau build according to the following rules: each line is read from left to right and feedbacks are analyzed in the order \(oo, rr, gr\), and \(or\). The abstract complexity measure then corresponds to the number of nodes in such a tableau.

In general, the tableau-generating algorithm could be halted either after computing the full tree or immediately after finding a consistent solution. In the first case all conjectures are evaluated and hence \(ooFB, rrFB\), and \(grFB\) feedbacks would be the best predictors. In the second case only the non-redundant conjectures are evaluated, whose number can be less than the total number of conjectures. The second case would allow solving an item with many conjectures much faster, resulting in a lower item rating. In this case \(oo, rr, gr\), and \(or\) would be better predictors.

Note that the DMM-items are difficult if: 1) a line with \(or\) feedback needs to be analyzed in order to find the solution; or 2) a line with \(gr\) feedback needs to be analyzed and one color that is in the solution was not included in any line of the DMM-item. We will show that none of those cases is possible within 2-pin DMM-items of 2 colors. First let us consider 1: In the 2-pin DMM-items there are two possibilities, a line consists of either (a) two pins of the same color, or (b) two pins of different colors. If (a) is the case the \(or\) feedback implies that one of the pins is of the correct color but in a wrong position, which does not make sense, because the other pin is of the same color and in the only remaining position. If (b) is the case the \(or\) implies means that one pin is not in the solution, which implies that the two pins have to be of the same color (since there are only two colors). But then it means that one of the pins in the line was in fact of the correct color and position to start with, which was not reflected by an adequate \(g\) feedback. This renders such a case impossible. Let us now consider 2: Here, a line must consist of two flowers of the same color (otherwise there would be no color that is not included in any of the lines, because there are only 2 colors). But then, the \(gr\) feedback does not give a unique solution. The latter is only possible if another line introduces the missing color. In conclusion, the properties of 2-pin DMM-items of only 2 colors make it impossible for any of them to fall into the “difficult” category.