4.4.1 ISO Design Principle Ratings
The descriptive findings show that the participants rated all of the assessed design principles as being above average (Table
1). The perceived suitability of the system for conducting continual diagnostic tasks in classrooms was rated quite high (
M = 3.73), and the participants considered the TDSS to be useful for supporting their diagnostic tasks. They also rated positively the self-descriptiveness of the software (
M = 4.07), its controllability (
M = 4.23) and conformity with user expectations
(M = 4.50). The suitability for learning was also assessed positively (
M = 4.25), which is relevant for prospective user trainings. We found the highest ratings for error tolerance (
M = 4.55). However, this scale contained many missing values because the corresponding items refer, for instance, to possible errors with data entry, but the participants only tested the system functions for data analyses.
Table 1
ISO design principle ratings
Suitability for the task | 14 | 3.20 | 4.22 | 3.73 | 0.36 |
Self-descriptiveness | 14 | 2.80 | 5.00 | 4.07 | 0.59 |
Controllability | 13 | 3.14 | 5.00 | 4.23 | 0.53 |
Conformity with user expectations | 14 | 3.00 | 5.00 | 4.50 | 0.54 |
Error tolerance | 4 | 4.00 | 5.00 | 4.55 | 0.53 |
Suitability for learning | 14 | 3.00 | 5.00 | 4.25 | 0.69 |
4.4.2 Verbal Statements Concerning System Usefulness and System Optimization
Verbally, the participants described aspects of system usefulness and possible modifications and extensions. In the following, we systematize participants’ statements about the challenges of teachers’ diagnostic activities during classwork, as set out in the theoretical part of the paper, and the functions for data collection and analysis that were implemented in the TDSS.
Considering classroom instruction as a complex collaborative system, participants considered the TDSS to be useful in reducing the complexity of decisions on differentiated instructional measures for distinct learner groups. Two participants mentioned the potential of system-supported
student grouping that they considered important, such as for implementing inclusive education. In that regard, a technologically assisted group formation necessitates the definition of attributes that guide the process of composing the groups, specifically member attributes (e.g. dominant learning style or domain-specific knowledge) as well as group attributes (e.g. degree of homogeneity of group member characteristics). In the case of automated assignment of students to groups, the grouping techniques still need to be determined (e.g. fuzzy clustering; Maqtary et al.
2019).
Three participants emphasized the potential of the TDSS to integrate multidimensional situational information, which supports adaptive instruction by allowing individualized assignment of tasks with varying difficulty levels, and by facilitating the detection of possibly differential effects of alternative instructional methods (e.g. partner work vs. individual work) on individual learning outcomes. Both application examples refer to the interaction between student characteristics and learning conditions. In that regard, the TDSS is considered useful in supporting instructional decision-making because it merges multiple data sources (student self-reports, standardized tests and descriptions of instructional characteristics).
Besides complexity reduction and data triangulation, some participants noted that it is important that using the TDSS does not produce additional complexity and information load. Three participants thus wished for improvements in the ergonomic design of the teacher user interface (e.g. more effective menu navigation, colour display, wording, information representation, graphs and tabular forms). Two other participants stressed the relevance of ergonomically designing the student user interface as well. Yet another participant suggested creating effective and automated solutions for importing existing student data (e.g. grades, class lists) into the TDSS.
Two teachers stressed that both the induction phase and the routine application of a diagnostic support system should be effective and should not take much time. Concerning time restrictions and time pressure in instructional processes, it is important to give feasible recommendations for implementing the TDSS within teachers’ daily classroom work. We assume that the use of the system would be more effective in student-centred than in teacher-centred phases of instruction because, under the first condition, the teacher is able to retreat from ongoing interactions, has no immediate pressure to act and can reflect about what is going on in detail, based on a thorough inspection of the information provided by the TDSS.
Moreover, the free-text answers reveal that the teachers are well aware of the fact that they often lack access to
valid and reliable student information that is indispensable for adequately addressing student heterogeneity in classes. Participants mentioned, for instance, the need for information on various performance indicators, preparatory training and educational background, absenteeism rates, social backgrounds, behavioural disorders and even addiction issues, in order to create learning tasks and materials that meet the differing prerequisites and needs of individual learners or learner groups. Three participants wished to have the option for an
individualized configuration of the TDSS by generating questions and tests for their students themselves and by integrating such additional items in the data collection process. Four teachers highlighted the usefulness of the TDSS for
plausibility checks of their subjective student-related or situation-related judgements and subsequent instructional decisions. In this respect, the collected multidimensional data could even enrich
collaborative case discussions in teacher teams, in addition to existing methods of information triangulation such as 360-degree feedback, thus facilitating multiple perspectives for decision-making. For the same purpose,
information exchange between teachers, school administration and parents is considered useful for coordinating subjective representations of educational situations (cf. Beck
1996; Salmon et al.
2012). However, with respect to extensive information exchange between different parties outside the single classroom’s boundaries, the participants also stressed the importance of compliance with prevailing
data protection regulations.