This research is aimed to determine how cognitive apprenticeship framework could be utilized in designing and utilizing an ontology based cognitive support system (OBCSS). It is also aimed to evaluate the use of OBCSS in educational settings to determine how much mental effort is needed for students to use it and to determine whether use of OBCSS decreases disorientation and supports navigation as stated in literature. Scales are used to get perceived behavioral data from the participants. This data is used to investigate experiences of users who have gender and computer experience diversities. Personal information form, mental effort evaluation scale and perceived disorientation scale are used to collect data. Log data is collected from learners’ interaction with system. Sample of this study are fifth and sixth grade students at three state schools and one private school in Central Turkey. At the first phase, data is collected from 83 students, and this data is used for Reliability and Validity of the disorientation scale. At second phase data is collected from 89 students, and this data is used for the analysis of the OBCSS. System is evaluated by collecting data from real time users. According to results of statistical analyses, mental efforts of learners did not differ across gender, daily average computer and internet use, and frequency of internet use for studying. Similarly, perceived disorientation and perceived ease of use did not differ across gender, daily average computer and internet use and frequency for using internet to study. According to log data, OBCSS can provide different personalized learning paths based on the current knowledge of learners.
Abbreviations
AHAM
Adaptive Hypermedia Application Model
ANOVA
Analysis of Variance
OBCSS
Ontology Based Cognitive Support System
UML
Unified Modeling Language
Introduction
As data on the web is constantly growing, it is a challenge for instructors and learners to reach out the resources appropriate to their needs, tasks or goals. Even if the resource is found, it is quite a challenge to further understand if requirements of teachers or students are met by using these resources. These problems were tackled with semantic web technologies in teaching and learning domain with the use of ontologies in educational environments (Aroyo & Dicheva, 2004; Cristea, 2004).
In early 2000s, some researchers claimed that using ontologies in artificial intelligence and education will help solve problems faced in searching process (Mizoguchi & Bourdeau, 2000). Later, researchers explained the use of ontologies in educational environments and how the ontologies will be utilized in teaching and learning process (Stojanovic et al., 2001; Brase & Nejdl, 2004; Lytras et al., 2003; Aşkar et al., 2007; Monachesi et al., 2008). Accordingly, the use of ontologies in educational environments can be classified in three groups:
Content access (Aroyo et al., 2002; Mitrovic and Devedzic, 2004; Lemnitzer et al., 2008; Lama et al., 2012),
Content creation (Simon et al., 2004; Castello and Gauthier, 2006; Boyce and Pahl, 2007; Oprea, 2011; Manganello et al., 2013)
Personalization (Henze et al., 2004; Karampiperis and Sampson, 2005; Fok, 2006; Vargas-Vera and Lytras, 2008; Chen et al., 2011)
Advertisement
Although research indicate that ontologies contribute teaching and learning process, there are still some deficiencies to address: First, contributions of ontologies to the process are not evaluated with experimental research in K-12 schools. Secondly, learners’ cognitive processes were merely considered in evaluation and the use of ontology. Considering learners’ cognitive profiles and expanding the research to K-12 educational settings will contribute the use of ontologies in teaching and learning process. Moreover, how ontologies would provide guidance to learners in their cognitive skill development remains unexplored. Providing an ontology-based teaching model and testing it in a K-12 setting would be the contribution of this study to the existing literature.
Therefore, the purpose of this research is to propose an ontology-based teaching framework to be used in educational settings. For this purpose, this research is aimed to determine how cognitive apprenticeship framework could be utilized in designing and implementing an Ontology Based Cognitive Support System (OBCSS). It is also aimed to evaluate the use of OBCSS in educational settings to determine how much mental effort is needed for students to use it and to determine whether use of OBCSS decreases disorientation and supports navigation as stated in literature. More specifically, this research sought answers to the following questions:
1.
How can OBCSS be designed and developed to be used in cognitive skill instruction?
2.
What kind of a learning path do learners build by using OBCSS?
3.
By using OBCSS, do ease of use and perceived disorientation differ across gender and computer usage?
Theoretical framework
Cognitive apprenticeship model
Cognitive apprenticeship is a teaching model that stands on traditional apprenticeship, but also integrates with elements of school education (Brown et al., 1989). Purpose of cognitive apprenticeship is to visualize thinking process of a learning activity either in student’s or in teacher’s perspective (Dennen, 2004). Cognitive apprenticeship successfully integrates professional development of learners and supports them to construct their own thinking processes.
According to Ghefaili (2003), cognitive apprenticeship model is based on the following models and theories:
Socio-cultural Theory of Learning
Vygotsky’s Zone of Proximal Development
Situated Cognition
Traditional apprenticeship
Learning environments that use cognitive apprenticeship model shall have following items and functions (Brown et al., 1989; Collins et al., 1991):
1.
Content: Indicates type of information that is necessary for expertise
a.
Domain Knowledge: Consists of concepts, procedures and facts related to a specific subject matter. Sources for this type of information are lecture books or lecture notes.
b.
Heuristic Strategies: Consists of necessary techniques and approaches to complete a task. Experts achieve this type of strategies by solving problems.
c.
Control Strategies: Controls the process to complete a task. These strategies are related to managing difficulties that are met during the process.
d.
Learning strategies: These strategies are used to learn domain knowledge, heuristic strategies and control strategies. General strategies to learn a new domain or specific strategies to learn complex tasks are example for these strategies.
2.
Method: Represents the methodologies to increase development of expertise.
a.
Modelling: In this method, as expert performs a task, learner observes the process to complete the task and creates a conceptual model of the process.
b.
Coaching: In this method, while learner is performing a task, he/she shall be supported, by being given clues and reminded in order to increase his/her performance.
c.
Scaffolding: In this method, teacher shall help student during a task.
d.
Articulation: In this method, teacher encourages student to specify his/her ideas and knowledge related to subject matter
e.
Reflection: In this method, students compare their problem solving process to an expert or their friends.
f.
Exploration: In this method, teacher guides students to solve problems by themselves.
3.
Sequencing: Represents sequencing of learning activities
a.
Increasing Complexity: Tasks are ordered in a way that they require more skills and more concepts in every step.
b.
Increasing Diversity: Tasks are ordered in a way that they require more diversity of strategies and skills
c.
Global before local skill: Before parts of the task are completed, general conceptual model of the task shall be provided
4.
Sociology: Sociology is related to social attributes of learning environment
a.
Situated Learning: Student’s environment that he/she performs tasks or solves problems shall be identical to real world environment for related tasks and/or problems
b.
Community of practice: Students shall communicate each other constantly in the learning environment
c.
Intrinsic motivation: Students shall construct their own personal goals to achieve skills and solutions
d.
Exploiting cooperation: Students shall study together to support their cooperative problem-solving skills
In cognitive apprenticeship literature, this model is successfully used e-learning environments beginning in 1990s (Murray et al., 2003; Liu, 2005; Dickey, 2008; Lee, 2011; Chandra and Watters, 2012; Khousa et al., 2015, Tawfik et al., 2018). San Chee (1995) used cognitive apprenticeship for teaching SMALLTALK programming language and stated that this application is defined as user friendly and supportive by students. In a more recent study, Kuo et al. (2012) created a web based environment to investigate if cognitive apprenticeship model improved cooperative learning skills and indicated that cooperative learning with cognitive apprenticeship model was very useful for field dependent students. In another research, Tsai et al. (2012) created a cognitive apprenticeship based system to improve argumentation cognitive skill. Results of the study show that students’ argumentation performance was improved by having used this web based cognitive apprenticeship system.
Adaptive hypermedia systems
An adaptive hypermedia system functions based on a user model that contains and utilizes the model to adapt the system for the user (Brusilovsky, 1996). It is believed that students access information as quickly as possible without cognitive overloading in an adaptive hypermedia environment. Brusilovsky (1996) stated that adaptive learning systems would have three criteria to provide this functionality:
Adaptive learning system shall be a hypertext or hypermedia system.
System shall have a user model
System shall adapt content according to user model that it contains.
Advertisement
Figure 1 shows classic structure of an adaptive learning system.
Fig. 1
“User modeling – adaptation” loop in adaptive systems, (Brusilovsky, 1996)
×
Adaptation is constructed in two phases: At the first phase, a user model is created. In this phase, as user interacts with the system, user information is gathered and kept in the user model. So adaptation is made according to this information in user model. As the user interacts with the system, the user model is updated accordingly. Adaptation can be made through information about the users based on their goals, preferences or needs (Martins et al., 2008; Ouf, Abd Ellatif, Salama, & Helmy, 2017; Wan & Niu, 2018).
At the second phase, adaptation is realized. In this phase content/presentation and/or navigation is adapted (Brusilovsky, 1996). Content/presentation can be adapted in five methods (Brusilovsky, 1996):
Additional explanations: In this method explanations are given user according to his/her level of knowledge. System hides too complicated or too simple information from user.
Prerequisite explanations: System gives prior information about a concept before presenting main information.
Comparative explanations: System emphasizes concepts related and/or unrelated with the concept that will be taught
Explanation variants: Explanations are varied according to user model by detailing page or section content
Sorting: Information that will be presented to user is sorted according to user history or level of information
Navigation can be adapted in four ways (Brusilovsky, 1996; De Bra and Calvi, 1998):
Direct guidance: Next presentation that will follow the current one is linked with a “Next” button.
Sorting: As in content adaptation, content is sorted based on users’ level of knowledge.
Hiding: Links that are not suitable for user is hided in current page.
Map adaptation: The content that has been studied by user is marked among the others in a graphical way.
AHAM reference model
In this research, OBCSS is based on AHAM (Adaptive Hypermedia Application Model) reference model which is one of the highly accepted models in the literature (Knutov et al., 2009; Papadimitriou, 2017). AHAM reference model is emerged as a base model in adaptive hypermedia design (De Bra et al., 1999). In this model basic components of adaptive hypermedia are defined as domain model, user model and adaptation model (Fig. 2).
In AHAM reference model, Runtime Layer keeps system working pages and interaction with user so that progress of user can be monitored. Presentation specifications layer updates interface model according to user status. Within-component layer, materials that will be presented to user will be kept, and it uses anchoring layer to communicate with storage layer.
In AHAM, it is important to use users’ actions to update the model, so that adaptation and presentation of the content could be made appropriate to user needs. For this purpose, adaptation rules are used. These rules also use information stored in the user model, to find relevant user model updates. Domain model cooperates with the user model to specify user’s knowledge about the current subject matter and to perform the adaptation of the content. Here pedagogical rules help the content adaptation.
Mental effort
Salomon (1984) defined mental effort as numbers of non-automatic elaborations to solve a problem or to learn a topic. Considering this definition, people can easily complete tasks that they get used to, with a little mental effort, and they use more mental effort for new or complicated tasks (Clark et al., 2006; Kalyuga, 2009).
In some of research, perceived total cognitive load is measured in a likert scale tool, which was merely named cognitive load (Van Merriënboer et al., 2006). It is assumed that, mental effort reflects total cognitive load as it refers to cognitive capacity as a dimension of cognitive load (Paas et al., 2003). Mental effort is measured with personal scoring tools and psychological measures (such as heart beat rate, brain activity, eye activity –blink rate, pupil enlargement-), but personal scoring tools are commonly used as it is easier to use (Paas et al., 2003; Van Merriënboer, et al., 2006; De Jong, 2010; Lin & Kao, 2018). Eye tracking (Wang, Tsai, & Tsai, 2016) and personal scoring tools are used because it is assumed that people can observe their own cognitive processes and score their mental effort (Paas et al., 2003).
Paas et al. (2005), researched relation between mental effort and learner motivation and developed a new methodology to calculate and visualize the effect of teaching conditions on learner motivation. As a result, learner group with higher prior knowledge, scored higher participation scores (higher performance and lower mental effort). In another research, Poehnl and Bogner (2013) used “alternative conceptions” and focused the effect of mental effort on conceptual change process. Researchers observed no raise of learning in group taught with alternative conceptions method.
Disorientation
Disorientation is defined as the feeling of being lost during navigation process in hypertext environments (Cangöz and Altun, 2012; Bhatti, Ismaili, & Dhomeja, 2017). According to Conklin (1987), source of disorientation is a design problem, which occurs during creation of paths in hypertext environment. People feel disoriented while navigating if they don’t know where to go next, if they know where to go next but they don’t know how to go or if they cannot make any connection between their current location and whole document structure (Edwards and Hardman, 1989). Thus, people feel disappointed and they lose their interest keeping up with the content in the environment (McDonald and Stevenson, 1998).
Most of the disorientation research state that users experience such disorientation in web-based environments, and further suggest that users employ several strategies to prevent themselves from getting lost, among which are using linear navigation (McDonald and Stevenson, 1996); utilizing navigational support such as bookmarks, history lists or maps (Dias et al., 1999). In addition, content designers are advised to employ paged navigation (Lee, 2005); navigational support based on users’ prior knowledge (Amadieu et al., 2009); and, content suggestions utilizing users’ computer experience (Shih et al., 2012).
Ease of use
Ease of use is the perception of using an innovation without putting too much effort on it (Davis, 1989; Rogers, 2003). Ease of use as a construct is an important factor to make users adopt an application (Davis, 1989) and is commonly used in evaluating designs of web-based teaching/learning environments (Revythi & Tselios, 2017). Liaw (2008) stated that satisfaction and perceived ease of use have positive effect on use of e-learning environments. Sun et al. (2008) pointed that perceived ease of use is a critical factor that effects satisfaction. Ali et al. (2013) empirically proved their adoption model and stated that ease of use and perceived usefulness has a significant effect on adoption.
Methodology
This section is divided in two parts: In the first section, design process of OBCSS is explained. In the second section, research model, participants, instruments and statistical analyses in the evaluation of OBCSS effectiveness are described.
Design of ontology based cognitive support system
Design of OBCSS is based on CogSkillNet- cognitive skill ontology which is developed by Aşkar and Altun (2009). CogSkillNet is a domain ontology that includes cognitive skills in K12 curriculum. CogSkillNet also aims to increase the use of learning objects by representing cognitive skills with ontology based on the relationship between cognitive skills and learning objects (Aşkar and Altun, 2009).CogSkillNet class and instances are given in (Fig. 3).
Fig. 3
CogSkillNet class and instances (Aşkar and Altun, 2009)
×
Design of adaptive hypermedia
OBCSS is designed according to AHAM. OBCSS has a domain model, a user model, an adaptation model, a presentation specifications layer and a within-component layer. Domain model is based on CogSkillNet cognitive skill ontology (Aşkar and Altun, 2009). Cognitive skills and relations among skills are defined in ontology. Learning objects and cognitive skill relationship is also defined in ontology so that relationships can be queried within CogSkillNet. As a result of this query, cognitive skills that will be presented to learners are determined so that learning objects that are related to these skills are presented to user.
User model in OBCSS is referred to learner model and is based on the learner model that is suggested by Kaya and Altun (2011). According to this model, in order to determine learners’ prior knowledge, OBCSS applies a pre-test to users at first. According to pre-test results, learner model is updated and the rest of the interaction between system and learner is added to the learner model.
Adaptation model, as the name suggests, adapts the content for learners based on the pre-test results by using content adaptation techniques as suggested by Brusilovsky (1996). The model suggests the system check for the wrong answers in pre-test and suggest related content for learners to study according to his/her wrong answers. Adaptation model also uses all defined techniques presented in section “Adaptive Hypermedia Systems”. Adaptive Hypermedia Systems.
Modelling OBCSS with cognitive apprenticeship framework
Previous research indicated that cognitive apprenticeship model can be used in web-based learning environments as a framework (Brown et al., 1989; Dennen, 2004; Wang and Bonk, 2001). OBCSS utilizes cognitive apprenticeship model as a framework and applies the related components in this framework. Elements that are used in OBCSS are summarized in Table 1.
Table 1
Use of cognitive apprenticeship items
Cognitive Apprenticeship Framework Item
Representation in OBCSS
Content
• Domain knowledge is modelled with concepts and relations between concepts defined CogSkillNet.
• Content is defined by learning objects and their relation with cognitive skills in CogSkillNet.
• Heuristic strategies are used to find cognitive skills that are related to other cognitive skills in CogSkillNet.
• Control Strategies are used to check learners’ errors and make them practice to fix these errors.
Method
• Modelling technique is used to make learners practice cognitive skills by using learning objects that have application presentation strategy
• Coaching and Scaffolding techniques are used to forming thinking processes of learners by making them study the cognitive skills that they gave wrong answers in pre-test. Learning objects that have presentation strategy lecturing of lecturing are used for this process
• Reflection and Articulation techniques are used to make learners comment on anything during their study about environment or learning process.
Sequencing
• Sequencing technique is used to sequence cognitive skills in mandatory study or practice phase by an order of increasing complexity and increasing diversity
Design of OBCSS software
OBCSS software design has been started with defining use cases and modules. A total of six modules are designed for OBCSS: Authentication module checks user if registered. Learner module is responsible for keeping and updating the learner model. Adaptation module adapts content after pre-test and practice phases for each user. Navigation module is responsible for displaying modules and pages to users after login. Database module is an interface for all modules to database. Ontology module is the interface for CogSkillNet to all modules.
OBCSS software is designed using UML (Unified Modeling Language) (UML, 2013). UML activity diagrams and sequence diagrams are used to represent the interaction between modules and work flow. It is decided to use Java programming language and Eclipse software development environment for software development. OBCSS is designed as a web-based application to serve multiple users at once (Fig. 4). Vaadin java framework is used for development and Apache Tomcat 7.0 is used as a web server.
Fig. 4
General OBCSS Design
×
Jena semantic web framework is used to run SPARQL queries on CogSkillNet. MySQL (MySQL, 2013) database server is used for OBCSS database.
First version of CogSkillNet is developed using OWL 1 version. During development of OBCSS, CogSkillNet ontology is also upgraded to OWL 2 version to meet some missing functionalities like property chains, extended data types or reflexive properties.
Evaluation of ontology based cognitive support system effectiveness
Design
This research is designed as causal comparative research. Causal comparative research aims to investigate cause and effect of differences between groups or aims to define causes of a situation or factors that affect causes of this situation (Johnson and Christensen, 2008). Causal comparative research tries to define cause and effect relations. By this point of view, Causal comparative research reminds experimental researches, but unlike experimental research, researchers do not interfere the situation that is investigated.
Participants and data collection process
Participants to this study were 89 middle school students. They were voluntarily invited to participate in this study from three state schools and one private school in a metropolitan city in central Anatolia. Once a consent form is obtained, 83 middle school students had been asked to join the first phase of the research, which was to be designed to establish the validity and reliability of perceived disorientation scale. At the second phase, 89 students were invited to use the system in their mathematics, biology, and Turkish lessons. All the participants were agreed to answer surveys, an all of them completed surveys. Effective response rate of the surveys is %100. The content presented with this tool was mainly to provide instruction on cognitive skills so that they could explore each cognitive skill and apply them in their courses. This part of data collection process was completed in 2 weeks and the data obtained from OBCSS use was analyzed to explore the research questions.
Instruments
In this research four data collection tools are used (Demographic information form, mental effort evaluation scale, perceived disorientation scale, and user logs-saved by OBCSS). Demographic information form is used to obtain age, computer use and frequency etc. data. The other tool, Mental Effort Evaluation Scale is developed by Zijlstra (1993) and adapted to Turkish by Çevik (2012). It is a nine-item scale and 1 stands for “extreme mental effort” and 9 stands for “no mental effort” when completing the task. Perceived disorientation scale, developed by Ahuja and Webster (2001) and is adapted to Turkish by Cangöz and Altun (2012). It is a five–Likert Scale from “Strongly agree” to “Strongly disagree”. There are 10 statements in the scale, first seven questions measure disorientation and last three questions measures ease of use. User logs included navigation patterns (visited learning objects, related cognitive skills, etc.), as well as screen behavior data (i.e., time spent on each page, visits, revisits, etc.).
Analysis
Data is analyzed in two phases: First descriptive analysis is run through scale data by using t-tests and ANOVA (Analysis of Variance) for gender, daily average computer use and daily average internet use. Second, log data is evaluated by content analysis based on students’ comments.
Perceived disorientation scale was originally developed for graduate students. Therefore, a confirmatory factor analysis is applied to validate the tool to be used with secondary school students. LİSREL 8.7 is used and finally fit indexes are acquired as: [χ2 (34, N = 83) = 37.57, p = .22, RMSEA = 0.046, S-RMR = 0.062, GFI = 0.92, AGFI = 0.86, CFI = 0.98, NNFI = 0.97, IFI = 0.98]. These indexes show that data shows acceptable and/or perfect fit. To specify the consistency of answers of the scale, reliability analysis is checked with SPSS 17.0 and Cronbach Alfa is found to be 0.86 for disorientation and 0.87 for ease of use. These values show that scale is highly reliable. Lastly, log data is collected through interaction of students with OBCSS.
Findings
Implementation of ontology based cognitive support system
OBCSS consists of six modules: Authentication module, learner module, adaptation module, navigation module, database module and ontology module. The algorithm and the flowchart representing the interactions between these modules is presented as follows (Fig. 5):
1.
First student logs in by using user name and password.
a.
If user name and/or password is wrong return to step 1
2.
Navigation module decides which page will be displayed to user
3.
If pre-test is not applied, adaptation module displays pre-test
4.
If pre-test is applied and mandatory study is not completed, cognitive skills related to wrong answers are queried from CogSkillNet and mandatory study is prepared
5.
If pre-test and mandatory study is completed, practice is applied
6.
If practice is completed, cognitive skills in CogSkillNet are queried and independent study is prepared
7.
All interaction data of user is recorded to database
8.
Learner module is an interface to database and ontology related queries.
Fig. 5
Interaction of OBCSS Modules
×
As the OBCSS language is Turkish, the following screenshots (Figs. 6, 7, 8 and 9) are provided in original language of system with a translation of the Figs. A pre-test question is represented in Fig. 6. As seen in this screenshot, students are asked about questioning as a cognitive skill. Question is about a research to understand blood circulation in human body. Research questions are what is blood circulation, where to start, where to end, how blood is purified in body? So, students are asked to identify the related cognitive skills to these questions. The button at top left is used to add comments, and the button at top right is used to log out. Buttons at bottom right are used to navigate previous and next questions.
Fig. 6
Pre-test question related to query skill
×
After administering the pre-test, results are displayed to user Fig. 7. In this screenshot wrong answers are listed along with the related cognitive skills. From here, OBCSS proposes students to study both wrong cognitive skills and other skills that are ontologically related to wrong ones.
Fig. 7
Pre-test results
×
A practice item is presented in Fig. 8. In this figure, a practice learning object related “to divide” presented to students. In this learning object, whole-piece relation is presented. Title of this page is “Task 2” and button is used to navigate to next task.
Fig. 8
Practice learning object related to divide skill
×
Independent study interface is shown in Fig. 9. Title of this page is “Study at your own pace”. System lists all cognitive skills to students. On the left side of the page, cognitive skills and sub skills are presented. On the right side of the page, learning object related to selected skill is presented. On this screenshot, “to separate” as a skill is selected and learning object content is about this skill and the ontologically related ones.
Testing of ontology based cognitive support system
OBCSS software is verified through unit tests (Patton, 2005; Myers et al., 2011), acceptance test (Hsia et al., 1997; Myers et al., 2011) and beta test (Patton, 2005). Unit tests are created and run in development process. After development process has finished, acceptance test is run and beta test of OBCSS is run with eight sixth grade students of a state school in Turkey.
Data analysis and interpretation
In this section t-tests and ANOVA results are listed for mental effort, disorientation and ease of use through “gender”, “daily average computer use”, “daily average internet use” and “frequency of studying using internet” variables. Daily average internet use is internet usage of a student for general purposes (e.g. online games, online video, social media, online music, chat etc.). Frequency of “studying using internet” refers to the internet usage of a student for only study purposes.
Mental effort, perceived disorientation, and gender
To investigate the effect of gender on mental effort and perceived disorientation, t tests were applied (Tables 2 and 3 respectively). In order not to make Type I error, Bonferroni correction adjustment was made.
Table 2
T test results for effect of gender to mental effort
Gender
N
Mean
Standard Deviation
t
p
Mental Effort
Girl
39
4.79
2.45
0.651
0.517
Boy
50
4.42
2.87
Table 3
T test results for effect of gender to perceived disorientation
Gender
N
Mean
Standard Deviation
t
p
Perceived disorientation
Girl
39
2.59
0.83
1.625
0.108
Boy
50
2.27
0.96
Ease of use
Girl
39
3.47
0.94
−1.120
0.266
Boy
50
3.71
1.07
According to the results as presented in Tables 2 and 3, no significant difference between gender was found in mental effort (t = 0.651, p > 0.05), perceived disorientation (t = 1.625, p > 0.05), and ease of use (t = − 1.120, p > 0.05) in their use of OBCSS .
Mental effort and daily average computer use
To investigate effect of daily average computer use to mental effort, ANOVA is applied (Table 4).
Table 4
ANOVA results for effect of daily average computer use to mental effort
N
Mean
Standard Deviation
F
p
Less than 1 h
43
4.56
2.49
4.547
0.424
Between 1 and 3 h
30
4.10
2.61
Between 3 and 5 h
9
3.78
2.68
More than 5 h
7
7.86
2.26
Total
89
4.58
2.68
According to results in Table 4, no significant difference was found between groups of daily average computer use, in regard to mental effort F(3, 85) = 4.547, p = 0.424.
Mental effort and daily average internet use
To investigate effect of daily average internet use to mental effort, ANOVA is applied (Tables 5).
Table 5
ANOVA results for effect of daily average internet use to mental effort
N
Mean
Standard Deviation
F
p
Less than 1 h
48
4.19
2.39
2.227
0.091
Between 1 and 3 h
25
4.52
2.74
Between 3 and 5 h
9
5.11
3.58
More than 5 h
7
6.86
2.47
Total
89
4.58
2.68
According to results in Table 5, no significant difference was found between groups of daily average internet use, in regard to mental effort F(3, 85) = 2.227, p = 0.091.
Mental effort and frequency of studying using internet
To investigate effect of frequency of studying using internet to mental effort, ANOVA is applied (Tables 6).
Table 6
ANOVA results for effect of frequency of studying using internet to mental effort
N
Mean
Standard Deviation
F
p
Once a week
22
4.91
2.77
1.229
0.305
Once in 5 days
6
2.50
2.07
Once in 3 days
17
4.88
2.47
Once in 2 days
23
4.26
2.37
Every day
21
4.95
3.12
Total
89
4.58
2.68
According to results in Table 6, no significant difference was found between groups of frequency of studying using internet, in regard to mental effort F(4, 84) = 1.229, p = 0.305.
Perceived disorientation and daily average computer use
To investigate effect of daily average computer use to perceived disorientation, ANOVA is applied Table 7.
Table 7
ANOVA results for effect of daily average computer use to perceived disorientation
N
Mean
Standard Deviation
F
p
Perceived disorientation
Less than 1 h
43
2.45
0.96
1.261
0.293
Between 1 and 3 h
30
2.50
0.92
Between 3 and 5 h
9
1.85
0.84
More than 5 h
7
2.46
0.52
Total
89
2.41
0.91
Ease of use
Less than 1 h
43
3.53
1.05
0.316
0.814
Between 1 and 3 h
30
3.60
0.85
Between 3 and 5 h
9
3.74
1.58
More than 5 h
7
3.90
0.59
Total
89
3.60
1.01
According to results in Table 7, no significant difference was found between groups of daily average computer use, in regard to perceived disorientation (F(3, 85) = 1.229, p = 0.305) and ease of use (F(3, 85) = 0.316, p = 0.814).
Perceived disorientation and daily average internet use
To investigate effect of daily average internet use to perceived disorientation, ANOVA is applied Tables 8).
Table 8
ANOVA results for effect of daily average internet use to perceived disorientation
N
Mean
Standard Deviation
F
p
Perceived disorientation
Less than 1 h
48
2.46
0.89
0.848
0.471
Between 1 and 3 h
25
2.48
1.02
Between 3 and 5 h
9
1.95
0.90
More than 5 h
7
2.42
0.67
Total
89
2.41
0.09
Ease of use
Less than 1 h
48
3.70
0.87
1.952
0.127
Between 1 and 3 h
25
3.29
1.07
Between 3 and 5 h
9
4.14
1.24
More than 5 h
7
3.38
1.22
Total
89
3.60
1.01
According to results in Table 8, no significant difference was found between groups of daily average internet use, in regard to perceived disorientation (F(3, 85) = 0.848, p = 0.471) and ease of use (F(3, 85) = 1.952, p = 0.127).
Perceived disorientation and frequency of studying using internet
To investigate effect of frequency of studying using internet to perceived disorientation, ANOVA is applied (Table 9).
Table 9
ANOVA results for effect of frequency of studying using internet to perceived disorientation
N
Mean
Standard Deviation
F
P
Perceived disorientation
Once a week
22
2.29
0.69
0.767
0.550
Once in 5 days
6
2.09
0.68
Once in 3 days
17
2.26
0.79
Once in 2 days
23
2.54
0.90
Every day
21
2.61
1.24
Total
89
2.41
0.09
Ease of use
Once a week
22
3.60
1.11
0.745
0.564
Once in 5 days
6
3.94
0.49
Once in 3 days
17
3.70
0.95
Once in 2 days
23
3.72
0.86
Every day
21
3.30
1.22
Total
89
3.60
1.01
According to results in Table 9, no significant difference was found between groups of frequency of studying using internet, in regard to perceived disorientation (F(4, 84) = 0.767, p = 0.550) and ease of use (F(4, 84) = 0.745, p = 0.564).
Interaction with OBCSS and navigation processes
Navigation paths
Learners are provided different learning paths based on their pre-test results in OBCSS. Cognitive skills are queried from CogSkillNet to create a personalized learning experience for each learner. In this section, a learner’s navigation path is described with a working example.
The navigation path is personalized for each user based on their pre-test scores through queries from the ontology. System analyzes the correct and wrong answers and maps each wrong answer with the related cognitive skill in the ontology. Such query yields the related concepts from the cognitive skill ontology and presents the learners as a navigation path along with the related learning objects. The system offers mandatory study and practice phases, first. Learners then can continue with study at your own pace section by listing, recording and selecting the other cognitive skills (Fig. 11).
Fig. 11
A learner’s navigation path with faulty answers in pre-test
×
User comments
While studying in OBCSS, students are encouraged to comment whatever they want on any phase of study. This process is also a part of cognitive apprenticeship model. In this research, 38 unique students added a total of 38 comments during their study. Twenty-four of these comments are related to system, pre-test questions and learning environment. Five of these comments are related to practice, 7 of these are about difficulties and/or relatedness of questions, 3 comments are about lack of hardware and 9 comments are suggestions about system. Fourteen comments consist of meaningless words and/or unrelated words/sentences.
Discussion
In this research, an ontology-based e-learning environment is created to be used in teaching cognitive skills by utilizing cognitive apprenticeship framework. In following sections, results of using OBCSS are discussed. Students having used OBCSS indicated that they did not get disoriented or did not put too much mental effort while using the system. In literature, the cognitive apprenticeship model has been suggested by researchers with a claim that such methodology make the content more effective for students to learn (Seel and Schenk, 2003; Wang and Bonk, 2001; Dickey, 2008). Dickey (2008) also stated that modeling, coaching, scaffolding, and exploration steps are essential to foster knowledge acquisition. In this research it is found that use of cognitive apprenticeship methodologies embedded in the web environment fostered the integration of technology for teaching and learning. According to statistical results, no significant difference was found across gender or computer experience for mental effort. This means that OBCSS can offer a personalized learning environment according to learners’ needs and offer a learning experience without requiring them to invest too much mental effort. In other words, it can be said that even if learners meet a new system and new tasks, most of the students can finish tasks without sustaining too much mental effort.
Most of the students did not perceive disorientation while using OBCSS. Previous research indicated that learners may get disoriented while studying on the web, regardless of users’ experiences in computer use (Altun, 2000; Altun, 2003). Therefore, it is important to make appropriate design choices for learners not to have them disoriented when studying cognitive skills. According to statistical results, no significant difference was found on gender or computer experience for perceived disorientation. This means that OBCSS can offer a personalized learning environment according to learners’ needs and offer a learning experience without making them disoriented.
In this research, system’s ease of use is also measured. According to the findings, most of the students considered system as easy to use. According to statistical results, no significant difference was found on gender or computer experience for system’s ease of use. This means that OBCSS is an easy to use system and can offer a personalized learning environment according to learners’ needs regardless of their gender nor their previous computer experience.
The findings of this study also indicated that learners could be guided through different learning paths based on their pre-test results. These paths are calculated based on their prior knowledge and their learning preferences. It is revealed that complexity of learning path is directly related with complexity of cognitive skills that construct this path.
CogSkillNet, cognitive skill ontology, could be used effectively when creating online courses to support and/or assess learners, and management of learning environments by integrating semantic web technologies and utilizing ontologies in learning environments (Anderson and Whitelock, 2004; Koper, 2004). Also, ontology use makes it possible to have relational searches and query new materials (Gasevic and Hatala, 2006; Brooks and McCalla, 2006; Shafrir and Etkind, 2006; Kaya and Altun, 2009; Yessad et al., 2011) and ontologies provide interoperation of different databases or repositories to search several sources at once (Dicheva and Aroyo, 2006). Ontologies also contributes learning domain by learner profile generation and suggesting proper materials to learners (Devedzic, 2006; Gaeta et al., 2009; Yessad et al., 2011) and generating user/learner models (Denaux et al., 2005; Sosnovsky and Dicheva, 2010; Kaya and Altun, 2011; Poulovassilis et al., 2012). As the primary purpose of this research is to assess design and development of OBCSS and to validate usage in a school environment, student success in learning cognitive skills using OBCSS is not evaluated in paper. Further research could explore this issue as an experimental design research.
To conclude, it is essential for ontology-based web applications to have access to learning object repositories to provide learners personalized and richer learning experiences. Metadata standards and their unique labels are essential part of ontology-learning integration. If not done automatically with certain algorithms, matching of concepts in ontology with learning object will have a serious workload. Further research is needed to validate automatic ontology mapping and matching.
In this research, OBCSS is used in teaching cognitive skills. To use OBCSS as a support tool at courses, concept and curriculum ontologies and integration of these ontologies are required, but first challenges and difficulties shall be addressed and solved for developing educational ontologies (Kaya and Altun, 2018). Domain specific ontologies in the field of education must increase and such ontologies would be possible via automatic ontology generation with contribution of data mining or automatic text extraction domains. By the increase of learning objects, domain and curriculum ontologies, all K-12 curriculum can be included in OBCSS.
Acknowledgements
This study is supported in part by the Scientific and Technological Research Council of Turkey (TUBITAK) with the project number “SOBAG 110 K 602”.
Funding
This study is supported in part by the Scientific and Technological Research Council of Turkey (TUBITAK) with the project “SOBAG 110 K 602”.
Availability of data and materials
Analyzed data is provided with the article.
Authors’ information
Galip Kaya, Ph.D.
Dr. Kaya is a software development team leader at HAVELSAN Inc., and has a BA in Computer Engineering and MA and PhD in Computer Education and Instructional Technologies. He has been working on developing education related ontologies and working as a software engineer since 2004. His main research areas include development, implementation, and evaluation of ontologies for educational settings. He has taken part in various research projects and published research in national and international journals, conferences, and in edited books.
Arif ALTUN, Ed.D.
Arif Altun is a professor of computer education and instructional technologies at the College of Education at Hacettepe University, Ankara Turkey. He has completed his doctorate study at the University of Cincinnati. He has been a visiting scholar at the psychology department at University of Wisconsin-Eau Claire in 2011 and at the Mind, Brain, and Education program at Harvard University during 2014–2015. In the last 10 years, he has been conducting research exploring the cognitive processes in digital learning environments at OntoLAB, to develop user modeling for ontology driven personalized learning environments.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
