Adaptive learning tools are data-driven systems able to meet the individual needs of students by adjusting instruction based on student behaviors and competencies (Bulger, 2016). One such tool, the Assessment and Learning in Knowledge Spaces (ALEKS), is used by millions of students across the United States (ALEKS, 2020b). Although this tool has a significant number of students using it to learn mathematics, there is no clear evidence of its effectiveness. There have been favorable studies with results that have shown its use for improving academic performance (Goodwin, 2017; Karner, 2016; Yilmaz, 2017), but there have also been studies that suggest that its impact is negligible (Mills, 2018; Nwaogu, 2012; Richard, 2019). Add this to the fact that there has been limited research on exactly what strategies teachers use with ALEKS and their perceptions of this tool.
There has been a significant amount of research devoted to finding a relationship between teaching strategies and academic success in mathematics (Anthony & Walshaw, 2009; Caro et al., 2016; NCTM, 2020). Research has shown that effective teaching strategies such as a focus on higher-order thinking skills, classroom management strategies, and feedback techniques have been some of the most influential factors in determining the success of a student in a mathematics classroom (Anthony & Walshaw, 2009; Bartell et al., 2017; Caro et al., 2016; Hattie & Timperley, 2007; NCTM, 2020; Shute, 2008). There also have been studies devoted to the role technology plays in academic success, with evidence suggesting that it improves the teaching of mathematics by allowing for more efficient methods of calculation, graphing, modeling, and data analysis (Sen & Ay, 2017; Wachira & Keengwe, 2011). Literature suggests that technology can provide better feedback to students and enhance engagement (De Witte & Rogge, 2014; Hattie, 1999; Ra et al., 2016; Roschelle et al., 2010). The body of evidence supporting effective teaching strategies in the mathematics classroom and the potential benefits of technology implementation in that space is substantial, but there is still a need for more specific research related to modern technologies like adaptive learning systems.
Advertisement
The research on adaptive learning systems has only spanned the past few decades and has constantly evolved as new technologies emerge with more sophisticated capabilities. Research has suggested that the use of adaptive learning systems can be beneficial when used as an intervention or as a supplement to other teaching strategies (Bochniak, 2014; Burns et al., 2012; Cheung & Slavin, 2013; Longnecker, 2013). However, other research has shown mixed results, with some studies showing improvements in measures like test scores while others have yet to show much of an impact (Campuzano et al., 2009; Hollands & Pan, 2018; Kelly, 2018). One of the areas lacking in the literature regarding adaptive learning tools is how teachers use the tools and an examination of their perceptions of their ease of use and usefulness.
There has been a limited amount of research that has attempted to determine teaching strategies using adaptive learning systems (Azevedo et al., 2005; Benjamin, 2020). Most studies have attempted to measure student success based on improvements in test scores but have lacked to provide any information regarding how the tool is used or teachers’ perceptions of it. Thus, this study aimed to answer the following research questions:
-
What teaching strategies are high school mathematics teachers implementing while using the adaptive learning tool, ALEKS?
-
How do high school mathematics teachers perceive the ease of use of the adaptive learning tool ALEKS in their classrooms/classes?
-
What are high school mathematics teachers’ perceptions of the usefulness of the adaptive learning tool, ALEKS in high school mathematics classes?
Adaptive Learning Tools
Adaptive learning tools are one piece of a digital learning setting in which data and feedback from the learner allow the system to change its functions to meet their needs (Bulger, 2016; Gemin et al., 2015). Cognitive tutoring systems, programs that utilize machine learning, and e-learning platforms are all adaptive learning tools used in digital learning settings. As users interact with adaptive learning tools, the content and workflow adapt to provide learning content that fits the needs of the user (Bulger, 2016; Hsieh et al., 2013; Murray & Pérez, 2015). This content can enhance the learning experience by adapting instruction, curriculum, or the actual learning path so that the student will have a more efficient learning experience (Hsieh et al., 2013; Murray & Pérez, 2015). The instruction can vary between students based on their responses to assessment questions. The adaptive software tool might provide a struggling student with remedial practice while a student displaying mastery of the topic would be provided with more challenging work. Although a teacher could provide a similar path for the student, the idea of adaptive learning tools is that they can assess and automatically provide instruction for the student.
There are several types of adaptive learning systems. A rule-based system functions in an if-then format in which input from the user drives the decision making of the program (Oxman et al., 2014). In this system, a student might receive a hint, repetition of content, or a new explanation if the system so decides. Math Space and IXL are examples of rule-based systems that follow this approach (Hollands & Pan, 2018; IXL, 2020; Math Space, 2020). Another type of adaptive learning technology is an algorithm-based system. This is more sophisticated software that uses functions to analyze the performance of a student over a longer period of time and is able to use the collected data to learn about the student (Oxman et al., 2014). An algorithm-based system essentially collects the historical data of a student’s learning and uses it to make decisions about instruction in the future. Khan Academy and ALEKS are examples of algorithm-based systems in which a user’s past performance contributes to the analysis of a student’s needs (ALEKS, 2020a; Barrett, 2018; Khan Academy, 2020). Programs like ALEKS provide students with opportunities to show work in their problem solving so that the system can assess learning needs by looking at their responses, number of attempts, and time needed (Roberts-Mahoney et al., 2016; Yilmaz, 2017). Many programs feature components of both rule-based and algorithm-based systems.
Advertisement
ALEKS Overview
When students log into ALEKS for the first time they are required to complete an initial assessment. This initial assessment determines what mathematical topics a student knows (and does not know) and helps to build what is known as their ALEKS Pie Chart (ALEKS, 2021). This pie chart displays their mastery level of topics assigned in the class and also topics that they still need to master. Teachers and students can use this information to set learning goals and monitor progress.
Some other tools that teachers can use to monitor students are the progress reports. Teachers can pull up the progress of an individual student or their class as a whole. Teachers can use this information to assess student understanding of mathematical topics. They can also use these reports to follow the progress of a student through the complete set of topics in a class. The reports generated in ALEKS can give teachers information beyond a percentage score or mastery level of a topic. For instance, the reports can provide teachers with the amount of time a student was engaged with ALEKS, the time it took them to complete an assignment and even the rate at which they completed tasks on ALEKS.
When students log into ALEKS they may choose to work on their My Path. The ALEKS My Path is where students can select topics to master to fill out their ALEKS Pie Chart. The My Path provides students with tasks to complete en route to mastering a topic as well as resources for managing their time and progress (ALEKS, 2021). When students are completing a task in ALEKS, there are several ways that the program offers assistance to students. ALEKS offers students explanations and immediate feedback when students input answers to ALEKS questions (ALEKS, 2021). This feedback can inform a student if their answer is right or wrong, provide hints, or offer students the option to click on the program to get a more detailed explanation and additional practice.
Research Studies with ALEKS
Much of what has been written centers around the connections between ALEKS and academic achievement. One of the first studies using ALEKS was conducted in a 14-day summer school session for Algebra I students. Students showed evidence of learning gains on an Accuplacer assessment (Sabo et al., 2013). The 31 students in this study used ALEKS for four hours per day, and those results were measured against those of an intelligent tutoring system. No significant difference was found between the two systems, but all students participating displayed learning gains in their knowledge of arithmetic and algebra. Goodwin (2017) explored using ALEKS in a freshman engineering class, with his study showing improved learning gains from students. This study used ALEKS as a summer intervention for incoming students. The students who spent more time using the program in preparation for the course outperformed those who did not as measured by class grades.
In a Yilmaz (2017) study, middle school students used ALEKS for 45 minutes per day, and the use of the tool was found to help to improve mathematical achievement. His study used a quasi-experimental design with 1110 students from fifth through ninth grade. The experimental group used the tool for 45 minutes per day as a part of their school day while the control group did not use it. Student performance was measured using the NWEA MAP test for mathematics as a pretest and posttest. Students using ALEKS outperformed the non-ALEKS-using group. Karner (2016) compared four years of high school Algebra I students, using the EXPLORE to PLAN assessment to determine whether ALEKS had an effect on growth. The EXPLORE to PLAN assessment is a curriculum-based assessment for mathematics from American College Testing (ACT) designed for high-school aged students (ACT, 2009). Karner used a pre-post quasi-experimental design in his study. The treatment group took an intervention class using ALEKS along with an Algebra I class while the control group took Algebra I with no ALEKS intervention. He found that ALEKS users had higher levels of improvement on the EXPLORE to PLAN assessment than the non-ALEKS users. These four studies have provided some evidence that the tool can be effective in multiple settings.
Some evidence has shown that the use of ALEKS produced improvements in academic performance, but there have also been studies that have not shown a significant difference. There is a need to study the ways that teachers use ALEKS in their classrooms to make a connection between the instructional practices and the use of the tool. There is also a need to collect data on the perceptions of teachers on the effectiveness of ALEKS. Research on the teaching strategies used and the attitudes of teachers towards its use can help to determine the circumstances under which ALEKS can be used effectively in high school mathematics classrooms.
Theoretical Framework
This qualitative study will have The Technology Acceptance Model (TAM) as its theoretical framework. TAM was developed by Davis et al. (1989) and has become one of the models most widely used to predict the use of a technology tool (Davis & Venkatesh, 1996; Sauro, 2019; Yousafzai et al., 2007). TAM is based on the Theory of Reasoned Action (TRA), and numerous studies have used this model (or an altered version of it) for researching the acceptance and usage of technology (King & He, 2006; Venkatesh & Davis, 2000; Yousafzai et al., 2007).
TAM was designed to show how an individual comes to accept and use a technology tool (Mugo et al., 2017). The model has proposed that two factors, perceived usefulness and perceived ease of use, have the most influence on the attitudes and behavioral intentions of an individual when considering the use of a particular technology (Davis et al., 1989). Perceived ease of use has been defined as “the degree to which a person believes that using a particular system would be free of effort” (Davis, 1989, p. 320), and perceived usefulness has been defined as how much an individual believes a tool will increase their ability to perform their job (Davis, 1989). Several research studies have supported the use of TAM for predicting the acceptance and usage of a particular technology (Davis & Venkatesh, 1996; King & He, 2006; Yousafzai et al., 2007).
Following the TAM framework, the purpose of this research study was to collect information about the teaching strategies used with ALEKS in high school mathematics classes and to examine their perceptions of its ease of use and usefulness. TAM suggests that a person’s attitude influences their behaviors, but that these attitudes are determined by the perceived usefulness and ease of use of a technology tool (Davis et al., 1989).
Methodology
This study followed a basic qualitative design in its methodology. Merriam and Tisdell (2016) indicated that basic qualitative design is interested in how people interact, how they construct their worlds, and what meaning they attribute to these experiences. They have suggested that, by using this design, the researcher is interested in how meaning is constructed and not discovered. Basic qualitative designs are inductive and comparative, using coding, categorization of data, and analysis of themes (Kahlke, 2014). Some of the main characteristics of a basic qualitative design are that it uses purposeful sampling, performs data collection via interviews, and provides a rich description of themes and categories (Merriam & Tisdell, 2016).
This research study used interviews with open-ended questions and conversations with high school mathematics teachers who used the ALEKS system. These interviews helped to gain an understanding of the setting, actions, and perceptions of participants who have implemented ALEKS in their classrooms. Interviews helped gain insight into the topic in teachers’ voices and develop a deep understanding of the teachers’ experiences (Fossey et al., 2002; Seidman, 2006). Furthermore, an interview format can encourage participants to share details of their experiences that they might otherwise feel uncomfortable sharing (Fossey et al., 2002; Ryan et al., 2007).
Participants
The participants of this study were five high school mathematics teachers chosen using purposeful sampling procedures. Purposeful sampling is appropriate for this study in particular since the participants need to meet the specific parameters of the research topic (Anney, 2014; Elo et al., 2014; Fossey et al., 2002; Miles et al., 2020). Emails were sent to school administrators across the Chicagoland area seeking volunteers to participate in the study. Ten candidates emerged from the request and were screened to determine whether they met the criteria of the study and to inform them of the requirements, procedures, and purpose of the study. Participants for this study needed to be high school mathematics teachers who had at least two years of teaching experience, who were teaching high school mathematics during the 2020–2021 academic year, who had at least one year of experience using ALEKS, and who planned on using the tool as part of their core curriculum. Of the potential candidates, five teachers were chosen to take part in the research. Not all schools in the Chicagoland area addressed the COVID-19 situation in the same way, therefore participants in this study used ALEKS in different instructional situations (remote, hybrid, in-person). Table 1 provides background information on the five participants.
Table 1
Participants background information
Name* | Grades Taught | Course Taught with ALEKS | Experience Teaching | Experience with ALEKS | Training with ALEKS |
|---|---|---|---|---|---|
Steve | Freshmen | Mathematics II Honors | 18 Years | 4 Years | Some Formal Training |
Tony | Seniors | College Algebra | 23 Years | 2 Years | Self-Taught, Help from Colleagues |
Bruce | Seniors | Transitional Mathematics & Statistics | 35 Years | 4 Years | Help from Local Community College and Colleagues |
Natasha | Sophomores & Seniors | Honors Pre-Calculus | 15 | 2 Years | Help from Local Community College and Colleagues |
Donald | Juniors & Seniors | Algebra II | 12 Years | 3 Years | Self-Taught |
Steve
He used ALEKS this school year in his Freshman Math II Honors class. Although he has used ALEKS for four years, this was the first time using the tool with Freshmen. At his school, the product was typically used for seniors who had struggled with mathematics in the past. He described his training with ALEKS as having occurred through some formal workshops with the teachers in his school, but also stated that he also learned how to use it by spending time working with the tools. Steve decided to use the ALEKS product because he felt that it offered the ability to give reliable assessments.
Tony
Tony had no official training with ALEKS and learned to use its features by working with colleagues at his school. He used ALEKS for the first time during the year before the study in his senior-level math class, and before that, he had some experience with the tool. His school was in a hybrid format for the school year, and he saw his students for 60 minutes per week in person and 85 minutes per week remotely. His rationale for using ALEKS was that he hoped to address a wide range of abilities and measure student growth throughout the school year. He felt that ALEKS had tools that would allow him to do that.
Bruce
He started using ALEKS to meet the credit requirements of the local community college, which was also using the tool. During this study, he was using ALEKS for the Quantitative Literacy and Statistics class. His school had only remote classes, and he would meet with students via Zoom for 165 minutes per week. On Mondays, he would meet with all of the students for 25 minutes, and then he would see the students on alternating days for 60 minutes. He was also able to meet with students if they signed up for office hours, which ran for 45–90 minutes per day. Bruce learned how to use ALEKS by receiving brief tutorials from staff members at the local community college and by receiving help from his colleagues.
Natasha
During the year of this study, she used ALEKS in a remote-only setting where she saw her students every other day for 60 minutes and then for 25 minutes one day per week as well. Natasha has been a mathematics teacher for 15 years. She taught Honors Precalculus with ALEKS, a class that has sophomores, juniors, and seniors in it. Natasha’s rationale for using ALEKS was that it could provide more support and personalization for students. This was especially important for her given that she would only see students for a limited amount of time per week compared to prior years.
Donald
Donald received no training on ALEKS. He piloted the program for his school and taught himself how to use it. He taught in a hybrid setting where he worked with students remotely for 85 minutes per week and in person for one sixty-minute period per week. Donald wanted to use ALEKS for its ability to deliver differentiated content, its ability to provide explanations for students, and for the built-in tools like the graphing calculator. He also wanted to use ALEKS because of the hybrid model his school was operating under; he felt it was a tool that would work well in person and also virtually.
Data Collection
Following Seidman’s (2006) interviewing methodology, a multi-interview approach (three semi-structured interviews per participant) was chosen for data collection to develop a deeper understanding of the context of a situation, and it helps the participant and researcher to develop trust. Since teachers needed to share details of their classroom experiences, establishing trust was important to the interview process. The multi-interview format has been recommended as helping to build relationships between the researcher and the participants (Knox & Burkard, 2009; Qu & Dumay, 2011). Another benefit of performing multiple interviews is that they allow the researcher to analyze data between rounds so that questions can be adjusted and themes can be explored more deeply (Fossey et al., 2002; May, 1991). Although each round of interviews had an outline of questions to be asked, changes were made as the data collection and analysis process began. A semi-structured interview can also be beneficial because participants can seem more conversational, allowing them to feel more comfortable sharing information (Knox & Burkard, 2009; Qu & Dumay, 2011).
Each interview had a clear purpose, and they were six weeks apart from each other. The first interview, within the first month of the school year, was used for collecting background information relevant to the research topic. During this interview, the researcher asked teachers to share their experiences and perceptions of ALEKS before the start of the school year. The interview also asked teachers to share their plans for how they intended to implement ALEKS in their classrooms during the current academic year. Questions examples were: During this school year, what teaching strategies did you plan to use with ALEKS in your mathematics classroom? and What are your perceptions of how easy to use ALEKS was going to be as a tool this school year? Why do you think that?
The second interview served to collect data on how teachers have been using ALEKS in their classrooms. The recommended purpose of this interview was to focus on the details of the experiences of the teachers (Knox & Burkard, 2009; Seidman, 2006). For instance, teachers were asked, how useful has ALEKS been for you to deliver instruction? How useful has ALEKS been to assess student work? What specifically is useful about using ALEKS in a remote setting?
The third and final interview, reflective in nature, allowed participants to consider the meaning of their experiences, share the teaching strategies they used, and comment on the level of ease of use and usefulness of ALEKS and the accompanying strategies. It also served as an opportunity to provide clarity or more detail to the information provided from the previous interviews (Knox & Burkard, 2009). Examples of the questions from the third interview are: in reflecting on how you used ALEKS for instruction, how would you describe its effectiveness? Looking back at your use of ALEKS throughout the school year, describe how you used ALEKS in your teaching this year in terms of assessing students and providing feedback.
Data Analysis
The data analysis began during the data collection stage to improve reflection, structural unity, and the reliability of the study (Elliott, 2018; Miles et al., 2020). Once the first round of interviews was completed, the audio-visual files were loaded into NVivo where they were transcribed and coded. This first round of the coding process was In Vivo coding which focused on what the participants were saying (Miles et al., 2020). After the initial round of coding was completed, the second round of coding was performed to organize the data into patterns and themes, a method known as Pattern Coding (Miles et al., 2020).
During the coding phase of the study, two researchers reviewed the interview transcripts, enhancing the trustworthiness of the coding process. Both performed initial coding on the same interview transcripts independently from one another. They then met to compare, discuss, and determine a coding strategy for subsequent interviews. This process improved the credibility and trustworthiness of the study by having multiple researchers come to the same conclusions (Lincoln & Guba, 1985). Upon completion of each coding session, the data was then organized into tables to compare and contrast codes among participants, deepen understanding of the themes of the interviews, and draw conclusions.
The next step was to use methods to organize the data into tables and matrices. A case-ordered descriptive meta-matrix was used to see the differences between participants and how they have used ALEKS in their classrooms (Miles et al., 2020). This matrix was created by using the queries feature in Nvivo, and it was organized by case and by the codes created from interviews. The crosstab queries feature was used in Nvivo to show the teaching strategies and perceptions of teachers in each of the cases. One crosstab query organized the presence of a code related to the teaching strategies used by teachers. Similar queries were run to create tables for the presence of ease of use and usefulness codes for each teacher to find similarities and differences between the different cases (Elo et al., 2014; O’Cathain et al., 2010). These tables also assisted in determining gaps in the data collection. Doing this after each round of data collection helped to develop further questions for each subsequent round of interviews.
Findings
Research Question #1
To answer the first research question: What teaching strategies are high school mathematics teachers implementing while using the adaptive learning tool ALEKS? themes that came out of the data analysis are (1) Assessment, (2) Data Analysis, (3) Feedback, (4) Practice of Learning Objectives, (5) Individual Pathways, and (6) Future ALEKS Use (see Table 2).
Table 2
Teaching strategies using ALEKS themes
Themes | Descriptions | Quotes |
|---|---|---|
Assessment | Types of assessments teachers used in ALEKS, how they used them, and if they factored it into the students’ grade | “Assessments are the same as assignments but it is more time-bound and they don’t get have access to as many other things like they didn’t maybe only get one chance instead of many chances” (Donald). |
Data Analysis | The ways that teachers use ALEKS tools to analyze student success, progress, and determine future lesson planning | “I can go and see their pie chart you know topics that they’ve mastered and topics that they haven’t and I can suggest couple topics” (Tony). |
Feedback | How teachers use the feedback provided through ALEKS and the ways in which they use ALEKS to give feedback to students | “My path is a good way to provide feedback on their goal-setting but then also like providing feedback to them like content-specific feedback” (Bruce). |
Practice of Learning Objectives | The expectations of what students are required to complete on ALEKS and how teachers communicate learning goals with students on ALEKS | “Basically, I was going to have weekly objectives goals for them to work on” (Steve). |
Individual Pathways | The use of the ALEKS My Path by teachers or descriptions of how teachers provide personalization for students. | “I had to basically use it as, you know, like an understanding vehicle for students to continue on their path to work with the weaknesses” (Bruce). |
Future ALEKS Use | How teachers plan to use ALEKS in the future | “I really want to give those students opportunities to just be able to do a little more working and practice on their own” (Natasha). |
Theme 1: Assessment
The most common way that teachers used ALEKS was as an assessment tool. Teachers interviewed stated that they used the assessments built into ALEKS. In interview two, Bruce explained that he used ALEKS for “a 30-question test for each chapter.” Although teachers used some form of the assessment tool in ALEKS, they supplemented their assessments with additional tasks. Referring to how he supplemented tasks with ALEKS, Donald stated in the second interview that he used “a video explanation of their topic with a rubric that I use that kind of gives them feedback on how well they are explaining the topic that they’re learning in ALEKS.” Tony also required students to provide additional items with their assessments in order to check their work:
Teachers in this study used ALEKS to assess students on the objectives that were taught during either in-person or remote class time, and even required students to supplement their assessments with additional explanations and/or examples of their work.I have the student take a picture of the work I required them to do. I would then go into the ALEKS and check to make sure that their time or the topics that they said they completed were actually completed, and I would give them a score.
Theme 2: Data Analysis
Participants mentioned that they used the data analysis tools provided by ALEKS to follow student progress toward the learning objectives. They used it as a way to assess students’ growth and progress as students worked through the individualized pathway that ALEKS provides. Tony used ALEKS mostly for the individualized pathway and in the second interview described using the ALEKS data analysis tools in the following way:
Participants also used ALEKS to identify student struggles and assist them with lesson planning. Teachers did this at the macro level by looking at class-wide struggles. For example, Steve, who used ALEKS for students to practice objectives that were taught in class, stated in the second interview: “I’m using the detailed question report prior to class to have one or two questions that I go over at the start of class.” Natasha commented in interview three that she also used the data analysis tools in ALEKS to identify class wide successes or struggles:This past week we just gave our second placement test, and what I want to see from the kids is some growth with one score to the next. I can go and see their pie chart. You know topics that they've mastered and topics that they haven’t, and I can suggest a couple of topics. I can actually go in and see the number of attempts, where kids are struggling, the ones they're skipping, and the ones they’re missing for one reason or another.
Participants also used ALEKS to identify the struggles or progress of individual students by checking their pie charts on ALEKS or the “time on” graph. Donald mentioned checking individual progress during the third interview, stating “I definitely am always checking that data. If they’re not turning the assignment in or not getting good scores, I follow up with the students individually.”I also monitor their formative results to see if they are ready, [or] if we need to maybe review anything in class together. I am using a lot of collected data [on] which of the objectives they are working on and how fast they move through the course of those problems.
Theme 3: Feedback
Participants in this study stated that they relied on the feedback provided through ALEKS when students worked independently. Teachers gave feedback to students based on information they collected from the ALEKS data analysis tools. Some of the feedback they gave was based on specific math topics. For example, Donald stated: “When I see that students are not doing well on a particular assignment, that’s when I maybe need to make out an additional lesson or make an additional video to give them feedback.” Additionally, teachers indicated they gave feedback to students about their individualized progress by having discussions with them or engaging in goal setting. Bruce, for instance, reflecting on his use of ALEKS to have conversations with students, during the third interview stated:
Teachers mentioned using ALEKS for instruction and for giving feedback during class time. Many of the teachers used ALEKS during class time to do “bell ringers” and “warm-ups”, described by teachers in this study as a set of problems for students to complete at the beginning of class. These exercises were typically related to the lessons taught in class, created to address student misconceptions, or given to provide feedback to students. Participants mentioned bringing up ALEKS on their computer screens during remote-learning sessions to give feedback about how to use the tool or to correct specific errors being made by students. Teachers actively incorporated ALEKS into class time, providing real-time feedback to students as they tackled problems. Donald, discussing his use of ALEKS during the second interview stated: “I can give them individual support and help during the instruction. They get real-time feedback in either audio, text, or video. Whatever they need.”The ALEKS My Path is a good way to provide feedback on their goal-setting but then also like providing content-specific feedback. We also get a lot of good reflection -good math reflection - that we can talk about, and they can also be proactive, which is another cool element.
Theme 4: Practice of Learning Objectives
Teachers, in this study, assigned a weekly goal for in-class objective completion based on what was taught during the week. These goals were related to units being taught in class. Some participants also had weekly goals for the students in the ALEKS My Path. It was expected that students would complete a certain percentage of work, spend a certain amount of time, or cover a certain number of topics per week or per semester. Others assigned the My Path but without specific goals for student completion. In the first interview, Steve outlined how he planned to use ALEKS during the school year:
The teachers all had different expectations for students in terms of practice. Tony stated that he wanted students to “use it a minimum of an hour a week” while Donald set objective-based goals in which the students complete “24 assignments that range from six to twelve questions each.” Several of the teachers combined using the individual pathway for students to work on during asynchronous learning time or homework with giving assignments related to the course itself.Basically, I was going to have weekly objective goals for them to work on, consistently ten topics a week. I use twelve basic problems, and then I give them three attempts per question and unlimited attempts at the assignment. My thoughts [were] that: for those who need additional work, ALEKS provides it because it will regenerate a new question.
Theme 5: Individual Pathway
The individualized pathway that ALEKS provides allowed for students to progress at their own pace while the teachers were able to monitor their progress. ALEKS also offers an initial assessment that collects information from students and uses its algorithm to create an individualized pathway for students to learn through the program. All of the teachers in this study mentioned using the initial assessment tool in ALEKS to get a “baseline” of what their students knew and did not know. In reference to using an initial assessment in interview two, Tony stated:
After the initial assessment, teachers used the ALEKS My Path in different ways. Teachers, Tony and Donald in particular, used it as a significant part of their class. Tony assigned students a set number of minutes for students to spend on it per week while Donald required students to complete a set number of objectives each week. Natasha and Steve, on the other hand, viewed the individualized pathway in ALEKS as optional. All of the teachers in the study offered opportunities for students to use the individualized learning but to varying degrees.My plan was to use ALEKS first of all to get a baseline of the kids’ skills. The first thing we did was: we took our first placement assessment the first couple of days in class that gave me an idea of where all my 17 kids were at and looked at their work, and it helped me plan.
Theme 6: Future ALEKS Use
Participants talked about using ALEKS in the future. Teachers mentioned ways they would add or change their use of the tool. Steve and Tony mentioned wanting to make better use of ALEKS for entrance and exit slips. Participants wanted to use the My Path portion of ALEKS more often and more effectively. Specifically referencing using the individualized pathway more, Natasha said in the third interview: “I really want to give those students opportunities to just be able to do a little bit more working and practice on their own.” A few teachers brought up doing more personalization using ALEKS and explaining the underlying purpose of using tools to help students use ALEKS more effectively. Donald and Bruce hoped to have students collaborate and communicate more about their mathematical understanding. For example, in the third interview, Bruce commented on his desire to have students work together: “Get kids working on ALEKS, working together in the classroom, and see kids sitting at a round table.” Overall, the teachers in the study had a desire to continue to use ALEKS and to improve the way they use it. When asked about the potential for students to be back in the classroom and how that might impact their uses of ALEKS, classroom teachers hoped to have the students interact more with each other, to use the program more for entrance and exit slips, and to give more in-person feedback.
Research Question #2
To answer the second research question: How do high school mathematics teachers perceive the ease of use of the adaptive learning tool ALEKS in their classrooms/classes?, data from the interviews underwent multiple rounds of coding and were then organized into a table that compared how teachers perceived ALEKS’s ease of use. The two themes identified from the data analysis were Easy to Use and Minor Struggles/Limitations. In this section these two themes are discussed in terms of how they contribute to answering this research question. These themes are based on Table 3.
Table 3
Ease of use of ALEKS themes
Themes | Descriptions | Quotes |
|---|---|---|
Easy to Use | Descriptions of if and how ALEKS was easy for them to use as well as examples of why it is easy for them to use | “I thought it would be user-friendly and after using it I found it to be even better than I expected” (Steve). |
Minor Struggles / Limitations | Descriptions of some of the challenges teachers had using ALEKS | “Just like troubles with some of the toolbars that show up in an iPad” (Bruce). “I would not say it is very intuitive” (Donald). |
Theme 1: Easy to Use
The overall opinion of the participants in the study was that they found ALEKS to be easy to use and navigate. Steve, during the first interview, described his experience with ALEKS: “I thought it would be user-friendly and after using it I found it to be even better than I expected.” More specifically, since teachers often used ALEKS for creating assignments and assessments, several of the participants discussed how easy it was to find objectives and assessment items. Natasha brought this up in the second round of interviews, saying, “Their objectives are very little and are detailed, so every problem that I want to assign has a description which is long enough to know what is in the problem.”
Theme 2: Minor Struggles/Limitations
Although the sentiment of the teachers was that ALEKS was easy to use, they did bring up some challenges they encountered using the tool. Participants shared insights into their processes of adapting to the tool; for example, Donald did not perceive the tool as being “very intuitive”, and Bruce indicated: “I had some trouble navigating and kind of understanding that at the beginning.” Donald and Tony brought up wanting to have more control over the assignments that students could have access to in their individualized pathway. Bruce noted the limitations of the technology with ALEKS such as the iPad would sometimes freeze or crash, and there was some frustration with ALEKS marking answers wrong because students did not write them in the correct form.
Participants also mentioned that like with other tools, ALEKS required time to get familiar with it to be able to manage it easily. For example, Tony indicated “It’s a matter of time for me and like how much time I have to get in there.”
In summary, the findings connected to answering the question of what the teachers’ perceptions were related to the ease of use of ALEKS revealed that teachers in the study found the tool to be easy to use, but, as with other tools, there are some challenges and areas that they feel could improve to make it better.
Research Question #3
To answer the question: What are high school mathematics teachers’ perceptions of the usefulness of the adaptive learning tool, ALEKS in high school mathematics classes? six themes came out of the data analysis: Useful Instructional Tools, Assessment Tools, Personalization, Negative Perceptions, Question Generation, and Remote Learning Usefulness. Themes descriptions and quote examples are presented in Table 4.
Table 4
Usefulness of ALEKS themes
Themes | Descriptions | Quotes |
|---|---|---|
Useful Instructional Tools | The ways in which teachers found specific tools in ALEKS to be useful | “Accessibility is very useful and powerful as a teacher” (Donald). |
Assessment Tools | Description of what teachers found useful about the assessment tools in ALEKS | “I like the possibility for students to access any helps with ALEKS throughout the assessment” (Natasha). |
Personalization | Descriptions of how teachers used ALEKS to personalize instruction and rationale for why it was useful for them | “It’s really useful so I started creating assignments just tailored for the specific students” (Natasha) |
Question Generation | How the ability for ALEKS to generate multiple practice problems for students is useful | “because of the way it generates multiple problems, examples, and allows for students to kind of fail into success in a more streamlined manner compared to me creating multiple worksheets.” (Steve) |
Negative Perceptions | Feedback from teachers on the ways in which ALEKS was not useful | “they got the questions right but it was in the wrong form so they didn’t necessarily read that wonderful little italics or small print” (Bruce). |
Remote Learning Usefulness | Due to COVID-19, teachers needed to teach remotely so these codes are what make ALEKS useful in remote teaching | “ALEKS has been very helpful especially in this time of remote learning to have a resource that can to do as much as ALEKS does” (Donald). |
Theme 1: Useful Instructional Tools
Teachers provided a variety of explanations about the usefulness of the ALEKS tools. Most of the teachers described the explanations and tutorials as being useful. For example, in the second interview, Donald brought up the usefulness of the accessibility of explanations in ALEKS:
Steve specifically cited the quality of questions asked in the ALEKS system as useful in his teaching, stating in the second interview that ALEKS “does give you some of that reasoning and conceptual understanding.” Feedback was commonly cited as a useful instructional tool within ALEKS. The ability of the program to provide immediate feedback to the students outside of class time was cited especially. Natasha, in her second interview, mentioned how useful was the feedback:[My students] have a button they can just push that takes them right to an explanation of the problem they’re working on right then. That’s accessible. So, we can make that learning accessible to the students. I think that’s very powerful, very useful for teaching.
Several teachers brought up the ability of ALEKS to help with efficiency, including planning and providing practice for students. Steve, in the second interview, referring to how helpful ALEKS was for providing extra practice stated that “in one respect it allows me to minimize time on remediation and maximize time on new instruction.”Immediate response, immediate feedback on how they’re doing with the objectives outside of class. When we are remote, we don’t have that much time together. Twice a week, it’s not a lot so they can practice. They are expected to practice on their own and ALEKS is great for providing feedback to the students.
Theme 2: Assessment Tools
Several teachers cited the ability to give secure assessments, the ability to generate custom assessments from learning objectives, and the ability to manipulate the time constraints as useful aspects of the ALEKS system. Specifically, Steve, Bruce, Donald, and Natasha pointed out the usefulness of the tools ALEKS provided for assessments. For example, Natasha, in her interviews, discussed that ALEKS allowed her to change both the time limits and the availability of access “helps”, and allowed for students to be able to be retested on specific questions. Donald, in the first interview, also stated that he believed copying and cheating were minimized during assessment because it “is really helpful that students can all have different questions. I feel that that minimizes copying and minimizes cheating and really gets what they seem to know.”
Theme 3: Personalization
Teachers brought up, on several occasions, that they found useful many of the features offered in ALEKS for personalized learning. These observations are connected to the systems capabilities to adapt based on student’s responses as well as allowing teachers to modify it to fit their needs. Natasha, Tony, and Donald all mentioned that ALEKS was useful because it could provide for individual gaps that students had in their learning. In particular, Natasha found ALEKS to be “really useful so I started creating assignments just tailored for the specific students”. Donald, in the third interview, discussed how the program was useful because it met students where they were. When talking about his students having to explain topics in ALEKS he said:
If they have 70 topics at the beginning, their questions might be a little harder, but they should be ready to do that, and so I feel like that helps to build confidence. I don’t care how easy or hard the topic is, you should be able to explain wherever you are. I feel like that’s been helpful because it’s not just a one-size-fits-all class.
Theme 4: Question Generation
Related to the previous theme, participants value the question generation feature in ALEKS. Not all teachers mentioned the importance of question generation, but several teachers discussed how useful it was for students to receive multiple attempts on questions and assessments. Steve, in the second interview, stated that having an unlimited number of problems for students was helpful “because of the way it generates multiple problems, examples, and allows for students to kind of fail into success in a more streamlined manner compared to me creating multiple worksheets.” Participants in this study indicated that two of the benefits of ALEKS were that it allowed them to address individual gaps in student understanding and that the question generation allowed students to practice as much or as little as needed. Bruce, during the second interview, expressed “I really like that regeneration”.
Theme 5: Negative Perceptions
Although the overall perception from the group of teachers was positive, some of the teachers had some negative perceptions as well. Negative perceptions varied across teachers, but some discussed not getting enough feedback about the specifics of what the students might be struggling with. Steve in particular brought up in the second and third interviews that he felt the feedback he received in ALEKS was not specific enough:
Other issues came in the form of how the feedback is delivered from ALEKS to students when they get a problem wrong. Bruce and Donald mentioned that they felt the written explanations were challenging for some students to understand. Bruce mentioned that the explanations in ALEKS were “math for math teachers and not math learners.” This sentiment was shared by Donald when asked about the usefulness of ALEKS in the third interview: “I had to do a lot of explanations and instructions. ALEKS doesn’t embed that. They’ve got their explanations, but they kind of are assuming that you’re already getting it.” Some other negative perceptions expressed by the participants in terms of the usefulness of ALEKS addressed minor challenges and limitations. Bruce expressed that sometimes ALEKS would not accept different forms for a response and would mark it wrong for students. Donald explained that even though ALEKS was a useful tool, teachers “can’t just say work on your learning path. That’s not going to lead to tremendous growth.”ALEKS from a teacher’s standpoint lacks a little bit because I can’t see those things [students’ detailed reports]. It’s not specific enough that I can go and diagnose the specific issue. I need to see really just visually not that they were just right or wrong, but I need to be able to see where the mistakes are.
Theme 6: Remote Learning Usefulness
All of the teachers found ALEKS to be especially useful in a remote learning environment for various reasons: (1) ability to access data on student task completion, (2) secure and customizable assessments, (3) the instructional tools, and (4) feedback. Due to the limited amount of time that teachers had with students during the course of this study (because of the effects of COVID-19), they felt ALEKS provided them with several resources and the ability to help students to learn at any time. For instance, in the second interview, Steve mentioned that the data provided by ALEKS helped him to see that students were independently practicing the course content, stating that it “makes it so much easier because now it is not a mystery what happened at home.” The ability of the program to provide secure assessments and the instruction provided by the tool are the main reasons Natasha found ALEKS to be useful. Describing the assessment features in the third interview, she stated:
Because of remote learning, it’s a good tool for formative learning because it gives students feedback right away. Since they don’t necessarily have a teacher with them, it allows them to practice anytime. It’s especially useful in the setting we are in. When students can test anytime it’s a great opportunity for students to test on their own time so we don’t have to use our already short days that we have together.
Discussion and Conclusions
There has been an abundance of research related to the adaptive learning tool ALEKS that has attempted to measure the effectiveness of the tools on academic performance as measured through test scores (Fang et al., 2019; Goodwin, 2017; Karner, 2016; Mills, 2018; Nwaogu, 2012; Richard, 2019; Sabo et al., 2013; Yilmaz, 2017). However, there have been a limited number of research studies that have explored teaching strategies using ALEKS (Benjamin, 2020; Padilla-Oviedo et al., 2016). Most of the information on how ALEKS is used by teachers has come from the actual company itself through their own publications (2020a; ALEKS, 2020c). This study adds to the body of independent research related to ALEKS and helps to fill gaps in the research by providing the voice of the teachers who are using the tool in their classrooms. This is significant because it not only fills gaps in the research related to ALEKS and adaptive learning tools but could also lead to more qualitative studies that attempt to explore how technology is used by teachers as well as their perceptions of the tool.
The findings of this study indicate that teachers used several teaching strategies related to assessment, data analysis, feedback, and instruction techniques. They used ALEKS to provide assessments for students, the results of which teachers used to analyze students’ understanding. They were then able to provide feedback to students on their misunderstandings at both an individual and class level. As concluded in previous research, feedback to students is considered to be one of the most effective strategies for student learning (Barry, 2008; Hattie, 1999; Havnes et al., 2012; Marzano et al., 2001; Shute, 2008). Additionally, computer-based feedback has shown evidence of being effective for student learning (Hattie, 1999; Roschelle et al., 2010). Teachers made use of the ALEKS My Path to allow students to work at their own pace. In this scenario, teachers acted as facilitators who could provide for individual needs, communicate with students about their progress, set goals, and give feedback. All of these have been supported by research related to effective teaching strategies (Arends et al., 2017; Bartell et al., 2017; Marzano et al., 2001).
The technology acceptance model (TAM) has suggested that the ease of use of a technology tool has an impact on its acceptance and on whether that tool will be adopted by a user or not (Davis, 1989). Several factors can influence an individual’s perception of ease of use when it comes to technology, specifically the design features of the tool and the level of support one needs to use it (Davis, 1989; Mugo et al., 2017; Venkatesh & Davis, 1996). Participants in this study consistently stated that ALEKS was easy to use and supported this perception by providing examples of design features in the program and sharing their experience/training with the tool. Difficulty using a technology tool can harm adoption by a user (Kopcha, 2012). Thus, the findings of this study were consistent with prior research related to TAM in which teachers found ALEKS to be easy to use and planned on using the tool in the future (Sauro, 2019; Yousafzai et al., 2007).
One of the factors that can influence an individual’s perception of usefulness when it comes to technology is its design features (Davis, 1989; Mugo et al., 2017; Venkatesh & Davis, 1996). Teachers in this study perceived that the tools in ALEKS were useful to them in many ways. They were able to share details about how ALEKS was useful to them. They mentioned its ability to provide instructional tools, its ability to provide reliable and customizable assessments, and its ability to allow for personalization for students. These examples of usefulness were also supported by the research related to technology use in mathematics classrooms (Bray & Tangney, 2017; Kynigos, 2019), differentiation (Bulger, 2016; Peshek, 2012; Ysseldyke & Tardrew, 2007), and the capabilities of adaptive learning tools (Hsieh et al., 2013; Murray & Pérez, 2015). Although some teachers shared some negative perceptions about the usefulness of ALEKS, they stated their intention to use ALEKS in the future. This seems to reflect that the positive perceptions outweighed the negative ones. This is consistent with prior studies related to TAM that have suggested that positive perceptions of the usefulness of a technology tool influence the behavioral intention of a person to use the tool in the future (Sauro, 2019; Yousafzai et al., 2007).
Limitations and Future Research
Although this study can contribute to the literature on adaptive learning and can provide examples of how ALEKS is used in high school mathematics classrooms, it has limitations and assumptions to be considered. This study took place in the Chicagoland area with five high school mathematics teachers using ALEKS for one academic school year. Given that this study interviewed a limited number of participants, future researchers could attempt to duplicate this study with a larger or more diverse sample.
This study also only explored the use of ALEKS over one school year that included the situations associated with the COVID-19 pandemic. Different contexts or longer use of the tool could lead to different experiences and perceptions of its ease of use and usefulness. Since ALEKS was the only adaptive learning tool that this study focused on, one cannot apply these generalizations to all adaptive learning tools as well. This was a study of one adaptive learning tool and its features, and its use may be different than others available to teachers.
Acknowledgements
This research is adapted from the first author’s doctoral dissertation, and is edited for the purposes of this publication. The first author wishes to thank all the participants that made this study possible.
Declarations
Research Involving Human Participants
All procedures involving participants for this study were approved by the IRB at Boise State University (101-SB20–184).
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Conflict of Interest
The authors have no conflicts of interest to declare relevant to the content of this article.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
