Teaching and Assessing Thinking Skills and Applying Educational Technologies in Higher Education

Since the turn of the millennium, economic organizations, educational councils, and accreditation bodies have continually advocated for emphasizing and integrating thinking skills such as critical thinking, creative thinking, systems thinking, and others, in higher education pedagogy in general and in science and engineering higher education in particular (ABET - Accreditation Board for Engineering and Technology, 2022; Ananiadou & Claro, 2009; NRC - National Research Council, 2013; World Economic Forum, 2020). Unfortunately, while many successful efforts have been made toward the integration of thinking skills into higher education pedagogy (Dori et al., 2023; Lavi et al., 2023), effective, measurable, and sustainable teaching of thinking skills is still a faraway goal for many higher education institutions (Barak & Shahab, 2023; Van Damme & Zahner, 2022).

Integrating thinking skills into higher education pedagogy requires suitable models, methods, and tools for both instruction and assessment (Davis et al., 2023; Kaur & Chahal, 2023). Some of these tools belong in the category of educational technology and their use has proliferated globally during the COVID-19 pandemic (Hodges et al., 2020; Kerres et al., 2022). The pandemic has caused the largest disruption to higher education in world history. The majority of higher education institutions worldwide were either completely or mostly shut down, and most of the world’s student population was adversely affected (Pokhrel et al., 2021).

Beside these challenges, the emergency remote teaching experience has also provided pedagogical and learning opportunities. The most notable change was the flexible learning possibilities and opportunities for instructors to explore different learning modalities, such as hybrid or blended, and various technologies for remote teaching (Barak & Shahab, 2023; Mefferd & Bernacki, 2023). Students were exposed to opportunities for virtual mobility and collaborative online learning in lieu of onsite, in-person collaboration. This may explain why institutions reported that community engagement has increased during this emergency remote teaching period (Marinoni et al., 2020).

In summary, while the pandemic had detrimental effect on teaching and learning in higher education, it has also been a catalyst for pivoting to more sophisticated, flexible, and technologically advanced forms of teaching, at least for those instructors who were willing to adapt their pedagogy.

With this ‘new normal’ in mind, fostering students’ thinking skills depends more than ever on educational technologies combined with appropriate pedagogy (Thompson et al., 2023). A novel opportunity for deeper and more meaningful use of these technologies has opened for enabling effective pedagogies and assessment methods, both remotely and in-class. For example, a smart classroom makes use of various interactive teaching platforms and information and communication technologies (Dai et al., 2023).

This special issue is dedicated to teaching, learning, and assessment, with or without the use of educational technologies. The 15 articles in this issue provide a snapshot of research on thinking skills, educational technologies, and their integration in higher education, specifically in science, technology, engineering, and mathematics (STEM) domains.

The articles in this special issue share four common themes: online or virtual courses and modules, science and engineering education, active learning methods, and critical thinking.

Some of the papers involve more than one theme, and therefore, they are mentioned twice and some of them are discussed in two themes. For example, Gadot and Tsybulsky (2023) are discussed in both the science and engineering education theme and in the critical thinking theme.

Online or virtual courses and modules are the most represented application of educational technology, with eight papers (Dutta et al., 2023; Gadot & Tsybulsky, 2023; Mefferd & Bernacki, 2023; Swain et al., 2023; Thompson et al., 2023; Usher & Hershkovitz, 2023; Wengrowicz et al., 2023; Yoel Rocker et al., 2023). This is not surprising given recent worldwide experiences of emergency teaching during the COVID-19 global pandemic (Marinoni et al., 2020).

Mefferd and Bernacki (2023) examined how providing opportunities in an online first unit of a science course for pre-reading and self-testing affected undergraduate biology students. The authors collected digital traces of students’ use of resources hosted in the course’s learning management system and their first unit exam grades. The authors hypothesized that students who were self-regulated, used advance organizers before lecture, and utilized quizzes to self-assess themselves would gain better achievement scores. Indeed, participants who pre-read in advance of the class and those who self-tested themselves via quizzes performed better on the unit exam than other students.

Usher and Hershkovitz (2023) explored the perspectives of massive open online course (MOOC) instructors on the process of data-driven instruction and their perceived barriers to data use for decision-making. MOOC instructors showed great interest mostly in data about social interactions between remote learners and problems with the MOOC educational resources. The main barriers to using educational data for decision-making were lack of customized data, real-time access, data literacy, and institutional support. These results highlight the need to provide MOOC instructors with professional development opportunities for the proper use of educational data for skilled decision-making.

Thompson and colleagues (2023) created MathBench biology modules as a freely available online resource to supplement existing course content to better prepare students for upper-level biology courses and postgraduate biology careers’ quantitative skill requirements. They tested the modules and integrated them into the curriculum of an introductory principles of biology course for natural science majors at a community college in the USA. The study also included a control group of students who did not use these modules. MathBench was effective in helping these students improve their quantitative skills and their ability to apply them to biological problems.

Wengrowicz and colleagues (2023) introduced MORTIF—Modeling with Real-Time Informative Feedback—as a new learning-by-doing feature in a massive open online course (MOOC) on model-based systems engineering. MORTIF enabled the learner to model, receive feedback, and resubmit an improved solution. The authors investigated the usability of MORTIF, its perceived contribution, engineering students’ favorite question type, and learning style. Applying the mixed-methods approach, using MOOC server data and online questionnaires, they found that the model resubmitting option increased as the course progressed and that MORTIF was effective in promoting meaningful learning.

Yoel Rocker and colleagues (2023) created an online course that develops the interpersonal communication skills of graduate students from a host of STEM disciplines. The various assessment tools deployed in this research constitutes a methodological contribution, as they can be applied to other undergraduate and graduate courses.

Science and engineering education is represented, with or without technology or mathematics education, in seven articles (Barak & Shahab, 2023; Dai et al., 2023; Gadot & Tsybulsky, 2023; Kaur & Chahal, 2023; Wang & Li, 2023; Usher & Hershkovitz, 2023; Yoel Rocker et al., 2023). The number of articles in this theme is larger than the number of articles focused on a particular science or engineering discipline, or on engineering in general.

Barak and Shahab (2023) discussed globalization trends, such as student mobility, where international students are part of the student body. They highlighted the need for a proactive approach and suggested a framework that involves nine technology-based activities to foster those skills.

Dai and colleagues (2023) investigated how the learning environment affects the interaction between instructors and first-year students in project-based learning (PBL) courses. Analyzing videos from traditional and smart classrooms, as well as surveys regarding students’ satisfaction from participation in PBL, the researchers found stronger student interaction with their peers and with instructors in smart classroom versus traditional classroom. More time was afforded to student discourse in the smart classroom than in the traditional one, and students were more inclined to participate in PBL in a smart classroom.

Kaur and Chahal (2023) investigated how personality and learning motivation can influence the level of computational thinking in undergraduate students of computer science and engineering. Based on online questionnaires distributed to students, the authors reported positive correlations between students’ computational thinking skills and character traits taken from the Big Five personality model of openness, extraversion, and conscientiousness, as well as learning motivation, especially self-efficacy. They suggested considering personality traits and learning motivation factors in the context of developing the computational thinking skill.

Wang and Li (2023) used Virtual Starmaze, a cognitive neuroscience tool for testing people’s selection of spatial strategies. They investigated third-year STEM students’ spatial strategy and its role in academic performance. They identified four types of behavior patterns: egocentric, allocentric, mixed behavior patterns, and shift behavior patterns. Students with high academic performance exhibited information-seeking behaviors during navigation in a maze more than students with low academic achievement. Finally, science or mathematics students exhibited different spatial strategies than technology or engineering students.

Usher and Hershkovitz (2023) provided insights into the challenges and opportunities of using educational data for decision-making among instructors who teach MOOCs in various science and engineering domains. Their findings can serve science and engineering educators who plan to use educational data to support their students and improve their instruction.

Active learning methods are represented in six papers (Dai et al., 2023; Dori et al., 2023; Dutta et al., 2023; Lavi & Marti, 2023; Swain et al., 2023; Wengrowicz et al., 2023; Yoel Rocker et al., 2023). These methods include project-, problem-, case-, and inquiry-based learning, with students working individually and in groups while applying higher order thinking skills (Lavi et al., 2021).

Dori and colleagues (2023) investigated the effect of hybrid courses and reading scientific articles on the biomedical engineering students’ scientific literacy as reflected by their question posing skill. The authors applied research tools that included pre- and post-questionnaires, students’ questions posted on the forum discourse, and research questions raised by the students on their scientific posters. Students improved their scientific literacy skills by studying the hybrid courses and participating in online discussions. The theoretical contribution of this study is the findings that forum discussions foster science and engineering students’ question posing skill.

Dutta and colleagues (2023) developed a learning system using augmented reality (AR) technology to help students learn about Karnaugh maps (K-maps), a technique in digital electronics used for design and equation-solving. Their research objective to determine the impact of AR applications on the critical thinking skills, learning motivation, and knowledge of students in a flipped learning environment. They deployed self-reporting questionnaires among second-year students in electronics engineering—two equally sized groups, experimental, and control. The authors reported that the use of AR technology in flipped learning mode improved students’ critical thinking skills through enhanced engagement and better visualization of discrete concepts of digital electronics.

Lavi and Marti (2023) investigated undergraduate engineering students’ creative thinking and critical thinking via case-based learning. The authors provided a theoretical background on problem-solving in engineering, creative thinking, and critical thinking, reviewed approaches to case-based learning in undergraduate engineering education, and outlined guidelines for instructional design and implementation with practical examples. They also discussed challenges in the implementation of their framework along with theoretical and practical contributions for fostering undergraduate engineering students’ creative and critical thinking.

Swain and colleagues (2023) investigated the relationships between students’ peer-to-peer interactions, group formation, choice of research topics for end-of-semester projects, and performance in an asynchronous online undergraduate ecology course. Over the semester, the average number of people that one individual interacted with first increased and then decreased, while the average total number of interactions mainly increased and then stabilized. An increase in interactions between two successive discussion networks was associated with better performance in related exams, but a decrease in such interactions did not necessarily yield the opposite outcome. The authors provide data analysis techniques that can be applied for further explorations of peer-to-peer instruction.

The hypothesis of Wengrowicz and colleagues (2023) that active learning with real-time informative feedback is a learning mode that students benefit from was validated using MORTIF (modeling with real-time informative feedback). The benefits included active learning, meaningful real-time feedback, and resubmission of an improved model.

Yoel Rocker and colleagues (2023) examined the effect of a graduate online course, taught in a science and technology research university, on graduate students’ interpersonal communication skills. Quantitative and qualitative research tools included students’ self-presentations, questionnaires, peer assessments, and reflections. The graduate students improved their interpersonal communication skills, including written and oral communication, peer evaluation and feedback, and self-reflection. They benefited from exposure to various knowledge and research fields and suggested rendering the course mandatory to all the graduate students in the university.

Critical thinking is the highest represented thinking skill in the special issue, with five papers including it within the research scope (Barak & Shahab, 2023; Davis et al., 2023; Dutta et al., 2023; Gadot & Tsybulsky, 2023; Lavi & Marti, 2023).

Barak and Shahab (2023) examined the way critical thinking (CT) is conceptualized and experienced by instructors and local and international students from China and accordingly design a culturally inclusive theoretical framework for CT cultivation. The study applied an integrated dual-analytic approach, where data was collected via a survey and semi-structured interviews. The findings show that the international students were more familiar with CT in theory and practice than the local students. The study applied a technology-enhanced instructional framework that integrates individual learning with collaborative and culturally inclusive assignments.

Davis and colleagues (2023) investigated whether students’ systems thinking performance in scenario-based assessments related to their self-reported assessments of systems thinking. They measured engineering students’ self-reported systems thinking and related competencies, including critical thinking, interdisciplinary skills, and contextual competence, as well as their performance on two newly developed scenario-based assessments of systems thinking. Self-assessment measures correlated with each other, while performance did not correlate with self-assessment. The authors question the extensive use of self-reporting to assess systems thinking in educational research and evaluation, as well as for the development of alternative forms of systems thinking assessment.

Gadot and Tsybulsky (2023) offered a pedagogical approach, digital curation, which effectively utilizes the fundamental components of critical thinking, including analysis, evaluation, and inference. The authors introduced DC2CT, a theoretical model that illustrates how critical thinking is manifested throughout the digital curation process. This model highlights activities that stimulate evaluation, analysis, and inference, as well as those that encourage personal and social engagement. The article points to the need for further research in this domain to investigate the broader impact of digital curation on various cognitive skills.

In a recent OECD report, Van Damme and Zahner (2022) called for developing and integrating various assessment tools both nationally and globally to foster the development of thinking skills in higher education with focus on critical thinking. They, along with others (ABET, 2022; Avargil et al., 2020; Lavi et al., 2021), point to the necessity of university graduates to develop twenty-first-century skills to meet the current employment market requirements. This special issue focuses on developing and assessing thinking skills in science and education disciplines whether they include educational technology or not.

Interdisciplinary collaboration among educators, researchers, and technology developers will be essential in advancing the field of educational technology and its impact on thinking skills development. By pooling expertise from various domains, educators can leverage the power of technology to create meaningful and transformative learning experiences for students.

Although the studies in our special issue span several continents and a variety of university types, there is a need for more research and development of additional STEM online courses that will employ digital assessment tools along with active learning pedagogies and collaborative learning environments.

In conclusion, this special issue sheds light on the critical importance of thinking skills development and assessment in higher education and underscores the need for continued research and innovation in the realm of educational technology. By embracing digital assessment tools, active learning approaches, and collaborative learning environments, educators can empower students to thrive in the rapidly changing landscape of the twenty-first-century workforce. Through ongoing dedication and interdisciplinary collaboration, we can pave the way for a more thoughtful, creative, and adaptable generation of STEM graduates capable of tackling the complex challenges of the future.