Supporting Student System Modelling Practice Through Curriculum and Technology Design

Models are generative epistemological tools, consisting of components and the relationships between them (Harrison & Treagust, 2000; Nicolaou & Constantinou, 2014; Schwarz et al., 2009), and are essential to scientists and engineers for representing a system and for explaining phenomena and predicting possible outcomes (Harrison & Treagust, 2000; National Research Council, 2012). Developing and using models is one of the core scientific and engineering practices described in A Framework for K-12 Science Education (National Research Council, 2012). Schwarz et al. (2009) defined the modelling practice as the ability to construct, use, evaluate, and revise models of the natural world. These prior works influenced how we discuss the four aspects of system modelling practice. To build scientific models, students need appropriate modelling tools that include scaffolds to support their ability to build and use models as explanatory and predictive mechanistic tools of phenomena (Clement, 2000; Lehrer & Schauble, 2006), but most students have few opportunities to meaningfully engage with models (Louca & Zacharia, 2012; Nicolaou & Constantinou, 2014; Schwarz et al., 2009).

Systems thinking is required for investigating and learning about complex systems and typically includes ideas such as the ability to consider the system boundaries, the components of the system, the interactions between system components and between different subsystems, and emergent properties and behaviour of the system (Passmore et al., 2014; Russ et al., 2008; Wilensky & Resnick, 1999). The difficulties of understanding complex systems are well documented (Booth Sweeney & Sterman, 2000; Dörner, 1980; Hmelo-Silver & Pfeffer, 2004; Jacobson & Wilensky, 2006). We suggest that engaging in modelling through the use of a system modelling tool with appropriate scaffolds can support students in developing a systems thinking perspective.

System dynamics (SD) was created at MIT in the 1950s to help humans think about complex systems (Forrester, 1968). With the introduction of computer simulations based on stock and flow models (Richmond et al., 1987), SD became more accessible and eventually began to be used in educational settings with children and young adults (Gould-Kreutzer, 1993). Within the five stages of SD modelling (problem identification, system conceptualization, model formulation, testing, and using), Martinez-Moyano and Richardson (2013) compiled a list of 27 best practice statements regarded of highest importance by practitioners and experts in SD. These include identifying the components of the system and the relationships between them, iteratively developing the structure of the models by adding detail as needed, comparing simulated with real behaviour, and iteratively testing and validating the model. Students face many challenges in thinking about complex dynamic systems (Stratford et al., 1998; Tadesse & Davidsen, 2020), and there has long been hope within the SD community for new tools to introduce these challenging ideas to an audience not as constrained by age and ability (Gould-Kreutzer, 1993).

To understand a system, explain it, and troubleshoot system problems often requires understanding causal relationships between components in the system so that one can predict and compare with real-world system behavior (Jonassen & Ionas, 2008). Student difficulties with causal reasoning have been well studied (e.g., Schauble, 1996; Zimmerman, 2007). The causal structures in complex systems can be especially difficult to reason through and to teach about (Perkins & Grotzer, 2000; Yoon et al., 2017). According to Jonassen and Ionas (2008), system modelling tools are the only class of tools that can enable learners to model both covariational and mechanistic attributes of causal relationships necessary for full causal reasoning.

Given the theoretical background discussed above, there is a need for better understanding of how instructional practices and technology tools can support students in system modelling.

In Bielik et al. (2019), we presented a theoretical examination of four aspects of system modelling practice illustrated with several student exemplars. These aspects are as follows: (1) defining the boundaries of the system by including components in the model that are relevant to the phenomenon under investigation, (2) determining appropriate relationships between components in the model, (3) using evidence and reasoning to construct, use, evaluate, and revise models, and (4) interpreting the behavior of a model to determine its usefulness in explaining and making predictions about phenomena. The first two aspects primarily grew out of challenges we observed when students construct system models. The second two aspects are related to our initial design ideas for a tool that would support sensemaking with models through comparative data analysis and ease of model construction. In the “Analysis” section, we describe criteria to evaluate whether students engage with each aspect and the results of applying those criteria to student work.

The modelling tool, a free, Web-based, open-source tool, was created by the authors in partnership with other project team members.¹ It is designed to scaffold student learning so that young students, beginning in middle school, can engage in systems thinking through constructing, using, evaluating, and revising models. The tool facilitates the modelling of a system and also makes it possible to calculate and visualize model output without requiring students to write equations or code (Damelin et al., 2017).

With our modelling tool, students begin by dragging images that represent components to a canvas and then linking the components with arrows to specify relationships (Fig. 1). For the system model to become a runnable model, each component is treated as a variable that can be calculated by the modelling engine. The next step for students is to define each relationship link in the model such that the impact of one variable can be calculated on all of the variables to which it is linked. Variable values are defined using a low-to-high scale. Students construct a verbal description of how one variable affects another by using drop-down menus, e.g., “An increase in temperature causes volume to [increase] by [about the same].” The resulting relationship is also depicted by a graph showing a visual representation of this relationship (Fig. 1). Defining relationships with words helps students overcome the mathematical obstacles typically associated with creating system models and allows them to focus on a conceptual understanding of the relationships between variables (Damelin et al., 2017).

To “run” the model, a student uses a slider to move an independent variable through a range of values. To facilitate analysis of the impact of this variable on the rest of the model, our modelling tool integrates CODAP, the Common Online Data Analysis Platform, a tool designed to support student exploration of data, including comparisons of model-output and real-world data sets (Finzer & Damelin, 2016).

This study involves data collected from an enactment of a high school curricular unit conducted in the spring of 2017 in an honors chemistry class. The enactment included 11 lessons of about 70 min each. Prior to starting the unit on the emergent properties of gases, all students completed an Introduction to Modelling unit, which directly addressed the issue of appropriate variables for a system model. The chemistry unit focused on the emergent properties of gases and was co-designed by the authors, together with a high school chemistry teacher who enacted the unit in her classroom. The unit was designed to explore the anchoring phenomenon of an oil tanker that imploded after being steam cleaned, as shown in an online video. This builds towards the driving question of the unit: ‘How can something that can’t be seen crush a 67,000 pound oil tanker made of half inch steel?’ A full description of the curricular unit can be found in Appendix 1. Descriptions of phenomena, activities, and expected target models in the unit are provided in Table 1.

Data was collected from 20 tenth grade students (14 male, 6 female) in a public school in the northeastern USA. Students were from an average socioeconomic level (27.5% of the students were eligible for free or reduced lunch, 88.5% White, 5.5% Hispanic, 2.5% Asian, 1.5% African-American). The teacher was an experienced chemistry teacher. Researchers observed the enactment and supported the teacher.

This aspect is characterized by the two following features:

Directing students to define the components in their models as measurable variables and not as objects in order to run a simulation of their model is not a simple task. It requires explicit focus of the teacher and repeated experiences. In the enacted unit, the teacher emphasized this issue when supporting the students in developing their models. Although most students produced initial models that included only variables that could be defined on a low-to-high scale, there were some exceptions. For example, in Fig. 1 of pair I initial model, the components Change in temp and Amount of pressure are defined appropriately as variables, but the component named Components of elements outside describes an object rather than a variable with specific characteristics, and cannot be defined on a low-to-high scale in any practical way. These students might have intended to describe the ratio of different elements in the air and to imply that the ratio might affect the air pressure. In any case, these students removed this variable in their first model revision (Fig. 2).

To exemplify this aspect, the model in Fig. 1, Change in temp is a variable of appropriate size and scope for the tanker phenomenon; temperature changes have an important effect on the outcome. Elevation, however, is an example of a variable that is not of appropriate size or scope. In their first revision (Fig. 2), these students associated elevation with Density of gas inside vs outside the tanker. Although the ratio of densities outside and inside is crucial to the phenomenon, the density outside does not change appreciably in the scenario and so does not help to explain the phenomenon. After a class discussion in which models containing elevation were discussed, these students (pair I) removed both Elevation and Density of gas inside vs outside as they were not needed within the scope of these students’ explanatory model. After their second model revision (Fig. 3) they wrote, ‘We did not think the elevation and density of the molecules in the tanker was crucial to the model.’

In their responses to the model reflection questions, students commonly mentioned that what they changed in their models was related to Aspect 1, in particular, variables (over 50% of responses after each model revision; over 80% after the second and third revisions; see Table 2). Students’ responses as to why they made these changes fell into one or more of the following categories: to have them better explain and describe the phenomenon under investigation, to make the models more correct, to have the models make more sense or be more logical, to include important or missing variables, to have more specific variables, or to remove variables that are not significant.

This aspect is characterized by the two following features:

There are several ways a link between two variables could be incorrect.

An example of this feature can be found in the initial model of pair I in Fig. 1; the sequence of two connections that link Size of tank to Amount of pressure do not match the real-world behavior of the system. The students changed this during their first model revision. As S13 wrote, “I got rid of this part of the model because an increase in volume does not lead to an increase in pressure. Actually, an increase in volume causes a decrease in pressure” (Fig. 4, third revision of pair I model). There was an additional change in their next revision (Fig. 5) that was also related to feature A, a change in the direction of causality between Amount of pressure and [V]olume. In the tanker scenario, a change in volume was not what caused the decrease in pressure; rather the decrease in pressure caused the volume to change—suddenly. These students continued to change relationships throughout each model revision, which was typical for all students.

In their responses to the model reflection questions about what they changed in their models, most students mentioned the relationships between the variables (74 and 75% after the first and second revisions, 100% after the third and fourth revisions; see Table 2). In their written reasons for making changes to these relationships, students mentioned: having the model make better sense and be more logical, aligning the model with new findings from experiments, making the model better fit what was learned in the lessons, having the model better explain the phenomenon under study, and including the necessary and relevant relationships.

There are two ways problems with the directness of relationships manifest themselves:

Gaps in causal chains sometimes resulted when students leapfrogged one or more steps in a causal chain of variables. Students were encouraged to add microscopic causal mechanisms, helping them to explain why one variable had an effect on another. Pair I students explicitly mentioned adding Kinetic Energy to fill a gap they perceived in their causal chain. After their third revision (Fig. 5), they wrote, “We added kinetic energy as a variable in between temperature and molecule speed in our model and had kinetic energy increase the same amount as temperature. These changes help respond to the unit’s driving question because they show how as temperature increases, so does kinetic energy and molecule speed about the same.” This is a situation where the model was used to explicate covariational and mechanistic attributes of the relationship between temperature and pressure. In a gas, temperature is defined in terms of average kinetic energy of the molecules. Explicitly including kinetic energy as a variable in their model allowed a direct link to molecule speed, which, in turn, allowed a direct link between speed and number of collisions, which provided a direct link to gas pressure (Fig. 4).

In this same model, the students added a causal mechanism, Number of molecule collisions, between Molecule speed and Amount of pressure in the tanker. The single link that also connects Molecule speed and Amount of pressure represents an indirect relationship. It complicates the model and adds no new information, and it was later removed.

Five of the nine student pairs who submitted complete model sets had indirect links in all of their model revisions (Table 4). However, after an initial increase in indirect links there was a drop in such links in their final models. In the third revision, there were 10 total indirect links in class models while in the final revision, there were 6 indirect links.

After designing and carrying out experiments, including using a molecular dynamics simulation of gas behavior, students revised their models. Some students cited evidence from the experiments and/or simulations as inspiration for changes in their models. Explicit references to evidence from experiments or other class activities were mentioned after revisions 1 and 2 (21% and 13% of the responses, respectively), but not mentioned after the third and fourth revisions (Table 2). This could have been due to the structure and order of the questions asked of the students, or perhaps because students tended to focus more on having the appropriate variables and relationships in their later model revisions. Observation notes suggest that more could have been done to support students in making links between their experimental results and the predictions their models generated.

An example where a student cited evidence from experimental results was S21’s written comments after the first model revision: “The biggest reason as to why I made the changes was because of the recent experiment that we conducted in which we learned about the relationship between pressure and volume. Also, based on this new knowledge, I removed things I thought weren’t applicable to the model.” Sometimes the link between experiment and model revision was more tenuous. For instance, to explain why she and her partner added a link to their model, S11 wrote, “We connected amount of air to pressure…. We did this because we learned that the amount of air affects pressure.” She did not mention where she had learned this relationship, although she correctly answered questions about an experiment she had conducted that explored it.

S13 and S20 exhibited this aspect of system modelling practice. They created a model that was testable, compared it with expected real-world behavior, and revised it accordingly. They also revised it to more clearly address the unit’s driving question, drawing a line of cause and effect all the way from Steam Cleaning to Implosion of Tanker (Fig. 6), although aspects of that chain were still problematic. They ran their model multiple times, and attempted several visualizations of the output, but appeared to have trouble deciding which relationships would be useful to explore. Nevertheless, they exhibited a clear ability to interpret the behavior of the model in terms of the visual representations of relationships, writing after their final revision, ‘We added steam cleaning because it increased the temperature, which increased the kinetic energy, which increased the molecule speed and number of collisions, which increased the pressure of the tank.” This is an impressive five-link chain that correctly described most of their model.

However, many of the students struggled with this aspect. This struggle was evident in how pair E students (S7 and S17) had air pressure in tanker as the outcome variable in all of their models (the last two of which are shown in Figs. 6 and 7), but never explicitly connected this to the driving question in any of their model reflection responses. Nonetheless, it was evident from their interviews that both students understood the connection between air pressure in tanker and the driving question about the tanker implosion. To interpret the behavior of their model, they ran it and constructed graphs of the relationships between three different variables and air pressure in tanker. Two of their graphs showed expected relationships; one did not. They could have used this unexpected behavior to evaluate the appropriateness of their model to address the driving question and to troubleshoot relationships in their model, but they did not do so. An excerpt from the final interview with S17 offers one possible explanation for why they were unable to do this on their own. After the student explained their model (Fig. 7) and exhibited some understanding of the phenomena involved, the interviewer asked him to trace lines of cause and effect between Size of tanker and air pressure in tanker. S17 responded: “… this is saying that the bigger the size of the tanker, the larger the amount of air in the tanker. So if there’s more air in the tanker, I’m guessing that it would have a higher air pressure.’ In this short, two-link causal chain, S17’s reasoning about each separate link would be correct only if the tanker could change size for the first relationship (bigger tanker means more air), but remain a constant volume for the second relationship (more air means higher pressure in tanker). Further consideration might have shown him that this reflected an inconsistency in the logic of his model and perhaps in his conceptual understanding of the cumulative effects of relationships.

In written responses to the model reflection questions, nine students (from five of the pairs) did mention changing their model to make it better explain why the tanker imploded or to connect the behavior of their model to real-world behavior, but most did so only once (Table 2).