Design feedback is an essential pedagogical tool that can help young novice designers navigate divergent and convergent paths while designing. However, design feedback is often met with resistance, which counteracts its potential to help novice designers evaluate their design and generate new solution directions. In this paper, we report on the construction and utilization of a design feedback intervention during a real-life design project with a group of primary school children (aged 8–12). The goal of the intervention was to stimulate young novice designers’ creative thinking by guiding the design feedback dialogues with their peers and clients. The intervention was designed according to the following key principles: (1) guide towards a shared understanding of the design through low-level convergent feedback, (2) stimulate critical reflection and evaluation of the design to help identify and internalize possible shortcomings through high-level convergent feedback, and (3) provide a way to move forward by guiding new generative thoughts through high-level divergent feedback. Overall, the results show that the intervention can support young novice designers, their peers, and clients in engaging in constructive feedback dialogues, thereby stimulating their creative thinking. Our main contribution entails a detailed understanding of the successes and obstacles within the feedback dialogues, as guided by the intervention. Based on these results, we propose a set of refined design principles to inform feedback interventions. With this research we hope to give insight in the complexity of design feedback dialogues, while also inspiring design educators to actively try out these key principles.
Hinweise
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Introduction
In the world we live today, being able to behave creatively is progressively seen as a skill of great value. It is therefore, no surprise that creativity is gaining more and more attention within all educational levels, and primary education is no exception. Designing is an undoubtedly creative activity (Barlex 2007; Goldschmidt 2014; Howard et al. 2007), and it offers excellent opportunities for children to develop their creative abilities (Lewis 2005, 2009; Rutland and Barlex 2008). When designing, children are confronted with ill-structured open-ended problems (Dorst 2003; Lewis 2005), for which no formula or single ‘right’ answer exists. Consequently, they are required to resort to creative thinking processes to explore the problem and solution space (Dorst and Cross 2001) and generate and develop solutions (Lewis 2009).
However, contrary to popular belief, engaging in creative thinking processes does not always come naturally to children (Luo 2015; McLellan and Nicholl 2009; Nicholl and McLellan 2007a, b). Creative thinking asks for a continuous shifting between divergent (generative) and convergent (evaluative) thinking processes which can easily be stagnated by several different factors. We have observed these instances of stagnation first-hand in the research project “Co-Design with Kids”, which this study is also a part of. In this research project, instances of stagnation became visible in the non-constructive design feedback dialogues the primary school children had with the clients and their peers (Schut et al. 2019a, b).
Anzeige
Design feedback is generally assumed to be beneficial to the creative design process (Crilly 2015). It is often utilized as a tool to guide creative thinking and the development of a design. However, previous work by the authors shows that children’s creative thinking is not necessarily stimulated or enhanced by the feedback conversations. In various cases, we observed that the students (pupils) showed resistance towards the feedback and a lack of evaluative and generative thinking processes (Schut et al. 2019a). When analysing the feedback dialogues, we observed problems with receiving, as well as constructing design feedback. Therefore, we believe that guiding how design feedback is given and received can create new opportunities to stimulate and enhance the children’s creative thinking while designing.
In this paper, we report on the construction and use of a feedback intervention meant to stimulate constructive feedback dialogues between design teams of primary school children, their peers and the clients, while participating in a real-life design project. The overall goal of this study is aimed at gaining insight in the possible ways of guiding a design feedback dialogue in order to stimulate young novice designers’ creative thinking. This study takes a design-based research approach, in which the feedback intervention is developed and optimized over the course of a series of case-studies (Bakker 2018; Van den Akker et al. 2006). We will uncover enablers and obstacles for generating productive feedback processes through applying the feedback intervention. Based on these insights we will recommend further improvements for applying productive feedback interventions.
Literature review
Creative thinking in design
Creative thinking processes are central to designing and can be seen as the key driver of the creation and development of a design. Central to creative thinking is the utilization of both divergent (DT) and convergent (CT) thinking processes (Goldschmidt 2014, 2016; Guilford 1967; Howard-Jones 2002). DT is generative in nature and entails the generation of new thoughts and ideas, and CT is evaluative in nature and entails reflection and evaluation of these (or existing) thoughts and ideas (Sowden et al. 2015). Following the dual-process theories of creative cognition, there is a continuous shifting between both modes of thinking when working towards a creative solution (Finke et al. 1992; Howard et al. 2008; Sowden et al. 2015).
These continuous shifts are not only essential for the generation of initial ideas, but also for the development of these initial ideas into complete final designs (Isaksen et al. 2010; Tassoul 2009). Through critical reflection and evaluation (CT), various shortcomings and sub-problems within the designs can be identified, which, in turn, can lead to the generation (DT) of new additions and alterations. Navigating these creative thinking processes is not an easy endeavour, especially for young novice designers. Therefore, design feedback is commonly utilized within design education as one of the types of guidance (Dannels et al. 2008; Nicholl 2004; Tolbert and Daly 2013).
Anzeige
Feedback in design education
Thus far, little is known about design feedback in the context of primary education (Schut et al. 2019b). However, feedback interventions are a common educational practice within the context of design disciplines at a university level and are known as design critiques (crits) or design reviews (Anthony 1991; Dannels and Martin 2008; Healy 2016). Students get the opportunity to update their instructors, peers and other stakeholders—such as real or simulated clients and potential users—on their process and the status of their design and collect feedback (Dannels and Martin 2008; Oh et al. 2013). These critiquing moments can be considered as central to design education and practice (Gray 2013a, b). It prepares the novice designers for the ‘real world’ by socializing them into the discipline (Oh et al. 2013) and it is seen as a primary form through which they acquire expertise from their instructors and other stakeholders (Dannels 2005; McDonnell 2016; Oak 2000) and develop their design skills (Hokanson 2012; McDonnell 2016).
Design feedback dialogues are often about improvement and development of a design and can directly impact the divergent or convergent paths a novice designer might take (Cardoso et al. 2014; Oh et al. 2013; Yilmaz and Daly 2016). Questioning can spark novice designers to think more deeply and broadly and help them to improve and develop their design (Stables et al. 2016a, b). Within general education, it is well recognized that dialogues can play an essential role in learning when guided in a constructive manner (Colfer 2017; Mercer and Littleton 2014). However, it is well known that design feedback is not always shared through constructive dialogue and therefore, does not always benefit the creative thinking processes (Anthony 1991).
Non-constructive feedback dialogues
At a primary school level, studies show that feedback dialogues can easily become non-constructive (Schut et al. 2019a, b). Specifically, Schut et al. (2019a, b) discovered that the convergent or divergent nature of design feedback does not automatically guide young novice designers towards the concurrent creative thinking processes. This entails that convergent feedback does not necessarily spark the evaluation of possible shortcomings and divergent feedback does not necessarily spark the generation of new additions and solutions. For example, Schut et al. (2019b) observed how convergent feedback was often met with defensive behaviour, resulting in the young novice designers trying to prove their design already met the design criteria (Schut et al. 2019b). This discrepancy between the direction the design feedback drives towards and the actual thinking processes that take place appears to occur at all educational levels.
At university level, several studies indicate how instructors appear to mainly utilize convergent feedback, through which they tend to focus on the clarification and reasoning behind certain design decisions (Cardella et al. 2014; Cardoso et al. 2014; Cummings et al. 2015; Daly and Yilmaz 2015; Yilmaz and Daly 2014, 2016). As a result, the instructors expected a level of reflection and evaluation of the students. However, the students were often found to become too descriptive (Cardoso et al. 2014) or even defensive, which, similar to the young novice designers (Schut et al. 2019b), led them to trying even harder to convince everyone of the quality of their design (Cardella et al. 2014; Cummings et al. 2015). Although favouring convergent feedback is an understandable strategy when wanting to eliminate ambiguity and encourage reflection and evaluation, these results show that students’ convergent thinking processes are not always that easily triggered.
At a primary school level, Schut et al. (2019b) also observed that the high level of convergent design feedback did not always promote constructive dialogue. Similar to the studies at a university level, it was observed that the convergent feedback did not automatically trigger reflection and evaluation with the young novice designers. Instead, they showed resistance to the feedback of their peers and the client by rejecting or ignoring it. Especially the convergent feedback of the peers, which actively asked the young novice designers to reflect and evaluate their past, present, and future design decisions, was often met with resistance (Schut et al. 2019b).
Overall, divergent feedback appears to occur rather infrequently in the studies at a university level (Cardella et al. 2014; Cardoso et al. 2014; Cummings et al. 2015; Yilmaz and Daly 2016) as well as at a primary level (Schut et al. 2019b). Therefore, there is not as much known about novice designers’ responses to this type of feedback, as with convergent feedback. Although divergent feedback would ideally stimulate a form of ideation, Schut et al. (2019b) observed that it was again often met with resistance, similar to the convergent feedback. Based on this review, we conclude that there is often a discrepancy between the goals of educators and other feedback givers and the actual thinking processes that are triggered.
Resistance towards design feedback
The reactions of the novice designers in the previously discussed studies display that they are not always open to feedback, which could hinder their creative thinking and, therefore, the development of their design. To be able to behave creatively while designing, it is essential to keep a balance between persistence and openness (Crilly 2015; Csikszentmihalyi 1999). On the one hand, a designer needs to be persistent in pursuing valuable ideas in the face of negative feedback (Crilly 2015), especially since creative ideas are often not recognized in the early stages (Tassoul 2009). However, on the other hand, one must also be open to the possibility of being misdirected (Crilly 2015).
Previous work by the authors uncovers how the young novice designers’ resistance and lack of openness when developing their designs can be explained through the occurrence of concept fixation (Schut et al. 2019a). This type of fixation appears to have been observed in multiple studies with young novice learners (Luo 2015; Nicholl and McLellan 2007a, b), although the difference of this type of fixation from fixation in the ideation phase is often not made clear. Concept fixation is a type of design fixation—see Jansson and Smith (1991) and Purcell and Gero (1996)—that can occur during the concept development stage of a design process and “leaves the designer stuck in a certain train of thought, adhering to the current (possibly unfavourable) state of the design (idea)” (Schut et al. 2019a). A creative design process is full of insecurities, and part of the process is to become comfortable with that (Lawson 2006). It is, therefore, understandable that novice designers tend to hold onto their ideas. Additionally, research by Baer and Brown (2012) suggests that too much ownership can lead people to have feelings of negative affect and loss towards proposed changes, which directly influences their openness to change.
Overall, too much persistence and adherence lead to a lack of critical reflection and evaluation, which already appears to be limited with young novice designers compared to experienced designers (Blom and Bogaers 2018; Goldschmidt 2016). This can leave possible shortcomings in the design undiscovered and, in return, dismiss any need for new divergent thinking processes to generate new solutions and elaborations. Therefore, guiding convergent design feedback constructively could help young novice designers to uncover shortcomings in a manner that does not evoke as much resistance.
Overall, this review brings us to the questions of whether and how guidance in giving and receiving convergent and divergent design feedback can help overcome unconstructive feedback dialogues. We want to address this gap in knowledge by proposing, implementing, and analysing a design feedback procedure that focusses on evoking desired concurrent creative thinking processes with young novice designers.
Stimulating convergent and divergent thoughts
Studies by Eris (2004) and Cardoso et al. (2016) utilizing Eris’ Question Driven Design Model showcase that high-level questioning can facilitate moments of critical reflection and evaluation within design teams of university students, which helps them realize previously unthought-of obstacles or concepts (Eris 2004) and makes them consider alternatives to their current design trajectory (Cardoso et al. 2016). Additionally, the results of both studies show that although high-level convergent feedback can facilitate moments of reflection and evaluation, the combination with high-level divergent feedback is needed to move the creative design process forward (Cardoso et al. 2016; Eris 2004). This necessity for providing a way forward is also affirmed by research into effective feedback for learning within a general educational context (Black and Wiliam 1998; Hattie and Timperley 2007). Additionally, a lack of divergence can hamper exploratory thinking and risk-taking, which are essential within creative processes (Daly and Yilmaz 2015; Tolbert and Daly 2013; Yilmaz and Daly 2016).
However, previous work by the authors shows that just posing high-level convergent and divergent feedback is not enough to ensure constructive dialogue and can still evoke resistance (Schut et al. 2019b). In that study, it was uncovered that (implicit) expectations and assumptions the peers and clients had about the design ideas lead to a lack of mutual understanding about possible shortcomings present in the designs. Interpretive challenges are known to cause communication problems during feedback dialogues (Higgins et al. 2001; Sadler 2010). Hence, before posing high-level feedback, a shared understanding of the design needs to be reached (Schut et al. 2019b; Stables et al. 2016a, b). Therefore, questions that ask for verification and clarification still appear to be a critical first step.
In conclusion, in order for design feedback to be effective in stimulating young novice designers’ creative thinking it needs to: (1) guide towards a shared understanding of the design through low-level convergent feedback, (2) stimulate critical reflection and evaluation of the design to help identify and internalize possible shortcomings through high-level convergent feedback, and (3) provide a way to move forward by guiding new generative thoughts through high-level divergent feedback. These conclusions will form the primary guidelines for the construction of the new design feedback intervention.
We are interested in the influence of the design feedback intervention on the divergent and convergent nature of the design feedback and the concurrent creative thinking processes of the young novice designers. This led to the following research question: What influence does the (peer)feedback intervention have on the convergent and divergent nature of the design feedback given by the client(s) and peers and the concurrent direct responses of the design teams? Our main contribution entails a detailed understanding of the successes and obstacles within the feedback dialogues within a primary school context as guided by the new feedback procedure; and possible future improvements.
Research design
To answer the research question we utilized a design-based research approach, comprising of a small pilot case-study and series of two full case-studies over the course of one school year (Bakker 2018; Van den Akker et al. 2006; Yin 2014). During the case-studies, groups of primary school children (ages 8–12) from two different schools were guided to design solutions for real-life design challenges. Part of this guidance consisted of the developed feedback intervention. The intervention was slightly adapted after each case-study. However, as will be described in the ‘design feedback intervention’ section below, most of these changes did not affect the guidance of the feedback dialogues, which is the focus of this article.
The case-studies described in this paper, are part of a bigger iterative cycle within the design-based research approach. During the preceding case-studies the problem of resistance was diagnosed (Schut et al. 2019a, b), while in the case-studies described here the focus lies on the creation of a feedback intervention to encourage creative thinking and help overcome or avoid this resistance. Although one could argue for additional iterations, we believe the current case-studies allowed us to do reliable data analysis and form an adequate first version of the design feedback intervention. Furthermore, we will indicate in the ‘discussion and conclusion’ section how the design principles underlying the feedback intervention could be enhanced for future use within an educational and/or research context.
Participants
The two case-studies took place at two different primary schools in the Netherlands. The selection of schools was based on their interest in design and technology education and a wish to experience a guided hands-on design project. From each school, one classroom of children participated in the study. Table 1 gives an overview of the two case studies, which we will now name case A and case B. In both school classes the children (age 8–12) had no previous hands-on experience with designing, apart from 8 children of case A who participated in a short pilot case-study a few weeks prior (Van Mechelen et al. 2018). Before the start of the design project, the children were divided into gender-mixed heterogeneous design teams of four children by the teacher. In case A, the children who participated in the pilot case-study were divided over the teams.
Table 1
Overview of cases A and B
Case A
Case B
Education type
Montessori
Dalton
Participants
School class of 27 children (age 8–12), 7 design teams total, of which 4 selected for in-depth analysis
School class of 27 children (age 9–11), 7 design teams total, of which 4 selected for in-depth analysis
Design theme
Outdoor education: Design something that enables regular indoor classes (calculus, language, geography, etc.) to be taught outdoors
Long-term hospitalized children’s wellbeing: Designing something for the park of the hospital that enables hospitalized and non-hospitalized children to play and be active together
Design sessions
6 sessions of 90–120 min
Spread over 3 weeks (1 session a day)
6 sessions of 90–120 min
Spread over 2 weeks (2 sessions a day)
Client(s)
1 adult representative of the Dutch NGO: Jantje Beton, which is an organisation that promotes outdoor play and activities
1 adult representative and 4 child representatives of a Dutch children’s hospital
Client(s) background
Responsible for the creation of several of Jantje Beton’s nationally promoted outdoor play events and activities for primary school children
Adult representative:
Head of the education department and manages the hospital’s children’s council
Child representatives:
Part of the children’s council and are experts through experience. They are either chronically ill or previously long-term hospitalized children. The council helps the hospital with improving any aspect of care
From each school class, four design teams were selected for in-depth data collection and analysis. The selection was made by the teacher and based on our request to select four teams that could represent the class as a whole concerning academic achievement and collaboration. This was done to ensure the collected data would represent the typical classroom variations in ability and level of engagement of the participants.
Design sessions
During the two case-studies, the design teams worked on solving real-life design challenges presented to them by real clients. For 2–3 weeks, they were guided through the entire design cycle in design sessions of 90–120 min. The different phases of the design sessions were structured as follows: introduction and sensitizing, exploring the design theme, defining a point of view, ideation, selection and detailing, feedback on the design ideas, concept development and prototyping, and feedback on the final designs. The design activities were based on tools and methods from the CPS tradition (Isaksen et al. 2010; Tassoul 2009), design tools from the Delft Design Guide (Van Boeijen et al. 2013) and lessons learned from previous research conducted by the authors. These lessons learned included, but are not limited to, the following topics: design fixation (Schut et al. 2019a), design feedback (Schut et al. 2019b), group dynamics (Van Mechelen et al. 2015), co-design processes (Van Mechelen 2016; Van Mechelen et al. 2019a), and children’s design skills (Van Mechelen et al. 2018, 2019b). Table 2 gives a concise overview of the content of each of the design sessions. Furthermore, the tools used in the design sessions can be found on the following websites: www.tudelft.nl/codesignkids (English version) and www.tudelft.nl/yourturn (Dutch version).
Table 2
Overview of the content of the design sessions
Design session
Content
0. Introduction and sensitizing
Pre-session one week prior to the start of the design project
Announce the start of the project and give a short introduction
Children received sensitizing assignments that they had to complete before the start of the first design session. These assignments were meant to trigger reflection about the design theme
1. Exploring the design theme
Introduction of the design cycle and design skills
The client (s) introduced the design theme
Exploring design theme through stories
2. Defining a point of view
Introduction to problem defining
Defining an ideal situation, design question, and design criteria
3. Ideation, selection and detailing
Ideation guided by brainstorming tools
Idea selection guided by selection tools
Detailing selected idea with an elaboration tool
4. Feedback design ideas
Each design team presents their design idea and receives feedback from the clients and their peers
Focus on improvement and elaboration of the design idea
5. Concept development and elaboration
Each design team selects design feedback to improve and elaborate on their design
Building models or prototypes to develop the design idea
6. Feedback final designs
Each design team presents their final design and receives feedback from the clients and their peers
The design sessions took place in a classroom setting, and all design teams took part simultaneously. The first and second authors facilitated the sessions. The teacher of the class was present during all of the design sessions to assist with any classroom management or behavioural issues. The researchers facilitated the design sessions instead of the teachers, to ensure that the content of the sessions was implemented as envisioned.
Design feedback intervention
Overall, the goal of the feedback intervention was to support a constructive feedback dialogue that would stimulate the design teams’ creative thinking and help prevent and/or overcome resistance. The feedback intervention was based on the conclusions from the literature review and a pilot case-study (Van Mechelen et al. 2018). It was implemented in the 4th, 5th, and 6th design sessions. Furthermore, a few changes were made to the procedure in case B based on the observations in case A. Since these changes primarily affected aspects of the procedure during the 5th design session, they are disregarded in this particular paper.
In the 4th and 6th session, a design critique took place, during which the design teams took turns to present their design and receive feedback from their peers (the other design teams) and the clients. In the 4th session, the focus of the critique was on improving and elaborating on the designs. In the 6th session, the focus was firstly on communicating the final designs in detail to the client and secondly on exploring future improvements and elaborations. During the 5th session, the design teams selected the feedback which they wanted to utilize to improve their design. Since the focus of this study is on the feedback dialogues, the focus will lie on the implementation of the feedback intervention during the 4th and 6th design sessions. Table 3 gives a concise overview of the procedure of the feedback intervention during these two sessions.
Table 3
Overview of the procedure of the feedback intervention in the 4th and 6th design sessions of cases A and B
Part
4th design session
6th design session
Preparation
Modelling exercise by facilitators to teach the intention and procedure of the feedback intervention
A reminder of the intention and procedure of the feedback intervention
Next parts form one turn per design team
Part 1
Design team presents design idea
Design team presents design prototype
Peers and clients are prompted to pose clarification questions to work towards a shared understanding of the design
Peers and clients are prompted to pose clarification questions to work towards a shared understanding of the design
Peers and clients receive scaffolding guidance by feedback forms
Peers and clients receive verbal prompts as a reminder
Part 2
Peers and clients are guided to construct specific compliments
“What do you not yet understand about the design?”
Peers and clients are prompted to pose specific compliments
“What do you not yet understand about the design?”
Peers and clients are guided to construct high-level convergent feedback
“We think this could be better about the design… because …”
Peers and clients are prompted to pose high-level convergent feedback
Part 3.
Peers and clients are guided to construct a high-level divergent how-question
“How can you …”
Peers and clients are prompted to pose high-level divergent feedback
Peers and clients are guided to construct possible solutions to their how-question
“We think this could maybe be a solution …”
The clients and a few peers share their feedback with the design team by reading the feedback forms out-loud
The quotations are copied from the feedback forms
Preparation
Before the start of the feedback procedure, the children and the clients participated in several modelling exercises to teach them how to construct the design feedback. During these exercises, the facilitators demonstrated the different steps of the feedback procedure and allowed the children and clients to practice each step through several examples. All participants were made aware that the goal of giving and receiving design feedback was to ‘help each other make their design better’. The two researchers facilitated the entire feedback intervention and acted as gate-keepers to guide the procedure as planned.
Additionally, a feedback form was utilized during the 4th design session as a scaffolding tool to help the peers and clients with the construction of their design feedback (part 2 and part 3). These feedback forms were filled in collaboratively by the peers in their design teams. When more than one client was present, they also collaborated on one form. After constructing the feedback forms, the clients and some of the peers could share their feedback by reading it out loud to the design teams. During the 6th design session, the feedback form was not utilized again. Instead, the facilitators utilized verbal prompts to remind to the peers and clients of the structure of the feedback form and the order in which they should pose their feedback.
Part 1: Guide towards a shared understanding of the design
To start, each design team took their turn to present their design to their peers and the clients, utilizing sketches and prototypes or models to clarify their idea. Directly after each presentation, their peers and the clients were invited by the facilitator to pose low-level convergent questions to help them verify and clarify their understanding of the design. As guidance, the facilitator posed the following question: “What do you not yet fully understand about their design?”. These questions should not directly contain a form of judgement or stimulate a higher level of reasoning. When a shared understanding of the presented design was reached, the procedure continued with guiding the construction of high-level convergent (part 2) and divergent (part 3) feedback.
Part 2: Stimulate critical reflection and evaluation of the design
This part involved the peers and clients in constructing high-level convergent feedback to help the design team identify and internalize possible shortcomings in their design through critical reflection and evaluation. Before giving high-level convergent feedback, the peers and clients were asked to mention one or more specific and particular positive aspects of the design, which they could write on the feedback form (4th session) or directly share with the design team (6th session). Afterwards, specific and concrete high-level convergent feedback was constructed. In the 4th session, this was guided by the feedback form by finishing the following sentence: “We think this could be better about the design… because …”. The ‘because’ was implemented to guide the construction and communication of an adequate argumentation for the uncovered critique. Previous research by the authors has highlighted this type of argumentation is often absent in a more unguided setting, while this type of transparency was found important to the design teams (Schut et al. 2019b; Van Mechelen et al. 2018). During the 6th session, the peers and clients could directly give high-level convergent feedback to the design team. They were reminded of the feedback form via prompts, yet were also free to pose other types of high-level convergent feedback. This was done intentionally in order to gain insight into the types of questions they would pose in a more unguided setting.
Part 3: Provide a way to move forward by guiding new generative thoughts
After the high-level convergent feedback, the peers and clients were invited to construct high-level divergent feedback to provide the design team a way to move forward by guiding new generative thoughts. In the 4th design session, this was again guided by the feedback form. First, they were guided to transform their high-level convergent feedback into a high-level divergent question: “How can you …”. Then, they were asked to construct possible solutions to this question, which could serve as an inspiration to the design teams: “We think this could maybe be a solution …”. In the 6th session, the peers and clients could directly pose the high-level divergent feedback to the design team. They were reminded of the feedback form via prompts, yet were again free to pose other types of divergent high-level questions. This was again done intentionally in order to gain insight into the types of questions they would pose in a more unguided setting.
Data collection
During both case-studies, all design sessions were audio and video recorded. Due to the focus of this study on the verbal feedback interactions, only the recordings of the 4th and 6th design sessions were selected for data analysis.
Data analysis
To analyse the data, we followed the Verbal Analysis approach (Chi 1997), which consisted of the following steps: segmenting the transcripts in units of analysis, selecting and developing the coding frameworks, applying the coding frameworks in a coding process, and pattern identification and interpretation. The transcriptions of the audio–video recordings of the interactions of the selected design teams during 4th and 6th design sessions during cases A and B were used as the primary data source.
Units of analysis: design feedback and direct responses
Units of analysis were created of consecutive feedback and direct responses based on the conversation content. Only the feedback and responses focussing on the design in question were considered. Within the units, pairs of feedback and response were formed. When multiple questions and comments were given in a row, or when multiple answers were given in a row, these would be grouped to form one pair consisting of multiple feedback and response codes.
Coding framework: types of feedback
To determine the nature of the feedback of the clients and the peers Schut’s Design Feedback Model was used, which is displayed in Fig. 1. The current model has evolved from the authors’ initial version (Schut et al. 2019b) and is originally based on Eris’ question-driven design model (Eris 2004). When instances of feedback could not be coded with the model, additional codes were formed through inductive coding and added to the model. The Design Feedback Model makes, identical to Eris’ model, a distinction between two levels of feedback: low and high. However, in this model, low-level feedback is divided into low-level questions and low-level comments. Low-level questions are mainly information-seeking questions and are posed when someone giving feedback wants clarification or verification about certain aspects of the design. Low-level comments are expressions that do not directly ask for a direct response and are mainly posed to express a positive or negative judgement or give a direct suggestion. High-level feedback is, similarly to Eris’ model, divided into deep reasoning questions (DRQs) and generative design questions (GDQs). High-level feedback asks for a higher level of reasoning and often entails reflection, evaluation, and/or generation. In the model, low-level feedback and DRQs are classified as convergent. This type of feedback is presumed to facilitate convergent thinking processes and share the common premise that a specific answer, or a specific set of answers, exists. GDQs are classified as divergent since they are presumed to facilitate divergent thinking processes by proposing alternative answers and prompting idea generation.
Fig. 1
Design feedback model. The feedback highlighted in grey emerged in the current study
×
Coding framework: types of direct responses
To determine the different types of direct responses of the design teams Schut’s Direct Responses Model was used, which is displayed in Fig. 2. The current model has evolved from the authors’ initial version (Schut et al. 2019b) and was originally constructed through open coding and comparison to the framework of Cardella et al. (2014) and Cummings et al. (2015). When instances of feedback could not be coded with the model, additional codes were formed through inductive coding and added to the model. The Direct Responses Model makes a distinction between different groups of direct responses, namely: resistance responses, report type responses, agreement type responses, deep reasoning response.
Fig. 2
Direct responses model. The responses highlighted in grey emerged in the current study
×
Coding process
All of the feedback and response pairs were coded with feedback and direct response codes from the coding frameworks. Additionally, we classified who posed the feedback to the design teams: the client(s) or their peers. Initially, the first author coded all of the feedback and response pairs of the entire dataset. Subsequently, the third author was trained by the first author in the use of the coding frameworks and independently coded all of the feedback and response pairs of the entire dataset. Consistency was promoted by routinely comparing and discussing the coded data per design team until full consensus was reached.
Since the third author needed to be trained and therefore needed practice in coding the data, the initial consensus when comparing the codes rose progressively. When initial consensus was not present, this was often due to a lack of contextual knowledge by the third author who was not present during the design sessions. Or due to confusion surrounding certain codes with a descriptive overlap, as not all descriptors are mutually exclusive. Therefore, full consensus reached easily during the discussions. All coding decisions were documented, and code descriptions were adjusted accordingly when needed. Thus, trustworthiness was ensured for by the consensus-seeking manner in which the coding was done.
Pattern identification and interpretation
To be able to analyse coded data tables were constructed with the code occurrences and code co-occurrences of cases A and B. These tables, together with the coded transcripts, were used as a means to interpret the data and look for patterns.
Findings
The following sections introduce the results of the qualitative analyses. Each section will start with the objective of the feedback procedure during that part of the design session and the expected design feedback (DF) and direct responses (DR). Subsequently, we will present actual occurrence and co-occurrence of the different types of DF and DR in cases A and B. Translated excerpts of the transcripts of the feedback dialogues are given to illustrate the results. In these examples, the children that were part of the design team are indicated as ‘child’ and, if necessary, a number. The peers and clients are indicated as such and, if necessary, a number.
4th design session: feedback procedure ‘part 1’
During each feedback round of a design team, the first part of the feedback procedure is meant to guide towards a shared understanding of the design by allowing the peer and clients to pose clarification type questions. Therefore, the expected DF from the clients and peers are primarily low-level convergent questions. The expected DRs from the design teams are those facilitating clarification and explanation.
Clarifying the design through low-level convergent feedback
As expected, the majority of the DF posed in both cases consisted of low-level convergent questions. With these questions, the clients and peers predominantly checked their understanding of the design through verification questions; and asked for additional information about the design through feature specification and concept completion questions. The DR of the design teams to these low-level questions in both cases consisted mainly of report.
Peer: So, you have to solve a math problem and the answer… Uhm… What did you have to do with the answer again? [Concept completion]
Child: The answer, that number is how often you have to score (in the goal). [Report]
Client: Does the (hospitalized) child have to pull it (the cable of the cable cart) themselves or someone else? [Feature specification]
Child: No, someone else who is there. [Report]
There were also a few instances of report uncertain, which shows that the design teams are not always fully confident when explaining their design in its infant state. Below is an example of this type of DR.
Peer: Where should you position yourself to score? [Feature specification]
Child: I think the player can decide themselves. [Report uncertain]
Beside report type responses, the low-level convergent questions in case A were also met with the DR confirm. This difference can be attributed to the higher occurrence of verification questions in this case.
Peer: So, you have to run as fast as you can to the right spot (on the playground)? [Verification]
Child: Yes. [Confirm]
The majority of the DRs during the first part could be considered as responses that enable a constructive feedback dialogue. However, there were also a few resistance responses present in both cases. Since low-level convergent questions ask for information and do not directly ask for any evaluative or generative thinking processes, these DRs were unexpected. Nonetheless, the example below illustrates how questions asking for additional information about certain aspects or mechanisms of a design can trigger design teams to parry the feedback.
Peer: How many times do you have to score? [Feature Specification]
Child 1: You should decide for yourself. [It’s you] You should use a sort of timer, and then you can…, and then as fast as you can. [Band-aid]
Child 2: How often you can score in a minute or something. [Band-aid]
Unexpected DRQ’s and GDRs
Aside from the expected low-level convergent questions, there were also instances of Deep Reasoning Questions (DRQs) and Generative Design Questions (GDQs). Especially in case B the peers posed several DRQs to the design team, consisting mainly of Future and Future description questions. Through these types of questions, the peers asked the design teams to reflect and evaluate possible future states of the design. The DRs to these high-level convergent questions varied between report type responses and responses indicating disagreement (disagree) or resistance (e.g., band-aids, already in there).
Peer: Yet at a certain moment, then all the (drawn) chalk boxes (on the schoolyard) are full (with numbers), and then you cannot continue anymore. [Future description]
Child: Yes, but you can just make it (the chalk drawings on the schoolyard) wet, then it’s gone. Then it’s vanished, gone (hand movement) (laughing). [Band-aid]
The example above illustrates the occurrence of resistance towards high-level convergent feedback. Since the clients and peers were prompted to not yet pose these types of questions at this part of the process, the facilitators tried to postpone the dialogue as much as possible. This can be seen in the example below.
Peer: I can imagine that some kids do not always have someone accompanying them. So what if that happens and they still want to go in the cable cart? [Future]
Facilitator: Is that a clarification question, or do you think this might be a critique? Is this something you could put on the feedback form?
Peer: Maybe yes.
4th design session: feedback procedure ‘part 2 and 3’
The second part of each feedback round corresponds with the 2th and 3th part of the feedback procedure, during which the constructed high-level convergent and divergent feedback were meant to stimulate the design teams’ reflective, evaluative, and generative thinking processes.
In the 4th design session, the peers and the clients utilized the feedback forms to construct written feedback. From these forms, a few were shared with the design teams on the spot by reading them out-loud, to which the teams could respond. Other forms were shared with the design teams later on in session five, but this part of the process is not analysed in this article.
Beforehand, we found it difficult to predict which type of DRs might occur as all feedback is shared consecutively due to the read out-loud format.
Direct responses to the feedback forms
As expected, the DF of the clients and peers in both cases was generally in accordance with the structure of the feedback form. This means that the feedback read out-loud by the client or by a design team usually started with a low-level comment (compliment), followed by high-level convergent feedback (DRQ) and followed with high-level divergent feedback (GDQ).
Almost all of the DRQs consisted of future description, which is one of the new feedback codes added to the model during analysis. Through this type of feedback, a person uncovers possible future strengths and shortcomings in the design by describing a scenario in which the design is used. As this type of feedback did not occur in any of the feedback sessions in our earlier case-studies without the feedback intervention, we are quite certain that the occurrence of this new type of DRQ can be attributed to the formulation on feedback form (“We think this could be better about the design … because …”). The form stimulated mental modelling and describing how a design might function in practice.
Below is an example of a feedback form that is read out-loud during case A. The peers give feedback on a design idea that combines math and sports. In the game, a player first solves a math problem and then scores in a goal to match the number of the answer.
Peer 1: We think the idea is a lot of fun. [Compliment] Uhm (…) ‘What could be better’ (…) Yeah, well, having to score that often is a lot. And that can be quite tiring. So, imagine you have 121 times, that is then the answer. If you then have to score 121 times you lose all your time trying to score. Then you can maybe only solve one problem. [Future description] Our ‘how-question’ is: How could you change it (the game) so you don’t have to score that often? [Method generation] And ‘this could maybe be a solution’: that you can divide it. The answer. [Proposal/Negotiation]
Peer 2: Yeah, so that you divide the answer again. That you again make a math problem with the answer to divide it. [….]
In this example, the GDQs consists of method generation followed by a proposal. As intended, almost all of the GDQs on the feedback forms consisted of method generation and proposal/negotiation.
In the analysis, we looked at the DR of the design teams towards the reading out-loud of the feedback forms. In both cases, low-level comments (compliments) were mainly met with a DR in the form of silence. In case B, a few instances of nodding and agree also occurred when the clients shared their low-level comments.
The DRQs of the clients co-occurred with the DR silence in both cases. However, DRQs from the peers were also met with a few instances of resistance. This could be explained through the expert position the clients take. Design teams might see them as more knowledgeable as their peers and are therefore more willing to accept their questions.
The DRs to the GDQs were more varied than to the DRQs, although again silence took up a substantial part. Other types of responses were mainly observed towards the GDQs of the peers in case A and the clients in case B. In case A, the GDQs of the peers were met with a few DRs indicating resistance, yet there were also DRs present indicating more openness, namely, acknowledge and ideation. In case B, DRs in the form of nodding, reflection, and ideation to the DF of the clients were present. This shows that the design teams were able to receive the feedback in a positive way.
6th design session: feedback procedure ‘part 1, 2 and 3’
In this design session the feedback procedure again makes a distinction between the different parts of the feedback procedure. However, since this time only verbal prompts were used, all of the feedback parts follow each other directly during each feedback conversation. The focus in this 6th session lies predominantly on revealing the final design to the clients and peers and less on possible further improvement. Therefore, we expect the clients and peers to work more towards understanding the design fully through low-level and high-level convergent feedback. Nonetheless, we expected all types of DF to be present in this session.
The peers’ focus on convergence
Overall, convergent DF was most prominent, yet all types of DF were present in this 6th session in both cases. In both cases the peers posed around the same amount and types of DF. Low-level convergent questions, low-level convergent comments, and high-level convergent questions all took up around a small third of the total DF of the peers. Additionally, there were a few instances of high-level divergent questions.
As expected, due to the guidance in the feedback procedure, the peers posed most of the low-level convergent questions right after the presentation of each design team at the start of the feedback dialogue. Similar to the results of the 4th session, the peers utilized this first part of the feedback procedure to check their understanding of the design through verification questions; and ask for additional information about the design through feature specification and concept completion questions. Especially in case A the peers predominantly posed verification questions, while in case B the questions were more varied. In both cases, the low-level convergent questions were often met with a report DR. In case A the confirming response also occurred often, which can be related to the higher occurrence of verification questions. Other DRs that stood out was the occurrence of a few instances of deep reasoning responses. Since these DRs indicate a form of higher-order thinking, we did not expect them to occur after any of the low-level convergent questions directly. Below is an example in which a low-level question evokes reflection and ideation.
Peer: What is actually the height of the cart (of the cable cart)? [Feature Specification]
Child 1: We did not really think about that… [Don’t know yet]
Child 2: Not too high I guess. [Reflection]
Child 3: Maybe a meter? [Ideation]
The peers did not adhere to the order of the feedback form when posing high-level feedback in this session. Instead of starting with convergent feedback and following with divergent feedback, the high-level feedback appeared to be mainly convergent with only a few instances of divergent feedback. The high-level convergent feedback (DRQs) of the peers was varied in case A, while in case B the majority consisted of future and future description. These two types of feedback are somewhat similar, in the sense that they uncover possible future functioning and often shortcomings in the design. Since in the 4th design session the high occurrence of the feedback type future description is presumed to be linked to the phrasing of the feedback form, the high occurrence of future and future description may be a lingering effect of the peers’ previous practice with filling in these forms.
In both cases, the DRs of the design teams to the DRQs were fairly similar in their distribution. Around half of the DRs consisted of report type responses, of which around half was report justify. Especially in case B report justify took a more prominent role. Report justify is a type of report that (re)explains the design with a certain level of justification from the designer.
Peer: I don’t always go sitting face forward down the slide. I also go on my back or belly or face backward. I don’t know if it’s nice to land on such a pillow if you do that. [Future description]
Child: Well, we think it is nice to land on such a pillow. Because it’s not really smart to go down the slide facing backward. That’s true. [Report justify]
Besides this, the DRQs of the peers were also met with resistance responses, especially band-aids and question not relevant.
Peer: Why does someone have to pull on the cable and is it not electric? [Expectational]
Child: We didn’t think an electric cable was needed. [Question not relevant]
Together the occurrence of report justify and resistance responses implies that the design teams were not as open to the DRQs posed by the peers as to the DRQs by the clients.
Opposite to this, there were a few instances of deep reasoning responses, in the form of don’t know yet, reflection, evaluation, and ideation. Although only a few of these DRs occurred, they do indicate that the DRQs have the potential to stimulate a higher level of reasoning. The examples below illustrate how DRQs can trigger reflection.
Peer: Imagine that the answer (to the math problem) is 64. You said that the side poles (of the goal) are 50 (points), and the top bar is a 100 (points). But there is not exactly 64 on any of them. [Future description]
Child 1: Well, you also have pons, and there are also numbers on there. [Report]
Child 2: Oh, yeah. Those (the pons) are not an uneven number. [Reflection]
Peer: How does it exactly work with the IV pole (on the slide)? [Procedural]
(team members whisper to each other)
Child: We don’t have a solution for that yet. [We don’t know yet]
Our data shows that these types of DRs predominately occurred when a design team already displayed a certain level of openness in the feedback dialogue preceding the DRQs.
Although the peers predominantly posed convergent feedback, instances of divergent feedback were also present. The high-level divergent feedback (GDQs) of the peers were varied in both cases, yet in case A proposals did take up a significant part of the DF. In both of the cases around half of the DRs to the GDQs consisted of resistance responses, with already in there being quite prominent. Through this response, the design teams dismiss that what is proposed by stating that it is already present within their design. Although a few other DRs were also present, the relatively high occurrence of resistance responses indicates that the design teams were not very open to the GDQs posed by the peers. Below are two examples that display how the design teams dismiss the GDQs of their peers.
Peer: Maybe it could be fun to do something with points? For example, who collects the points? Or maybe play in teams? [Proposal/negotiation]
Child: The player with the most points who wins, that’s it. [Already in there]
Peer: Maybe you can think a bit more about how it will work with the wheelchair? [Method generation] Because there is not always someone else present. [Future description]
Child: No, there is always someone else present. [Question not relevant]
The clients’ focus on verification and generation
Since the client of case A was not present during this final design session, we only discuss the DF posed by the clients of case B. Around half of the DF of the clients consisted of low-level convergent questions, focussing mainly on verification. This distribution of DF indicates that the clients’ focus lies predominantly on gaining a good understanding of the designs. This is to be expected, since the focus of the design session was revealing the final designs to the clients. Around half of the DRs to the low-level convergent questions consisted of confirming and a third of report. These responses indicate a neutral posture from the teams, with an apparent willingness to elaborate on the explanations of their designs.
Furthermore, around a third of the DF consisted of low-level convergent comments, mainly compliments. The many compliments could indicate a positive attitude from the clients towards the designs. These comments were mainly met with DRs in the form of nodding. Additionally, there were some instances of acknowledge and agree; and even some reflection responses. The DRs indicate that the reception of this DF by the design teams was rather positive, which could explain the level of openness of the design teams towards the rest of the DF.
There were only a few instances of high-level convergent feedback (DRQs) in the form of future descriptions. The rest of the DF was classified as high-level divergent feedback (GDQs), predominantly consisting of proposals. These proposals showcased how the clients focussed on possible future adaptations and elaborations of the designs. Around half of the GDQs of the clients were met with agreement type DRs in the form of nodding, acknowledge or agree. Additionally, a bit less than half of the GDQs co-occurred with deep reasoning responses, in the form of ideation, reflection, and don’t know yet. Together, these DRs of the design teams indicate openness towards the high-level divergent questions of the clients.
The key to openness: verification and compliments
Compared to the DRs towards the DF of the peers, the DF of the clients was met with more openness from the design teams. Below is an example of such a constructive feedback dialogue between one of the clients and a design team. The design that is discussed was rather unusual and received some critical feedback from the peers at an earlier point in the feedback dialogue. The team reacted to this DF of their peers with several resistance responses, trying to prove the quality of their design. However, the manner in which the team reacts to the DF of the client is very different.
Client: So the idea is that you will build an ice-rink? [Verification]
Child 1: Yes, I guess so. [Confirm] [Insecure]
Client: And this means that the ice-rink is there all year round? [Verification]
Child 1: Yes. [Confirm]
Child 2: It depends on if you want to ice-skate in the summer. [Reflection]
(…)
Client: So the ice-ring will be there permanently? [Verification]
Child 1: Yes. [Confirm]
Client: And what else could you maybe do with the ice-ring? [Ideation] If it’s a hot summer, will there then also be ice on it? [Feature specification]
Child 1: Maybe you can then also use it as a pool. [Ideation]
Client: Yes. And maybe you can make this thing float so they can swim with it. [Proposal/negotiation]
(team members nod)
Child 1: Yes.
The example illustrates how the client first puts their effort towards gaining a shared understanding of the aspect of the design they want to direct their high-level feedback to. After, the client proceeds to pose their high-level feedback, which then evokes a concurrent convergent or divergent response from the design team. As described earlier, the design teams react more positively and openly to the high-level feedback from clients than their peers. The manner in which the client poses the high-level feedback might have added to this openness.
Discussion and conclusion
The design feedback intervention was developed with the intention to guide constructive feedback dialogues and stimulate young novice designers’ creative thinking. In our previous studies on design feedback in an unguided setting, feedback of clients and peers was frequently met with resistance (Schut et al. 2019a, b). The results of the current study demonstrate how different types of guidance support young novice designers, their peers and clients in engaging in constructive feedback dialogues. Several successes were uncovered during the analysis, showing how the intervention was able to (1) guide towards a shared understanding about the design, (2) stimulate cognitive modelling, (3) encourage an initial calmness towards high-level feedback, and (4) evoke deep reasoning responses.
Realizing a shared understanding
The intervention aimed at establishing a shared understanding of the developed design at the start of a design feedback dialogue through low-level convergent feedback. In general, this led to a constructive start of the feedback dialogues, in which the design teams displayed openness towards the questions and comments of their peers and the clients by confirming certain understandings or providing additional information about their design. The problem of undiscussed expectations and assumptions about the designs, that we observed in unguided settings (Schut et al. 2019a, b) was therefore no longer as prevalent. Several studies confirm the importance of clear communication and a shared understanding about the design between all parties for a feedback dialogue to become valuable (Sadler 2010; Schut et al. 2019b; Stables et al. 2016a, b), which the intervention can successfully guide towards.
Stimulating cognitive modelling
In all the sessions—whether the feedback form was used or verbal prompts to adhere to the feedback structure—we observed the repeated occurrence of the high-level convergent feedback types future and future description, especially from peers. Since these types of feedback did not occur during our previous study on design feedback of young novice designers in an unguided setting (Schut et al. 2019b), we are rather confident that they can be attributed to the guiding sentences of the feedback form. Future and future description showcase the peers’ ability to engage in high-level convergent thoughts. By imagining how the designs would be used throughout time they construct mental models of the design mechanics and user interactions. This ability of cognitive modelling is argued to be the core of designerly thinking and fundamental to one’s creative abilities (Baynes 2010). The intervention supports young designers in practicing and expressing this type of modelling.
Initial calmness towards the feedback form
All feedback given in the 4th session highlighted where there was room for improvement and explained why this was thought of as a problem. Then this feedback was concluded with a divergent question, usually in the form of method generation or proposal/negotiation. The primary direct response of the design teams to this high-level feedback of their peers and the clients was silence, which shows a level of calmness. Although this does not directly give a clear insight into their thinking processes, a positive impact of the intervention is clear when we compare it to the feedback sessions in an unguided session, where high-level feedback was often met with resistance (Schut et al. 2019a, b). In the guided setting, the design teams appear to not immediately reject the feedback. In our future work we will focus our efforts on analysing the feedback uptake of the design teams when building their prototypes in the fifth design session.
Deep reasoning responses
On several occasions the design feedback of the clients and the peers was able to evoke deep reasoning responses from the design teams. Through these reflective, evaluative, and generative responses, the teams showcased their willingness to further elaborate on and improve their design. As our previous study on design feedback in an unguided setting indicates that these types of responses, especially reflection and evaluation, are not widespread (Schut et al. 2019b), the current feedback intervention is thought to encourage generative and evaluative thinking.
Unexpectedly, the high-level convergent feedback, especially that of the peers did not provoke the majority of the reflection and evaluation responses with the design teams. Instead, these responses were often preceded by high-level divergent feedback of the clients of case B during the 6th design session. Also, the other deep reasoning responses, ideation and don’t know yet, co-occurred often with this type of feedback. As described, verbal prompts were used during the 6th session as a form of guidance. With more freedom in constructing their high-level feedback, the clients deviated from the format of the feedback form, while still utilizing the core principles of the design feedback intervention. We speculate that these modifications contributed to the stimulation of these desired deep reasoning responses with the design teams. The following modifications were identified:
Firstly, the clients repeatedly checked the shared understanding of the designs. They did this primarily by posing several verification questions to the design teams right before sharing any high-level feedback. By raising these questions and getting responses, the clients’ assumptions and expectations of the designs are confirmed, and a shared understanding of the design arises. The difference with the feedback form on paper is that this form only guided towards creating a shared understanding of the design at the start of the feedback dialogue. Since clear communication and having a shared understanding is important in these feedback dialogues (Sadler 2010; Schut et al. 2019b; Stables et al. 2016a, b), the clients’ modification to check this understanding repeatedly throughout the feedback dialogue is an effective way to encourage a constructive dialogue.
Secondly, the clients almost solely constructed high-level divergent feedback, while only a few instances of high-level convergent feedback were observed. As described, divergent feedback plays a crucial role in helping the receiver of the feedback ‘move forward’ (Black and Wiliam 1998; Hattie and Timperley 2007) and stimulating their creative thinking (Daly and Yilmaz 2015; Tolbert and Daly 2013; Yilmaz and Daly 2016). We speculate that by focussing primarily on the additional opportunities and possibilities of the designs, combined with the clear shared understanding, the clients were able to create an open, positive atmosphere.
Lastly, another possible contributor to this open atmosphere is the many compliments the clients shared when compared to their peers. While sharing their feedback, the clients often added several positive remarks about the designs. Although the design feedback intervention does give guidance in constructing positive remarks before sharing high-level feedback, this is often shared separately and not in such a high volume. Although the opinions on praise as an effective form of feedback for learners appear to vary (Hattie and Timperley 2007; Voerman et al. 2014), our results show that in the context of design feedback dialogues it could encourage more openness with young novice designers. We speculate that all the modifications of the clients together contributed to the open atmosphere, which has been vital in encouraging the design teams’ deep reasoning responses.
Occurrence of resistance
Although constructive feedback dialogues were observed, the design teams also showcased instances of resistance towards the questions and comments of the clients and their peers. This is not entirely unexpected since novice designers of all educational levels have been observed to engage in responses indicating resistance, like parrying feedback and becoming defensive (Cardella et al. 2014; Cardoso et al. 2014; Cummings et al. 2015; Schut et al. 2019a, b). Additionally, the goal of the design feedback intervention was not to eliminate all resistance. It can be argued that some resistance can be seen as positive since it shows signs of persistence, a quality that is needed when engaging in creative processes (Crilly 2015; Csikszentmihalyi 1999). One of the key learning processes of novice designers is figuring out how to balance between persistence and openness. Since this is not an easy task, it is understandable that the young novice designers, on occasion, lost this balance.
Several factors might have evoked or enhanced the design teams’ resistance. As the data collection focused on the design feedback and the direct responses, it is not possible to discuss all possible factors that might have contributed to the occurrence of this resistance, such as the occurrence of concept fixation (Luo 2015; Nicholl and McLellan 2007a, b; Schut et al. 2019a), high levels of ownership (Baer and Brown 2012) or a lack of design skills (Blom and Bogaers 2018; Schut et al. 2019a, 2019). We can, however, discuss how certain design feedback might have influenced the occurrence of this resistance. Most instances of resistance from the design teams co-occurred with high-level convergent feedback of their peers during the 6th design session. The design teams thus lacked openness towards the high-level feedback of the peers. We presume that there might be several causes that have contributed to this resistance.
To start, the peers primarily shared high-level convergent feedback with the design teams during this session. Although convergence is needed when wanting to understand a design, an excessive focus is known to contribute to unconstructive dialogues (Cardella et al. 2014; Cardoso et al. 2014; Cummings et al. 2015; Schut et al. 2019b). As described before, the lack of divergent feedback can leave the young novice designers without a clear ‘way to move forward’ (Black and Wiliam 1998; Hattie and Timperley 2007) and can hamper their exploratory thinking and risk-taking processes (Daly and Yilmaz 2015; Tolbert and Daly 2013; Yilmaz and Daly 2016). The verbal prompts during the 6th session might not have sufficed in stimulating the peers in constructing high-level divergent feedback. Since the strict and specific guidance of feedback forms did showcase their ability to formulate this type of feedback, a different type of guidance during the 6th session might be more successful.
Furthermore, the literature suggests that the design teams could have perceived their peers, rightly or wrongly, as incompetent or lacking the expertise to give them feedback (Panadero 2016). This could explain why the design teams showcased more resistance towards the feedback of their peers and not towards the clients, who were introduced as experts by profession or experience. Although the peers might not be perceived as knowledgeable as the clients, they could take on the role of an extension of the design team. By highlighting the ability of the peers to help adjust and elaborate on the designs, thereby focussing on the common goal of improvement, the design teams might be more open to the suggestions of their peers. Besides this, the peers need to be guided in creating a more open atmosphere when sharing feedback. Possibly, the clients’ modifications can provide insight and inspiration for new ways of guiding peers in achieving more openness.
Additionally, the young novice designers might just not have agreed with some of the shortcomings that the peers uncovered. Their disagreement does, however, not necessarily justify their resistance. Reflective and evaluative thinking are thinking skills that are still in the process of developing with this age group. Studies show that engaging in convergent thoughts when designing does not always come easy to young novice designers (Blom and Bogaers 2018; Schut and Blom 2019). Although the construction of high-level convergent feedback does showcase their ability to engage high-level convergent thoughts, this does not mean they are able to reflect and evaluate their own designs critically. A study by Van Loon and Van de Pol (2019) suggests that elementary school children are more prone to detect errors in others’ work than in their won. Therefore, it might have been easier for the young novice designers to reflect and evaluate the designs of their classmates, instead of discovering shortcomings in their own work.
Updated design principles
To be effective in stimulating young novice designers creative thinking through design feedback dialogues, the feedback intervention was based on three main design principles: (1) guide towards a shared understanding of the design through low-level convergent feedback, (2) stimulate critical reflection and evaluation of the design to help identify and internalise possible shortcomings through high-level convergent feedback, and (3) provide a way to move forward by guiding new generative thoughts through high-level divergent feedback. Although the results indicate that the feedback intervention based on these design principles has shown to be effective in eliciting responses indicating divergent and convergent thinking, opportunities for improvement were also uncovered.
The first guiding principle, (1) guiding towards a shared understanding, was implemented at the start of a feedback dialogue by directing the clients and peers towards low-level convergent feedback. Generally, this led to a constructive start of the feedback dialogues and could, therefore, be considered as a successful design principle. However, the constructive feedback dialogues between the clients and the design teams during the 6th design session, and the modifications to this guiding principle made by clients, could still provide inspiration for improvement. One of the modifications was the constant checking of the feedback givers of their shared understanding right before posing high-level feedback. We speculate that due to this clear understanding, the design teams feel that their design is well understood and feel acknowledged in their decisions. This might improve openness and reduce resistance towards the following high-level feedback, leading to more openness. Therefore, we propose that the first guiding principle should emphasise the need to continuously work towards a shared understanding throughout the dialogue, especially before posing high-level feedback.
The second and third guiding principle were meant to directly guide the young novice designers towards (2) reflective, evaluative and (3) generative thinking processes by means of high-level convergent and divergent feedback. However, the results showcase that, similar to previous studies (Schut et al. 2019a, b; Cardella et al. 2014; Cardoso et al. 2014; Cummings et al. 2015), the divergent or convergent nature of the feedback does not necessarily elicit the expected concurrent creative thinking processes. It could be argued that we need to let go of the idea that convergent or divergent feedback would direct the novice designers exactly into that concurrent thinking process. The intention of the feedback giver, either pushing towards convergence or divergence, could therefore be attained by either type of high-level feedback. Hence, there might be more merit in consciously and carefully alternating and iterating both types of feedback until the set intention is reached.
However, we do argue that more focus needs to directed to the use of high-level divergent feedback within design feedback dialogues. The results show that the peers engage little in this type of feedback, creating an imbalance which could have contributed to the resistance of the design teams. Opposite to that, the clients engaged almost primarily in high-level divergent feedback during the 6th session. The dialogues that followed this feedback of the clients were observed to be generally constructive. We presume that this repeated use of high-level divergent feedback contributed to a positive and open atmosphere and directed the designers’ attention towards ‘moving forward’ and improving their design. Additionally, we want to put specific emphasis on the many concrete compliments shared by the clients, which we believe to be another main factor contributing to the openness that was present in these specific dialogues. This approach of the client may have clarified to pupils that, besides judging the current state of a design, feedback is also meant to encourage current iteration rounds.
Based on our findings, we propose a set of revised principles for constructive design feedback dialogues among young novice designers, which stimulates their creative thinking: (1) continuously guide towards a shared understanding through low-level questions, especially before posing high-level feedback, (2) emphasise positive aspects of the design through specific and concrete compliments, (3) stimulate reflective, evaluative and generative thoughts through alternating and iterating high-level divergent and convergent feedback, and (4) repeatedly provide a way forward, thereby fostering improvement, by posing high-level divergent feedback.
Future work
Our results show that the developed and implemented design feedback intervention can guide young learners and clients in sharing low and high-level, as well as, convergent and divergent feedback. Furthermore, it was observed how this design feedback and the concurrent direct responses from the young novice design teams were able to develop into constructive feedback dialogues. Although the current feedback intervention led to several success, we believe the updated design principles offer opportunities for optimizing these conversations even further. With this study we hope to inform researchers and design educators on the complexity of design feedback dialogues and perhaps inspire new design feedback interventions based on the described design principles.
Future research on these thinking processes in the context of feedback dialogues between, and with, young novice designers could provide valuable insights on the development of their creative thinking skills. Therefore, we will expand the current results by analysing additional data to uncover how the design teams selected, discussed and utilized the feedback while elaborating and adjusting their design. Together, these studies will provide an in-dept view of the influence design feedback can have on young novice designers’ creative thinking.
Acknowledgements
The NWO/NRO project Co-design with Kids: Early Mastering of 21st Century Skills is funded by NWO/NRO under the HC21 call (number 409-15-212).
Compliance with ethical standards
Ethical standard
Ethical clearance from the Ethics Committee (HREC, TU Delft) has been obtained.
Informed consent
Informed participation consent was obtained from the parents. We thank the participating school staff, pupils and the clients.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
