Innovative Approach to Assist Architecture Teachers in Choosing Practical Sessions

Abstract

This article presents the first results of the project Architecture 360, which focuses on learning alternatives for developing working skills in higher education courses, and specifically construction competences for architecture students. The project aims to help teachers to choose the best learning solutions for their classes from numerous alternatives of strategies, dynamics and activities. The assistance is based on developing a new approach that combines several methods (strengths, weaknesses, opportunities and threats (SWOT); multi-criteria decision-making; Delphi; and the Knapsack problem) and draws from teachers’ experience, a panel of experts’ expertise, the revised Bloom Taxonomy and neuroscience for education. The new approach to assisting university teachers in choosing the best practical learning alternatives was successfully developed and validated for the case study of a course at Barcelona Architecture School. In general, the approach defined the main strengths, weaknesses, opportunities and threats of 26 learning alternatives. In the case study, the following optimized set of alternatives were identified: blended learning, challenge-based learning, reflective learning, videos of real cases, case studies, site visits, interactive simulation and gamification. Moreover, 23 activities were analysed. It was concluded, for instance, that active alternatives would improve implementation, including teachers’ available teaching materials and dedication outside class.

1. Introduction

Universities have many important roles in our society [1], such as providing a proper higher education learning environment for millions of future professionals each year [2]. However, some studies indicate that higher education studies cannot always provide graduates with skills and knowledge that meet employers’ expectations [3]. To achieve these expectations, universities and faculties follow diverse strategies [4]. For example, many institutions incorporate work-integrated learning (WIL; Appendix A presents a complete list of abbreviations) activities [5]. WIL alternatives include work placements and internship programmes [6].

Apart from these strategies, regular courses during university degrees aim to cover certain practical or professional competences of their students to promote readiness to practice [7]. These courses work on a specific part of the practical discipline. This part is learnt in depth, including theoretical and practical aspects, so that students gain related working skills. Numerous experiences are carried out, from site visits [8] to virtual reality applications [9]. These alternatives are not used excessively and the same course can combine several of them [10]. Nevertheless, COVID-19 pandemic prevention and lockdown measures increased the need for blended learning [11] and e-learning [12]. All these experiences and alternatives have singular, specific characteristics. For instance, in terms of experience, some are well known and applied, while others are new and under implementation, like face-to-face and virtual laboratories [13]. They also vary in the dedication required of students and teachers, and in students’ engagement.

Choosing the most suitable practical alternative or set of experiences for a specific course is a crucial multi-criteria decision-making process that should consider the characteristics of the alternatives and many other factors. This article divides the factors into stakeholders and contextual aspects. Numerous stakeholders are involved in higher education [14]. However, students and teachers are the main stakeholders in the learning process, which is the focus of this research paper. In addition, various contextual factors [15] are considered crucial in this project. These are the definition of courses (the objectives, competences, contents and assessment) and aspects of the institutions (budget, spaces, resources, programmes and industry). For example, the level of collaborative complicity between professionals and the university is fundamental, as are the laboratories available for each degree. Among other factors, students’ learning processes and cognition level [16] and aspects of educational neuroscience [17] should also be considered.

This article presents a new approach that aims to assist teachers in choosing the best set of activities for the practical sessions of a specific course. The approach considers the characteristics of the learning alternatives, stakeholders and contextual factors. It draws on a previous review of related technical literature [18], based on which the authors define this new approach and its six main steps following Delphi, expert seminars and focus groups. This article presents the first version of the approach, which focuses on practical learning in architecture schools. The steps include a multi-criteria decision-making methodology called an integrated value model for sustainability assessment (MIVES) and a Knapsack algorithm, which is based on a similar approach by the authors that successfully helped teachers to choose the most suitable active learning activities in lectures for large groups [19]. Thus, the main difference in the present approach is its focus on practical sessions, and there is a general improvement based on the results of the previous approach’s implementation. To validate this new approach, the authors applied it for the first time to a specific course at the Universitat Politècnica de Catalunya (UPC). The next section describes potential learning alternatives. Then, Section 3 presents the new approach, Section 4 identifies the problem, and Section 5, Section 6 and Section 7 are the results, discussion and conclusions of applying this approach for the first time.

2. Alternatives Analysis

To identify the main experiences used for university students to learn specific work-related competences, a literature review was carried out in July 2021 [18]. This review analysed the number of related publications in the Web of Science (WoS) Core Collection database [20]. It considered 64 results from a search on university active learning activities for working and practical learning, and 86 results from another search focused on WIL activities. Studies that were found during other steps of this research project were also considered. Other experiences that have not been reviewed or published yet are expected to be considered in future research phases. The review followed a rigorous methodology [21], to include the maximum number of experiences. Publications were analysed from the perspective of general to detailed issues, and factors from the publication date to learning alternatives were considered. Depending on the approach to the learning scale, the review considered three types of alternatives: (t1) online strategies, (t2) learning dynamics and (t3) activities during class or outside the university. At the same time, considering the learning methodologies, the review classified three main interrelated groups of alternatives: (g1) recent digital technologies, (g2) active learning and practical activities, and (g3) real experiences.

Table A2. Types and groups of practical session alternatives.

	(t1) Online Strategies	(t2) Learning Dynamics	(t3) Activities
(g1) Recent digital technologies	Blended Learning [48] e-learning [49]	TEAL [50]	Blended and e-learning activities [51] Interactive simulations [52] Social media activities [53] Videos of real cases [54] Virtual learning activities [55]
(g2) Active learning		CBL [56] TBL [57] Flipped classrooms [58] PBL [59] Reflective learning [60]	Case studies [61] Discussions [62] Gamification activities [63] Interdisciplinary activities [64] Problem-solving activities [65] Storytelling [66]
(g3) Practical and real experiences		Industry–community projects [67] Internships [68] Placements [69] Dual VET [70]	Real material practices [71] Hands-on activities [72] Role play [73] Site visits [74]

Legend: technology-enabled active learning (TEAL), challenge-based learning (CBL), project-based learning (PBL), team-based learning (TBL), vocational education and training (VET).

in Appendix B presents these alternatives.

The first type, t1, includes blended and e-learning alternatives, which represent two different intensities in the use of online resources within g1. The first combines online and face-to-face learning, while the second exclusively uses online resources and communication. The second type, t2, mainly encompasses active learning and practical experiences. Active learning includes challenge-based learning (CBL) and team-based learning (TBL), like modified case-based learning exercises called active learning groups [22]. Practical experiences include degree apprenticeships [23], placements and dual vocational education and training (VET) [24]. The third type, t3, has activities within all the groups of alternatives. The first are activities based on recent digital technologies. They may involve experiences that are part of blended and e-learning (e.g., practical activities on the web, such as practical active learning stations) [25] or virtual learning activities (such as virtual laboratories and virtual learning explorative activities) [13]. Other active learning activities include case studies and storytelling experiences. Finally, real experiences are carried out, such as active practices with real material [26], role play or work activity simulation [27], and onsite visits to observe professionals in the workplace [28].

The outcomes of the WIL experiences that were highlighted by the studies were generally the contribution of these experiences to personal and professional growth. During WIL, the co-presence of industry members and teachers is essential. Other valued points were, from major to minor importance: the work–study–life balance, industry involvement and support for WIL activities, cases of parallel rather than integrated learning in which university and industry synergies are insufficient, equity among students’ opportunities, cultural dissonance, for instance between students and the placement environment, competences, technology integration and employability.

3. Methodology

To find the best methodology to help teachers to choose and organize their practical sessions, the authors relied on a review of the assessment of learning activities (Lr2), in which up to 201 publications were eligible and were analysed. Lr2 identified similar previous studies. However, they were limited to fewer indicators and alternatives and used methodologies that were more appropriate to these limits. The methods used in these studies were diverse: surveying and interviewing (30 publications), quasi-experimental design (25), statistical analysis (22), qualitative analysis (22), quantitative analysis (12), case studies (9) and other methods in the rest of the publications. The closest methods to this article’s new methodological approach were the development of frameworks in some studies [29,30,31]. This research project follows the six-stage methodology presented in Figure 1, which relies on a similar former project by the authors on the subject of lectures [19].

This approach was chosen because, based on its previous application, it can give useful advice to teachers about which alternatives or sets of alternatives are most suitable for their practical sessions. The approach can include the main indicators, adapted to the characteristics of any case study, assess any type and number of alternatives, and provide integrated quantitative results.

The following subsections explain these six phases: in (P1) and (P2), the teaching team classifies the professional contents and practical alternatives respectively; in (P3), the MIVES–Delphi tool is used to assess these alternatives; in (P4), the teaching team estimates the available and required sessions; in (P5), the Knapsack algorithm suggests sets of alternatives for the practical sessions; and in (P6), the teaching team analyses these sets and reaches a final decision.

3.1. Phases 1 and 2

During these first phases, the teaching team for the course follows Bloom’s Taxonomy, revised by Anderson [32], to classify the practical course contents and feasible alternatives. The alternatives are also studied according to the strengths, weaknesses, opportunities and threats (SWOT) technique [33]. The resulting matrix depicts the main characteristics of the alternatives. SWOT results are based on an extensive literature review [18] and contain strengths and weaknesses that are more commonly related to internal issues, and opportunities and threats that are focused more on external factors [34]. This taxonomy considers three thinking levels: (a) lower-order thinking level (LOTL), such as remembering and recounting concepts; (b) middle-order thinking level (MOTL), such as applying and understanding ideas; and (c) higher-order thinking level (HOTL), such as analysing, evaluating and creating your own proposals. These practical contents are the course-related working skills that students are expected to acquire, as explained in the introduction to this article. The feasible alternatives are practical alternatives from the review in the previous section and other sources that the teachers consider applicable to their course context.

3.2. Phase 3

This third phase uses the integrated value model for sustainability assessment (MIVES) because a multi-criteria decision-making method was needed to assist teachers in taking multi-criteria decisions that have multiple indicators with different values and tendencies. Among the available multi-criteria decision-making methods [35], MIVES was chosen because it is a consolidated method that has been successfully applied since the 2010s [36]. It incorporates value functions that allow integrated assessments of various indicators with different units and tendencies. It provides a global sustainability index and partial sustainability indexes. It enables agile sustainability assessments for specific case studies and boundaries. Finally, it can be combined with robust weighting methods such as Delphi. Thus, Phase 3 defines the assessment limits, considering contextual factors, among others. Then, it builds a requirements tree (RT) based on related technical literature. It weights requirements, criteria and indicators following Delphi. Next, it defines the indicators’ value functions and finally it assesses the alternatives.

This assessment system’s limits are higher education alternatives that cover certain professional competences to promote students’ readiness to practice, as explained in the previous sections.

Table 1. Decision-making requirement tree with weights as a percentage.

Requirements	Criteria	Indicators
R1. Applicability (26%)	C1. Application (58%)	I01. Ease of application (53%)
		I02. Flexibility for adaptation (47%)
	C2. Transferability (42%)	I03. To other teachers (57%)
		I04. To other disciplines (43%)
R2. Economic (21%)	C3. Cost (38%)	I05. Direct costs (45%)
		I06. Logistic and scheduling issues (55%)
	C4. Time (62%)	I07. Dedication in class (39%)
		I08. Teachers’ dedication outside (28%)
		I09. Students’ dedication outside (33%)
R3. Environmental (11%)	C5. Impact (100%)	I10. Extra-environmental impact (100%)
R4. Social (42%)	C6. Learning process (Chickering and Gamson principles, among others) (35%)	I11. Roles, talents and ways of learning (23%)
		I12. Encouraging cooperative work (28%)
		I13. Autonomous work (30%)
		I14. Students’ cognitive load (19%)
	C7. Interaction (Chickering and Gamson principles, among others) (40%)	I15. Students’ interest and participation (33%)
		I16. Students and faculty contact (22%)
		I17. Feedback to students’ time (19%)
		I18. Learning outcomes (cognition and affect) (26%)
	C8. Innovation (25%)	I19. University learning (53%)
		I20. Teachers’ new functions (47%)

presents the requirements tree with its indicators, criteria and requirements based on: (1) the aforementioned MIVES for large groups [37], (2) the Lr2 state of the art, and (3) the characteristics of practical sessions, drawn from the authors’ experience [10] and the literature review [19]. Requirements tree components were defined to obtain a limited number of discriminative indicators, as required by MIVES [38]. The definition process involved including (a) three indicators from Lr2 and (b) three indicators that are crucial for practical alternatives, and (c) adapting and complementing two former indicators, so that they do not overlap with the former indicators. These are presented and explained in

Table A3. List of new and adapted indicators for this RT, aspects assessed and grouping process.

New Indicator	Aspects Assessed by This Indicator	Grouping Process
(a1) Autonomous work	Capacity to promote students’ ability to learn and act by themselves, such as entrustable professional activities [75].	Lr2 determined that these indicators were necessary to assess practical alternatives in 13, 21 and 94 studies, respectively.
(a2) Students’ interest and participation	Ability to promote engagement, interaction, participation and attendance. Includes having fun learning, satisfaction and high expectations [76].
(a3) Learning outcomes diversity	The capacity to evaluate results was related to a) cognition: course-related content knowledge, programming knowledge, skills and competence, creativity; and b) affect: confidence, attitude, feeling and perceptions. Ability to perform summative and formative assessments [77].
(b1) Direct costs	A simplified cost analysis regarding the extra costs of each active learning alternative [29,78]. This takes into account the learning materials, resources, transport, insurance, etc., following legal and ethical requirements. It classifies the evaluated alternatives according to six groups of extra cost affordability: (1) no cost; (2) from the course budget; (3) from private foundations’ funds; (4) from the university’s competitive funds; (5) from national public competitive funds; (6) from international competitive funds.	These indicators are crucial for practical sessions because many alternatives involve more costly, complex resources and management, such as virtual learning activities [55] and site visits [74]. Similarly, numerous activities imply that teachers assume new competences, from interactive simulations [58] to gamification [79].
(b2) Logistic and scheduling issues	Analysis of the extra operational processes required by each alternative, including those involving management of resources, time and space, such as required learning spaces and different course scheduling compatibility [80]. It focuses on these requirements without considering whether the alternatives are flexible because it was already included in a former indicator.
(b3) Teachers’ new functions	Number of new roles in teachers’ work that the alternatives require per semester [81].
(c1) Roles, talents and ways of learning	Ability to allow different roles, talents and ways of learning, styles, approaches, learning and pacing and presentation methods, cultures, recognition of reward and respect for creativity. This also includes students’ abilities, learning level, leadership, collaboration, initiative, attitude, effort, research, communication and a written report, as well as individual tasks [82].	These indicators were adapted and complemented following Lr2.
(c2) Students’ cognitive load	The extent to which the cognitive load theory (CLT) is followed, according to the information processing model [83] and methods to manage cognitive load [84].

in Appendix B. This process also discarded three indicators that were used in the predecessor. One of these indicators was excluded because it focuses on innovation in lectures. The other two, which are related to stakeholders’ satisfaction, were not considered in this study as sole indicators because the new indicators b2, b3, c1 and c2 already include them. The definition process included three new criteria (C3, C6 and C7), following the reasons explained previously for the indicators. The requirements are the three main sustainability pillars (economic, environmental and social), plus the applicability requirement from the expert seminars.

This project followed the Delphi-based approach explained by Casanovas-Rubio and Armengou [39] to define requirements tree weights, choose the experts who participated in assigning the weights and manage the related surveying process. First, the authors used Delphi to select and ask the participation of 22 qualified experts, the expertise and main data of which is summarized in

Table A4. Delphi panel of experts’ main information.

N	G	Position	Research Field	N	G	Position	Research Field
1	F	Lecturer and consultor	Building structures	11	M	Associate professor	Light in architecture
2	M		Construction	12	F		Energy in architecture
3	M			13	M	Lecturer and consultor	Building facades
4	F	Associate professor	Rehabilitation and restoration	14	F		Energy in architecture
5	F			15	M		Simulation tools
6	F	Lecturer and consultor		16	M	Associate professor	Building structures
7	F			17	F	Lecturer and consultor	Acoustics
8	M	Associate professor	Building facades	18	M	Associate professor	Management
9	M			19	F	Lecturer and consultor	Materials
10	M	Lecturer and consultor	Construction	20	M		Construction

Legend: N—number; G—gender; F—female; M—male.

in Appendix C. Twenty of them completed the surveying procedure and proposed weights by the direct assignment method. The number of experts was higher than the recommended minimum number of panellists for this approach [39]. Consensus of the experts was reached when the median absolute deviation was below one-tenth of the possible values range, which was 10% because the range of weights was from 0% to 100%. The project needed two rounds of surveys to reach this consensus. The first round invited experts to propose weights for the indicators, criteria and requirements of this project requirements tree, according to their judgement. The only condition was that the total weight of the sum of requirements and each group of criteria and indicators must equal 100%. The second round invited experts to consider adjusting their own first-round weights, taking into account the mean values of the first round and keeping their weights within their preferences. Experts were asked to justify any weights that deviated more than 10% from the mean of the weights in the first-round survey. To reduce judgement-based bias, the two rounds of the surveys had randomized question order, iteration and anonymity.

Table A5. Delphi approach results from the first-round survey.

DT Elements	Weights Assigned by Panellist (%)																				Mean	Median	Median Absolute Deviation (%)	Consensus
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20
R1	30	20	20	30	30	20	15	20	60	40	25	20	20	25	30	20	30	20	30	20	26.3	22.5	7	Yes
R2	10	25	20	30	5	20	15	10	10	10	40	30	20	25	40	30	20	20	10	20	20.5	20	8	Yes
R3	10	15	20	10	0	10	35	35	10	10	0	0	0	10	0	10	10	20	10	10	11.3	10	6	Yes
R4	50	40	40	30	65	50	35	35	20	40	35	50	60	40	30	40	40	40	50	50	42.0	40	8	Yes
C1	70	30	40	60	50	60	40	30	80	75	80	60	60	65	70	65	70	60	70	60	59.8	60	11	No
C2	30	70	60	40	50	40	60	70	20	25	20	40	40	35	30	35	30	40	30	40	40.3	40	11	No
C3	40	60	40	40	25	40	30	50	25	50	60	40	30	40	20	40	30	30	30	20	37.0	40	9	Yes
C4	60	40	60	60	75	60	70	50	75	50	40	60	70	60	80	60	70	70	70	80	63.0	60	9	Yes
C5	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100.0	100	0	Yes
C6	35	20	50	20	40	30	25	30	70	40	20	50	50	40	30	40	30	30	30	40	36.0	32.5	10	Yes
C7	35	45	25	60	40	50	50	40	10	40	40	20	35	30	40	40	50	40	50	40	39.0	40	8	Yes
C8	30	35	25	20	20	20	25	30	20	20	40	30	15	30	30	30	20	30	20	20	25.5	25	6	Yes
I01	80	50	40	60	50	50	60	80	65	50	60	40	40	55	30	50	50	50	60	40	53.0	50	9	Yes
I02	20	50	60	40	50	50	40	20	35	50	40	60	60	45	70	50	50	50	40	60	47.0	50	9	Yes
I03	80	30	60	50	50	50	40	50	75	50	50	50	75	65	70	60	50	80	60	60	57.8	55	11	No
I04	20	70	40	50	50	50	60	50	25	50	50	50	25	35	30	40	50	20	40	40	42.3	45	11	No
I05	60	70	20	30	20	50	75	10	30	30	60	50	40	50	70	60	40	70	30	20	44.3	45	17	No
I06	40	30	80	70	80	50	25	90	70	70	40	50	60	50	30	40	60	30	70	80	55.8	55	17	No
I07	50	50	70	40	40	60	20	45	35	25	40	20	40	35	15	30	30	50	50	20	38.3	40	11	No
I08	20	25	20	20	30	20	40	10	32	50	30	50	30	40	25	30	40	25	20	40	29.9	30	8	Yes
I09	30	25	10	40	30	20	40	45	33	25	30	30	30	40	60	40	30	25	30	40	32.7	30	7	Yes
I10	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100.0	100	0	Yes
I11	15	20	20	20	25	30	30	30	10	25	20	30	25	40	30	20	20	20	20	20	23.5	20	5	Yes
I12	25	45	30	30	25	20	30	20	35	25	25	25	25	25	20	30	30	20	35	40	28.0	25	5	Yes
I13	35	30	30	30	25	30	20	30	40	25	35	25	25	25	30	20	30	30	35	30	29.0	30	4	Yes
I14	25	10	20	20	25	20	20	20	15	25	20	20	25	10	20	30	20	30	10	10	19.8	20	4	Yes
I15	60	40	50	30	40	30	25	20	25	35	30	25	40	30	30	25	20	25	25	20	31.3	30	7	Yes
I16	15	30	20	30	20	20	25	30	10	15	30	20	10	25	15	30	15	20	25	30	21.8	20	6	Yes
I17	10	20	20	20	20	20	15	20	10	15	20	20	25	25	15	20	15	20	25	40	19.8	20	4	Yes
I18	15	10	10	20	20	30	35	30	65	35	20	35	25	20	40	25	50	35	25	10	27.8	25	10	No
I19	65	40	50	50	50	50	60	50	60	40	70	70	60	40	40	60	50	70	30	40	52.3	50	9	Yes
I20	35	60	50	50	50	50	40	50	40	60	30	30	40	60	60	40	50	30	70	60	47.8	50	9	Yes

and

Table A6. Delphi approach results from the second-round survey.

DT Elements	Weights Assigned by Panellist (%)																				Mean	Median	Median Absolute Deviation (%)	Consensus
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20
R1	27	20	30	30	30	25	20	20	45	30	25	25	25	25	30	25	30	22	28	20	26.6	25	4	Yes
R2	18	25	20	30	13	19	20	10	15	20	30	25	20	25	30	25	20	20	16	20	21.1	20	4	Yes
R3	10	20	10	10	5	10	20	30	10	10	10	0	0	10	0	10	10	18	10	10	10.7	10	4	Yes
R4	45	35	40	30	52	46	40	40	30	40	35	50	55	40	40	40	40	40	46	50	41.7	40	5	Yes
C1	70	30	45	60	55	60	50	40	65	65	65	60	60	65	65	60	60	60	65	60	58.0	60	6	Yes
C2	30	70	55	40	45	40	50	60	35	35	35	40	40	35	35	40	30	40	35	40	41.5	40	7	Yes
C3	40	60	40	40	27	40	30	50	30	40	50	40	30	40	30	40	35	32	33	30	37.9	40	6	Yes
C4	60	40	60	60	73	60	70	50	70	60	50	60	70	60	70	60	65	68	67	70	62.2	60	6	Yes
C5	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100.0	100	0	Yes
C6	35	25	40	30	38	35	30	30	45	35	30	40	45	37	30	35	40	33	33	40	35.3	35	4	Yes
C7	40	50	30	50	40	42.5	45	40	35	40	40	30	35	33	40	40	40	40	45	40	39.8	40	3	Yes
C8	25	25	30	20	22	22.5	25	30	20	25	30	30	20	30	30	25	20	27	22	20	24.9	25	3	Yes
I01	70	50	40	55	50	50	55	70	60	50	60	50	45	55	40	55	50	50	58	50	53.2	50	6	Yes
I02	30	50	60	45	50	50	45	30	40	50	40	50	55	45	60	45	50	50	42	50	46.9	50	6	Yes
I03	60	30	60	50	55	55	50	60	65	55	55	50	65	65	65	60	50	70	60	60	57.0	60	6	Yes
I04	40	70	40	50	45	45	50	40	35	45	45	50	35	35	35	40	50	30	40	40	43.0	40	6	Yes
I05	50	65	30	40	34.25	45	53	30	35	40	55	45	40	50	60	50	40	50	40	40	44.6	42.5	8	Yes
I06	50	35	70	60	65.75	55	47	70	65	60	45	55	60	50	40	50	60	50	60	60	55.4	57.5	8	Yes
I07	50	55	50	40	40	40	30	40	35	35	40	35	40	35	30	35	30	45	40	30	38.8	40	5	Yes
I08	20	20	30	20	30	30	35	20	32	35	30	35	30	25	30	30	30	28	25	30	28.3	30	3	Yes
I09	30	30	20	40	30	30	35	40	33	30	30	30	30	40	40	35	40	27	35	40	33.3	31.5	5	Yes
I10	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100	100.0	100	0	Yes
I11	20	20	20	20	25	25	25	30	15	23	20	25	25	30	30	25	20	20	20	20	22.9	21.5	3	Yes
I12	25	30	30	30	25	25	30	20	35	27	25	25	25	27	20	30	30	22	32	40	27.7	27	4	Yes
I13	35	30	30	30	30	30	25	30	35	27	35	30	25	27	30	25	30	30	32	30	29.8	30	2	Yes
I14	20	20	20	20	20	20	20	20	15	23	20	20	25	16	20	20	20	28	16	10	19.7	20	2	Yes
I15	55	30	40	30	35	30	30	30	30	30	30	30	40	30	30	30	35	27	28	40	33.0	30	4	Yes
I16	20	30	20	30	20	20	25	20	15	20	30	20	10	25	20	25	15	20	24	20	21.5	20	3	Yes
I17	10	20	20	20	20	20	15	20	15	20	20	20	25	25	20	20	15	20	24	20	19.5	20	2	Yes
I18	15	20	20	20	25	30	25	30	40	30	20	30	25	20	30	25	35	33	24	20	25.9	25	5	Yes
I19	65	50	50	50	50	50	55	50	60	50	65	60	60	46	50	55	50	55	45	50	53.3	50	4	Yes
I20	35	50	50	50	50	50	45	50	40	50	35	40	40	54	50	45	50	45	55	50	46.7	50	4	Yes

in Appendix C show the results of these two rounds.

Then, the project defined the assessment of each indicator considering the previous MIVES for lectures and related technical literature. The result was value functions for each indicator. The values ranged from 0 to 1, which represented maximum and minimum satisfaction, respectively, for the 20 indicators’ values. The addition of each set of indicators’ adimensional values (V_i,k) resulted in eight criteria satisfaction values (V_CRi,k), each set of which could be added to obtain four requirement satisfaction values (S_Ri,k). Finally, the addition of the requirement satisfaction values resulted in the global sustainability index (GSI_k). The calculations follow Equations (1)–(3).

$$V_{CRi,k}=\underset{i=1}{\overset{j}{\Sigma }}{\lambda }_{i,k}·V_{i,k}(x_{ind})$$

(1)

$$S_{Ri,k}=\underset{i=1}{\overset{j}{\Sigma }}{\lambda }_{CRi,k}·V_{CRi,k}$$

(2)

$$GS{I}_k=\underset{i=1}{\overset{j}{\Sigma }}{\lambda }_{Ri,k}·S{I}_{Ri,k}$$

(3)

The definition of the value functions depends on the shape that each indicator assessment requires. Most functions are defined by five parameters, as shown in Equation (4). These parameters determine the shape of the function and how each indicator variation is transformed to the 0 to 1 scale.

$$V_{ind}=B\cdot \left[1-e^{-ki\cdot {\left(\frac{\left|Xalt-Xmax\right|}{Ci}\right)}^{Pi}}\right]$$

(4)

These five parameters are as follows: (1) Xalt is the abscissa for each assessed alternative indicator that generates a V_ind value; (2) Pi is a shape factor that determines the curve shape, such as concave, convex, lineal and “S” shaped; (3) ki defines the value for the ordinate point Ci (4); and (5) B is the factor that is capable of maintaining the function within the adimensional range, which Equation (5) depicts.

$$B={\left[1-e^{-ki\cdot {\left(\frac{\left|X\max -X\min \right|}{Ci}\right)}^{Pi}}\right]}^{-1}$$

(5)

The functions for indicators I13 and I20 are different because they have an increasing tendency of satisfaction up to a maximum value, from which they decrease back to zero. In consequence, these functions follow the quadratic Equation (6).

$$Y=a{X}^2+bX+c$$

(6)

Table 2. Definition of the indicators’ assessment.

	Assessment Parameters
I01	Work required before, during and/or after class, i.e., prepare and handle material, correct, give feedback
I02	Adaptability to different space, time and resource characteristics, i.e., spaces, types or number of sessions
I03	Available technical literature on each alternative, its relation to the case study and ease of use
I04	Number and interdisciplinarity of the related 6-digit UNESCO nomenclature on areas of expertise
I05	Costs per student that must be covered to complete each activity and comply with legal and ethical issues
I06	Number and complexity of logistic and scheduling arrangements required for each alternative
I07	Average time dedicated per session during class, by teachers outside of class, or by students after class, respectively
I08
I09
I10	The energy consumption and waste generation that each alternative involves
I11	The number and diversity of roles, styles, approaches and methods that each alternative allows
I12	The extent to which each alternative incorporates, allows and promotes teamwork and dynamics
I13	The extent to which each alternative enables and promotes students’ autonomous learning processes
I14	Cognitive load theory (CLT), according to the information processing model and methods, to manage the cognitive load
I15	Students’ satisfaction with engagement, interaction, participation, attendance, diversion and expectations
I16	Promotion of contact between students and faculty and a sense of belonging to the institution
I17	Average feedback time from teachers on students’ tasks and activities
I18	Average quantitative and qualitative results from grades and commentaries
I19	Previous university research projects and technical literature about each alternative
I20	Number and complexity of new concepts and skills required by teachers for each activity

presents the definition of the indicators’ assessment and

Table 3. Indicator units and function shapes.

	Unit	Equations	Shape	Xmin	Xmax	C	K	P	a	b	c
I05	Points	(4) and (5)	DL	0	100	50	0.05	1	-	-	-
I06	Points	(4) and (5)	DL	0	100	50	0.05	1	-	-	-
I13	Points	(4) and (5)	IL	0	100	50	0.05	1	-	-	-
I14	Points	(4) and (5)	IL	0	100	50	0.05	1	-	-	-
I15	Points	(4) and (5)	IL	0	100	50	0.05	1	-	-	-
I18	Points	(4) and (5)	IL	0	100	50	0.05	1	-	-	-
I20	Points	6	Pb	-	-	-	-	-	−0.04	0.4	0

the units and function parameters of this study of new indicators. Most indicators maintain the units and function shapes assigned in the lecture-based MIVES. Some indicators have changed their code: indicators I07, I08, I09, I10, I11, I12, I16, I17 and I19 correspond to the previous study’s [19] indicators I05, I06, I07, I08, I12, I10, I11, I09 and I14, respectively, and their respective value functions. The six new indicators I05, I06, I13, I14, I15 and I18 have linear functions for this first application of the new approach, while other shapes that are more sensitive to each indicator’s tendency will be considered in future applications. The function for indicator I20 has a quadratic equation that: (a) crosses the origin at X = Y = 0; (b) increases up to the vertex at Y = 1; and (c) decreases symmetrically until it crosses the X axis at X = 1.

3.3. Phase 4

The teaching team estimates the available practical sessions for the studied course and the minimum number of sessions for each learning alternative, relying on their experience and the previous literature reviews.

3.4. Phase 5

The Knapsack algorithm generates proposals of sets of activities based on the estimation from previous sessions. This algorithm is used because it provided similar sets in the previous methodology for large lectures [37] and it was successfully combined with MIVES in several former research projects [40]. Knapsack maximizes some values according to each study measure to obtain one or more sets that have values equal or less than the established measures and have the maximum satisfaction for the main defined value. In this study, the best set of alternatives was chosen for the practical sessions in a full semester. Thus, the two parameters of the Knapsack problem are that the value is the GSI and the weight is the number of sessions these activities need to be implemented.

3.5. Phase 6

In this last phase, the teaching team chooses the best set of alternatives, taking into consideration the Knapsack results and their experience, expertise, the technical literature, Bloom’s Taxonomy revised by Anderson, and neuroscience in education [41]. The Knapsack results include the MIVES–-Delphi assessment (Section 3.2, Phase 3), which is based on feasible alternatives from the SWOT analysis. The teachers’ final decisions consider the specific needs and context of each course, group and students. They check the automatic results and, for instance, make sure that the thinking level of the course contents is compatible with the activities (Phase 1).

4. Identification of the Problem

This new methodology was applied to the Construction II degree course at Barcelona Architecture School (ETSAB). Since 2019, the teaching hours of this course have been divided into 40% lectures and 60% practical sessions, to develop in depth the course contents from theoretical and practical perspectives, so that students gain the related professional skills and competences [42]. This course is a mandatory undergraduate third-year course that has four sessions per year, two each semester, one morning shift and one afternoon shift, with around 80 and 60 students, respectively. Since the pandemic, this course has combined blended and e-learning, depending on the restrictions.

Construction II invites students to ask themselves about how architects construct buildings’ structures, and what architects should consider when they design, represent and supervise the construction works of their architectural projects to build the best performing architectural structures [43]. Furthermore, teachers encourage meta-cognitive development and deep learning through extra activities to discuss the course curricula that, along with the objectives, are aligned with these main questions on the course. This course covers transversal competences, such as teamwork and autonomous learning, general competences, such as understanding structural design, construction and engineering architectural problems, and specific competences, such as knowledge of offsite construction systems [42]. It includes formative and summative assessments, with partial accumulative continuous assessments, submission of a practical assignment, two theoretical exams during the course, and an optional final exam including theoretical and practical parts. Since the last decade [19], teachers of this course have tried to improve students’ learning process by understanding their background and improving their motivation and learning autonomy. To achieve this, teachers continuously collect information from questionnaires, surveys and informal encounters at the beginning, middle and end of the course, as presented in previous articles [18,37,44]. These assessment results justify the present research project and its characteristics, including its participative approach (Figure 1) and its new indicators (Table A3 in Appendix B).

The practical sessions in this course cover three main topics: site soil, foundations, and structures of buildings. Classes on soil focus on which data are required from the soil to design buildings’ foundations and structure, and how architects can obtain this soil information. Foundation and structure activities cover ways that architects can design these elements to obtain the best results, construction processes for these elements, and how architects can provide specifications and draw details of their designs to optimize the construction processes and outcomes of architectural structures. Until now, this has been achieved mainly following team-based learning, case studies, problem-solving and hands-on activities organized around project-based learning (PBL) on an architecture project.

5. Results

This section presents the results of applying the new approach presented in Figure 1 to the Construction II course. The outcomes of each phase are detailed.

5.1. Phase 1

The learning activities involved in the course’s practical sessions and their respective thinking levels are: (1) read and understand soil data sources, MOTL; (2) propose a justified hypothesis of soil for a specific building site, HOTL; (3) extract data from this soil that students will require for their next steps, MOTL; (4) propose specific foundations and structure for a particular soil site and building design, HOTL; (5) design these specific foundations and structure following given methods, MOTL; (6) give specifications about these structural elements following given instructions, MOTL.

5.2. Phase 2

The aforementioned literature reviews [18] classified these alternatives into three types and three groups. Appendix B, Table A2 classifies these learning alternatives and gives references to understand them in detail. These reviews also classified them depending on thinking levels from Bloom’s Taxonomy, revised by Anderson, on which these alternatives could work. The classifications and the SWOT analysis (

Table A7. Strengths of the alternatives assessed using SWOT.

Alternative	Main Strengths
Blended learning	Versatility between face-to-face and online, many advantages from both
e-learning	Direct adaptability to face-to-face meeting restrictions (pandemic), fewer real spaces required for face-to-face learning
TEAL	Closest to presenting students’ new technological skills and habits. Interactive
CBL	Capacity to engage, motivate and enthuse students
TBL	Teamworking that combines members’ skills, knowledge and efforts
Flipped learn.	Allows classes to focus actively on resolving doubts and work on the foundations of previous individual tasks outside the class
PBL	Develop projects that are closer to reality and more practical
Reflective learning	Deep learning. Upper part of the brain
Ind.–com. proj.	Participate in real projects in the industry and in favour of the community
Internships	Participate in real professional activities, in the real environment and context and with controlled expectations and results
Placements
Dual VET	Integration of learning into professional work and the university
Interact. simul.	Closest to students’ videogames, with which students often interact individually and collectively
Social media	Students know and are willing to use this environment. Connects university to students’ other life and activities
Videos real cas.	Watch specific issues in detail, repeatedly, at a chosen speed and schedule. Material available on open websites and platforms
Virtual learning	Interactive and close to students’ skills and habits
Case studies	Attractive to students, engages them, connects to reality
Discussions	Attractive and easy to implement. Requires few resources, versatile
Gamification	Engages students and relaxes them during the learning process. Both brain sides
Interdisciplinary	Reproduces the multidisciplinary professional world. Combines skills, points of view, etc. Promotes brain interrelations, long term memory
Problem-solving	Active and close to reality, easy and versatile to apply
Storytelling	Reaches the right part of the brain, connects and redirects different brain parts (Zadina 2015; Torrijos-Muelas, González-Víllora, and Bodoque-Osma 2021)
Real material	Organoleptic contact with real materials
Hands-on activ.	Manual contact with materials, while achieving real proposals
Role play	Reproduces professional situations in an easy way that involves and engages students
Site visits	Introduces a real professional scenario, in contact with real materials, elements and the environment

,

Table A8. Weaknesses of the alternatives assessed using SWOT.

Alternative	Main Weaknesses
Blended learning	Often the real online capability is low. Difficult for teachers to have skills and materials prepared for face-to-face and online activities
e-learning	Difficult to engage students to participate and focus on course activities when they tend to multitask. Its idiosyncrasies require teachers and students to adapt their time
TEAL	Previous teachers’ and students’ training and working hours or budget to subcontract
CBL	Requires teachers’ preparation of material and students’ engagement
TBL	Teachers and students must learn strategies to control and manage TBL
Flipped learn.	Students are required to work outside the classroom before class
PBL	Requires students to acquire specific previous knowledge
Reflective learn.	Difficult for teachers to fully support all students
Ind.–com. proj.	Large amount of work that depends on the support of industries, community feedback, the general social context, etc.
Internships	Professional environment, university and students are required to work together
Placements
Dual VET	Further work to interconnect the university and professional learning contexts
Interact. Simul.	Previous teachers’ and students’ training and working hours or budget to subcontract. Budget for the right software and hardware
Social media	Students may be distracted due to multitasking or using these applications beyond the course activities during the course
Videos real cas.	Students do not perceive the real environment and senses. If teachers prepare this material, training and working hours are required or budget to subcontract
Virtual learning	Unless existing tools are used, requires teachers’ training and working hours or budget to subcontract. Budget for the right software and hardware
Case studies	Are not representative enough, illustrate only part of the course contents
Discussions	Requires previous preparation from students and teachers
Gamification	Requires teachers’ preparation of material and students’ engagement
Interdisciplinary	Requires complicity and teamwork between the disciplines
Problem-solving	Preparation and renovation. Re-use might cause obsolescence and copying problems
Storytelling	Requires preparation by teachers and theatre/stage skills
Real material	Hygiene; supply, transport and storage of materials
Hands-on activ.	Spatial, machinery and material barriers/limits. Difficult to reach real scale
Role play	Requires previous preparation by teachers and students’ involvement
Site visits	Safety risks, management, group size and visibility, weather, partial view; only one point of the building process, low density of concepts

,

Table A9. Opportunities of the alternatives assessed using SWOT.

Alternative	Main Opportunities
Blended learning	Opens possibilities beyond alternatives that are exclusively face-to-face/online
e-learning	Easily incorporates experts, participants and environments from far away into classes. Easy recording of classes for viewing after a class is over
TEAL	Learning during and after class. Self-learning, self-directed learning
CBL	Enthusiasm for the challenge is transmitted among students and to the course in general
TBL	Promote and improve teamwork competences
Flipped learning	Recorded audios or videos of classes can be reused for students to study/consult later on, for students to come to classes. Sharing between schools
PBL	Can promote learning transversally among subjects, courses, etc.
Reflective learning	Reflections that are made might promote and improve the deep learning process of other students, courses, etc.
Ind.–com. proj.	Strengthen the three vertexes and expand interconnections between industry, community and university. University serving society
Internships	Students complement and test their learning process at the university. Collaborations between the professional world and the university and its outputs. Research
Placements
Dual VET	Students are better prepared for the professional world. Improve the image and consideration that the professional world has about the university
Interact. simul.	Bring the virtual world to the university. Start virtual university learning
Social media	Promote the use of social media in university teaching, engage students and their feeling of belonging to the university community, improve social relations
Videos of real cases	Libraries of useful videos. Time lapse. Merge teacher’s comments with videos to use videos to promote specific learning (i.e., site risks from unsafe sites). Potential of learning from examples, mirror neurons, empathy.
Virtual learning	Able to promote communication and feedback among students and with the teacher
Case studies	Learning transversally, incorporating other knowledge areas. Work with real buildings and visit them. Potential of learning from examples, mirror neurons, empathy
Discussions	Promote critical thinking, communication skills
Gamification	Promote teamwork with gamification. Brain learns through enjoyment
Interdisciplinary	Promote connections between areas, schools and universities, also at other levels such as research
Problem-solving	Promote self-learning and autonomous working. Enable teamwork as well
Storytelling	Promote interactions between brain parts. Introduce moral messages
Real material	Promote the importance of different senses. Work on material properties
Hands-on activ.	Improve real knowledge of the behaviour of materials, e.g., concrete hardening
Role play	Promote communication skills, understanding of others
Site visits	Understand real professional environment, for example on-site risks

and

Table A10. Threats of the alternatives assessed using SWOT.

Alternative	Main Threats
Blended learning	Implies taking care of the material, platform, facilities, etc. of face-to-face and online material
e-learning	Loss of face-to-face contact between the teacher and students and its advantages
TEAL	Students multitasking, disconnection from the course activities, etc.
CBL	Students do not follow or are not enthusiastic about the challenge
TBL	Some team members do not work, leave the group, and unbalanced or incohesive groups
Flipped learning	Students do not look at the material/do not do the task before class
PBL	Students do not work enough for the project to advance, they lack knowledge
Reflective learning	Due to the difficulty of related teamwork, it becomes individual, introspective
Ind.–com. proj.	Industries’ low implication. Communities partially reluctant about the initiative
Internships	Low support from the professional world. Students have inadequate training. Students attitude is not acceptable/adequate for the employer
Placements
Dual VET	Insufficient integration, interrelation between professional and university learning. They advance in parallel
Interact. simul.	Low contribution to learning because there are software programming limitations/difficulties
Social media	Students’ multitasking, distractions; application outage
Videos real case	Students get false ideas, perceptions, misunderstandings
Virtual learning	The virtual material is not really useful or relevant due to technological barriers. The material is too specific, for only one context, not replicable
Case studies	Not applicable to the course, obsolete, uninteresting, difficult to access interesting and holistic case studies
Discussions	Low participation from students, get stuck in a topic, go beyond the main discussion
Gamification	Game challenges not aligned with students’ skills, knowledge, etc.
Interdisciplinary	No teamwork, respect, comprehension among the disciplines
Problem-solving	Not adequate in terms of topic, difficulty, duration, etc.
Storytelling	The teacher unable to capture students’ attention, involvement, etc.
Real material	Hygiene requirements not achievable, obsolete materials
Hands-on activ.	Lack of material, nuisances to other classes, damage/deterioration/dirty spaces
Role play	Low involvement/participation of students. Shy students. Uninteresting roles
Site visits	Building site stakeholders’ opposition, bad weather, passive students, not an interesting point in the building, lockdowns

in Appendix D) confirmed that most of the activities were feasible alternatives for the case study. However, there were three alternatives that could not be introduced due to their incompatibility with the undergraduate course organisation and students’ schedule. These were internships, placements and dual vocational education and training (VET). Currently, some Construction II students carry out these activities outside of the framework of the course. Thus, feasible alternatives are: (A1) blended learning, (A2) e-learning, (A3) technology-enabled active learning, (A4) challenge-based learning (CBL), (A5) team-based learning (TBL), (A6) flipped classrooms, (A7) project-based learning (PBL), (A8) reflective learning, (A9) industry–community projects, (A10) interactive simulations, (A11) social media activities, (A12) videos of real cases, (A13) virtual learning activities, (A14) case studies, (A15) discussions, (A16) gamification activities, (A17) interdisciplinary activities, (A18) problem-solving activities, (A19) storytelling, (A20) real material practices, (A21) hands-on activities, (A22) role play and (A23) site visits.

5.3. Phase 3

The global sustainability index (GSI) and the requirement satisfaction values were the main results of this phase.

Table 4. Global sustainability index (GSI) and requirement satisfaction.

Code	Alternative	R1	R2	R3	R4	GSI
A1	Blended learning	0.66	0.79	0.69	0.60	0.67
A2	e-learning	0.78	0.77	0.57	0.38	0.59
A3	TEAL	0.64	0.68	0.57	0.56	0.61
A4	CBL	0.69	0.88	0.56	0.68	0.71
A5	Coop. TBL	0.66	0.80	0.73	0.55	0.65
A6	Flipped learning	0.59	0.66	0.56	0.57	0.59
A7	PBL	0.71	0.77	0.56	0.55	0.64
A8	Reflective learning	0.72	0.89	0.91	0.57	0.71
A9	Ind. com. proj.	0.63	0.41	0.69	0.57	0.56
A10	Interact. simulation	0.64	0.67	0.56	0.57	0.61
A11	Social media	0.84	0.90	0.56	0.51	0.68
A12	Videos of a real case	0.78	0.84	1.00	0.51	0.70
A13	Virtual learning	0.75	0.67	0.56	0.44	0.58
A14	Case studies	0.77	0.86	0.81	0.58	0.71
A15	Discussions	0.51	0.94	0.56	0.67	0.68
A16	Gamification	0.62	0.74	0.81	0.68	0.69
A17	Interdisciplinary	0.57	0.41	0.69	0.69	0.60
A18	Problem-solving	0.75	0.91	0.81	0.42	0.65
A19	Storytelling	0.56	0.95	1.00	0.49	0.66
A20	Real material	0.68	0.86	1.00	0.43	0.65
A21	Hands-on activities	0.47	0.79	1.00	0.63	0.66
A22	Role play	0.71	0.81	0.91	0.50	0.66
A23	Site visits	0.60	0.60	0.91	0.51	0.60

Legend: technology-enabled active learning (TEAL), challenge-based learning (CBL), project-based learning (PBL), team-based learning (TBL).

presents the requirement satisfaction values and GSI, while

Table A11. GSI of the alternatives and sustainability satisfaction of their indicators, criteria and requirements.

	I01	I02	C1	I03	I04	C2	R1	I05	I06	C3	I07	I08	I09	C4	R2	I10	C5	R3	I11	I12	I13	I14	C6	I15	I16	I17	I18	C7	I19	I20	C8	R4	GSI
A1	0.85	0.67	0.77	0.07	1.00	0.51	0.66	0.81	0.90	0.86	0.74	0.78	0.74	0.75	0.79	0.69	0.69	0.69	0.62	0.00	1.00	0.69	0.57	0.37	0.67	0.71	0.58	0.56	0.77	0.64	0.71	0.60	0.67
A2	0.85	0.81	0.83	0.44	1.00	0.70	0.78	0.81	1.00	0.91	0.74	0.57	0.74	0.69	0.77	0.57	0.57	0.57	0.50	0.00	1.00	0.69	0.55	0.33	0.33	0.71	0.58	0.47	0.00	0.00	0.00	0.38	0.59
A3	0.85	0.67	0.77	0.02	0.98	0.47	0.64	0.61	1.00	0.83	0.74	0.21	0.74	0.59	0.68	0.57	0.57	0.57	0.44	0.00	1.00	0.61	0.52	0.64	0.27	1.00	0.61	0.62	0.99	0.00	0.52	0.56	0.61
A4	0.77	0.57	0.67	0.47	1.00	0.72	0.69	1.00	0.90	0.95	0.74	0.81	1.00	0.84	0.88	0.56	0.56	0.56	0.92	1.00	0.51	0.33	0.71	0.73	0.79	0.06	0.81	0.64	0.77	0.64	0.71	0.68	0.71
A5	0.67	0.62	0.65	0.46	0.92	0.68	0.66	1.00	1.00	1.00	0.88	0.00	1.00	0.67	0.80	0.73	0.73	0.73	0.83	1.00	0.00	0.33	0.53	0.73	0.79	0.64	0.81	0.75	0.46	0.00	0.24	0.55	0.65
A6	0.54	0.81	0.67	0.02	0.99	0.48	0.59	0.81	1.00	0.91	0.74	0.77	0.00	0.50	0.66	0.56	0.56	0.56	0.37	0.00	0.51	0.63	0.36	0.37	0.54	0.71	0.58	0.53	0.94	0.96	0.95	0.57	0.59
A7	0.67	0.45	0.57	0.89	0.93	0.91	0.71	1.00	0.90	0.95	0.37	0.67	1.00	0.66	0.77	0.56	0.56	0.56	0.62	1.00	0.51	0.33	0.64	0.69	0.79	0.06	0.81	0.62	0.00	0.64	0.30	0.55	0.64
A8	0.85	0.72	0.79	0.29	1.00	0.63	0.72	1.00	1.00	1.00	0.74	0.72	1.00	0.82	0.89	0.91	0.91	0.91	0.37	0.57	1.00	0.37	0.61	0.42	0.85	0.06	0.78	0.54	0.46	0.64	0.54	0.57	0.71
A9	0.54	0.45	0.50	0.68	0.94	0.80	0.63	0.41	0.31	0.36	0.37	0.21	0.74	0.45	0.41	0.69	0.69	0.69	0.76	1.00	0.00	0.65	0.58	0.90	0.85	0.06	0.74	0.69	0.10	0.64	0.36	0.57	0.56
A10	0.85	0.77	0.81	0.01	0.83	0.40	0.64	0.21	1.00	0.64	0.88	0.21	0.88	0.69	0.67	0.56	0.56	0.56	0.37	0.00	1.00	0.49	0.48	0.86	0.27	1.00	0.58	0.68	0.98	0.00	0.52	0.57	0.61
A11	0.77	0.85	0.81	0.77	1.00	0.88	0.84	0.81	1.00	0.91	0.94	0.67	1.00	0.89	0.90	0.56	0.56	0.56	0.37	0.57	0.51	0.41	0.48	0.86	0.27	1.00	0.51	0.67	0.00	0.64	0.30	0.51	0.68
A12	0.67	1.00	0.83	0.60	0.86	0.72	0.78	0.61	1.00	0.83	0.94	0.57	1.00	0.86	0.84	1.00	1.00	1.00	0.37	0.00	0.51	0.81	0.39	0.69	0.67	0.87	0.51	0.67	0.20	0.64	0.41	0.51	0.70
A13	0.85	0.77	0.81	0.49	0.85	0.66	0.75	0.21	1.00	0.64	0.88	0.21	0.88	0.69	0.67	0.56	0.56	0.56	0.41	0.00	1.00	0.49	0.49	0.86	0.27	1.00	0.51	0.67	0.00	0.00	0.00	0.44	0.58
A14	0.74	0.67	0.71	0.72	1.00	0.85	0.77	0.81	1.00	0.91	0.88	0.67	0.88	0.82	0.86	0.81	0.81	0.81	0.47	0.57	0.51	0.53	0.52	0.77	0.77	0.71	0.61	0.72	0.00	0.96	0.45	0.58	0.71
A15	0.42	0.51	0.46	0.25	0.95	0.58	0.51	1.00	1.00	1.00	0.88	0.81	1.00	0.90	0.94	0.56	0.56	0.56	0.92	1.00	0.26	0.53	0.67	0.73	0.74	0.40	0.81	0.69	0.66	0.64	0.65	0.67	0.68
A16	0.67	0.77	0.72	0.01	1.00	0.48	0.62	0.61	0.86	0.75	0.74	0.45	1.00	0.74	0.74	0.81	0.81	0.81	0.47	1.00	0.51	0.69	0.67	0.86	0.67	1.00	0.61	0.78	0.98	0.00	0.52	0.68	0.69
A17	0.67	0.45	0.57	0.19	1.00	0.57	0.57	0.21	0.31	0.26	0.37	0.21	0.88	0.50	0.41	0.69	0.69	0.69	0.71	1.00	0.00	0.61	0.56	0.90	0.85	0.06	0.71	0.68	0.83	0.96	0.89	0.69	0.60
A18	0.77	0.77	0.77	0.56	0.93	0.73	0.75	1.00	1.00	1.00	0.74	0.83	1.00	0.85	0.91	0.81	0.81	0.81	0.41	0.00	1.00	0.49	0.49	0.42	0.81	0.71	0.71	0.64	0.00	0.00	0.00	0.42	0.65
A19	0.57	0.85	0.70	0.02	0.77	0.37	0.56	1.00	1.00	1.00	0.94	0.77	1.00	0.91	0.95	1.00	1.00	1.00	0.37	0.00	0.00	0.85	0.25	0.51	0.67	1.00	0.55	0.65	0.55	0.64	0.59	0.49	0.66
A20	0.57	0.67	0.61	0.89	0.62	0.76	0.68	0.61	1.00	0.83	0.94	0.67	1.00	0.89	0.86	1.00	1.00	1.00	0.57	0.00	0.00	0.57	0.24	0.82	0.67	0.87	0.41	0.69	0.00	0.64	0.30	0.43	0.65
A21	0.57	0.67	0.61	0.03	0.52	0.26	0.47	0.61	0.90	0.77	0.74	0.67	1.00	0.80	0.79	1.00	1.00	1.00	0.62	1.00	0.00	0.53	0.52	0.95	0.70	0.32	0.64	0.69	0.72	0.64	0.68	0.63	0.66
A22	0.65	0.62	0.63	0.74	0.88	0.81	0.71	0.81	0.81	0.81	0.74	0.67	1.00	0.80	0.81	0.91	0.91	0.91	0.60	0.57	0.26	0.53	0.47	0.69	0.70	0.32	0.81	0.65	0.00	0.64	0.30	0.50	0.66
A23	0.51	0.72	0.61	0.34	0.88	0.59	0.60	0.61	0.31	0.45	0.74	0.27	1.00	0.69	0.60	0.91	0.91	0.91	0.37	0.00	0.00	0.69	0.22	0.86	0.70	0.71	0.51	0.71	0.61	0.64	0.62	0.51	0.60

in Appendix E presents the complete results with all the indicators and criteria satisfaction values. These tables show that challenge-based learning (CBL), reflective learning and case studies achieved the highest GSI of 0.71, while industrial and community projects achieved the lowest GSI of 0.56. Thus, the range of GSI was only 0.15 points. The interval of values for the satisfaction of all requirements was from 0.38 to 1.00. In the indicators of requirement satisfaction, social media and hand-on activities had values of 0.84 and 0.47 for applicability satisfaction (R1), respectively. For the economic requirement (R2), discussions and storytelling were rated 0.94 and 0.95, respectively, while industry–community and interdisciplinary projects had a 0.41 satisfaction value. For the environmental requirement (R3), videos of real cases, storytelling, real material and hands-on activities achieved complete satisfaction, while many activities achieved 0.56 (A4, A6, A7, A10, A11, A13 and A15). Interdisciplinary activities achieved 0.68, while e-learning had a value of 0.38 for social requirement satisfaction (R4). Considering all the alternatives, the highest average satisfaction was in the category of R2 economic, while the lowest was in R4 social.

5.4. Phase 4

The teaching team defined two scenarios of a maximum number of practical sessions per semester (12 and 14). This number can change because of external factors, such as the university calendar and local restrictions, including compulsory or unforeseen days off. The team also prepared the classification presented in

Table 5. Minimum number of sessions the activities require to be implemented.

Exclusivity	Type and Group	Minimum Required Sessions	Alternatives
Can be combined with any activity	t1 or t2, g1	1	(A1) Blended learning, (A2) e-learning, (A3) TEAL
Requires exclusive session/s	t2 or t3, g2 or g3	1	(A5) TBL, (A6) flipped classroom, (A8) reflective learning, (A10) interact. simulation, (A11) social media, (A12) videos of a real case, (A14) case studies, (A15) discussions, (A16) gamification, (A18) problem-solving, (A19) storytelling, (A20) real material, (A22) role play, (A23) site visits
		3	(A13) Virtual learning, (A21) hands-on activities
		8	(A4) CBL, (A7) PBL, (A9) ind.-com. proj., (A17) interdisciplinary

Legend: technology-enabled active learning (TEAL), challenge-based learning (CBL), project-based learning (PBL), team-based learning (TBL).

, which organizes alternatives according to their exclusivity and the number of minimum required sessions. The first three alternatives can be combined with any of the other 20 following learning options, because the first two are general course strategies and the second is a combinable learning dynamic (see Phase 2). The other 20 alternatives require exclusive sessions. Thus, to be used in the course that was studied, these alternatives require a minimum of 1, 3 or 8 specific sessions. These numbers could vary in other contexts.

5.5. Phase 5

In the first scenario of a 12-session course, the best Knapsack results were (a1) among the first three alternatives and, giving a total GSI of 0.66, blended learning (A1) was selected for all sessions; and (b1) among the other 20 learning options, with a total GSI of 0.72, CBL (A4) was selected for 8 sessions, and videos of real cases (A12) for 4 sessions. In the second scenario of a 14-session course, the best sustainability results were (a2) among the first three learning alternatives, giving a total GSI of 0.65, A1 was selected for all sessions; and (b2) between the other alternatives, with a total GSI of 0.72, A4 was selected for 8 sessions and A12 for 6 sessions.

5.6. Phase 6

The course teachers’ final decision ratifies Knapsack proposals for the first three alternatives (a1 and a2), although A1 could be replaced in the case of external conditions. For example, in the case of pandemic lockdowns, e-learning (A2) would be extended to all sessions, to achieve a lower GSI of 0.59. Among the other alternatives, teachers proposed: (b1) for the 12-session course scenario, with a total GSI of 0.71, CBL (A4) for 8 sessions and reflective learning (A8), videos of real cases (A12), case studies (A14) and site visits (A23) for one session each; and (b2) for the 14-session course scenario, with a total GSI of 0.70, CBL (A4) for 8 sessions and, A8, Interactive simulation (A10), A12, A14, Gamification (A16) and A23 for one session each. To sum up, the following changes were applied: (a) multiple activities that required at least one session were applied instead of repeating one activity 4 or 6 times; (b) activities A8, A10, A14, A16 and A23 were added. These changes were applied, although the GSI was slightly lower because the range of activities could improve students’ learning process by introducing variety and surprise factors that could further engage and awaken students’ brains with joy and wonder [45]; A8 allows work on the important aspect of students’ meta-cognition; A10 is an online activity that the teaching team is developing as an alternative to A23 in the case of threats (Table A10 in Appendix D); A14 works with real cases as examples that help students’ understanding [46]; A16 is a learning-by-playing alternative that also increases students’ engagement and readiness to learn [41]; and A23 works with a real environment so that students learn from a different, unique perspective.

6. Discussion

The first phase of this study proves that most alternatives are compatible with the medium- and higher-order thinking level of this course’s practical sessions [18], although CBL, PBL and discussions are optimized for HOTL contents. Therefore, the four MOTL activities that were presented in phase one’s results cannot be resolved with these alternatives. These results explain why, during previous courses, some MOTL activities such as extracting, reading and understanding data were difficult to perform using PBL. The SWOT confirms the potential and threats of the 26 alternatives that were studied. The complexity of the alternatives justifies the need for a methodology such as that developed in this project to address implementation in courses. In terms of the types and groups of practical session alternatives (Table A2, Appendix B), the three activities with the highest GSI are active learning activities (g2). Each of the 23 alternatives has its own performance regarding each requirement’s average satisfaction: recent digital technologies (g1) have the highest satisfaction value for applicability because of their notable performance; active learning alternatives (g2) have the highest satisfaction value for economic issues due to their low costs; and real experiences (g3) have the highest environmental value because of the low added environmental impact. The results for these alternatives are in relative terms and are not applicable to the environmental impact of higher education activities and facilities that require specific studies [47] beyond the boundaries of this research project.

The resulting requirements tree (Table 1) includes the 20 main discriminative indicators classified into 4 requirements: applicability, economic, environmental and social. This classification allows the study of satisfaction in each sustainability branch. The resulting GSI and requirements for the sustainability indexes can be applied to the context of the specific case study, which focuses on the learning process (Table 2). Satisfaction with the applicability requirement ranges from 0.47 to 0.84, with an average satisfaction of 0.67/1.00. This is due to the ease of application of most alternatives (I01) and their flexibility (I02). However, in general, the alternatives are difficult to transfer among teachers (I03). Thus, to improve their applicability, more and better material should be available to the teaching community, especially regarding new digital alternatives (A10, A13 and A16). Satisfaction with the economic requirement is even more varied as it ranges from 0.41 to 0.95, with an average satisfaction of 0.77/1.00. This is due to the general notable satisfaction with logistic issues (I06) and students’ dedication after classes (I09), good direct costs (I05) and dedication in class (I07), and fair satisfaction with teachers’ dedication after classes. Satisfaction with the environmental requirement is high and medium for all alternatives. It ranges from 0.56 to 1.00, with an average satisfaction of 0.74/1.00. This confirms that all the assessed learning activities have a similarly low extra-environmental impact. Satisfaction of the social requirement is lowest. It ranges from 0.38 to 0.69, with an average satisfaction of 0.55/1.00. One reason is the general complementary behaviour between the capacity to encourage cooperative (I12) or autonomous work (I12), in which most alternatives perform outstandingly in one, but poorly in the other, except for some alternatives of A4, A7, A8 and A16. Indicators I19 and I20 detect diversity within the activities’ innovation and teacher training to achieve new skills to apply the alternatives. No relation was found between these GSI or requirement satisfaction (Table 4) and the minimum sessions required for a learning alternative to be applied (Table 5).

Using the scenarios in

Table 6. Description of the weighting scenarios considered in the sensitivity analysis.

	Weighting Scenario Description	R1	R2	R3	R4
Ws1	This research project weighting, based on Delphi	26	21	11	42
Ws2	Equal weights for all indicators	25	25	25	25
Ws3	Applicability requirement-driven scenario	55	15	15	15
Ws4	Economic requirement-driven scenario	15	55	15	15
Ws5	Social requirement-driven scenario	15	15	15	55

and data, a sensitivity analysis further analyses these course context implications and the robustness of the new approach. The five scenarios are: (Ws1) this project’s reference weights (Table 1); (Ws2) a neutral scenario with the same weight for each requirement; (Ws3) prioritizes the applicability of the learning alternatives with the highest weight; (Ws4) gives more weight to the economic requirement because the cost issues are considered crucial; and (Ws5) focuses on social issues and gives the highest weight to this requirement. This research did not consider an environmental requirement-driven scenario due to its limited importance in this project in terms of weight and number of indicators.

The sensitivity analysis presented in Figure 2 and Figure 3 confirms the robustness of this approach, which presents a similar tendency for most alternatives. Exceptions are, for example, (A15) discussions that have higher satisfaction in a cost-driven scenario, and (A21) hands-on activities with lower satisfaction in an applicability-driven scenario. On the other hand, some have more similar GSI in all scenarios (A3, A6, A10 and A23) than others (A2, A9, A11 and A15). Moreover, alternatives A2, A11, A12, A15 and from A18 to A23 have a broader difference between scenarios.

Phase 5 is a crucial step that gives a set proposal to teachers based on the new methodology (Figure 1) that mitigates bias possibility in the outcome. This new methodology calculates the resulting GSI of these two sets and other alternative sets and can relate them to current course scenarios. Combining MIVES and Knapsack allows a sensitivity analysis to be carried out to obtain the best set of learning alternatives for each of the five scenarios in Table 6 and their GSI.

Table 7. Sensitivity analysis combining MIVES and Knapsack.

Weighting Scenarios	12 Sessions		14 Sessions
	Proposed Set	GSI	Proposed Set	GSI
Ws1	A4 (8); A12 (4)	0.72	A4 (8); A12 (6)	0.72
Ws2	A12 (12)	0.78	A12 (14)	0.78
Ws3	A12 (12)	0.78	A12 (14)	0.78
Ws4	A19 (12)	0.83	A19 (14)	0.83
Ws5	A8 (2); A16 (10)	0.70	A8 (4); A16 (10)	0.70

presents the combined analysis results. It gives the expected outcomes because the best performing learning alternative for each requirement with the best GSI is the chosen alternative for each requirement-driven scenario. By mixing results from different scenarios, the teachers’ proposal can be reached (phase 6). This result should be further investigated, if it can be used for future automated versions of this research’s new methodology.

However, in this present version, Phase 6 is essential to adapt better the set solution from Phase 5 to the specific students and context, relying entirely on the teaching team’s thinking process. In the case study, the Phase 6 solution has a GSI of 0.71. This is a 9% improvement on the current practical sessions in Construction II that have a GSI of 0.65: PBL (A7) in 8 sessions, plus cooperative learning (A5), case studies (A14), problem-solving activities (A18), hands-on (A21) activities and site visits (A23) in one session each. The next step is to adjust this course to the new SET progressively and monitor it.

7. Conclusions

This article presents the successful development of a new approach to assist teachers in choosing the best set of strategies, dynamics and activities for the practical sessions of architecture courses. The development of this approach incorporated several methods, such as a SWOT analysis to define the main strengths, weaknesses, opportunities and threats of 26 alternatives for learning specific work-related competences at university. The strengths of this new approach include its robust methods and the incorporation of the teacher’s team experience into its result.

The first application of this method successfully helped teachers in the case study to improve their practical sessions, with a new set of alternatives that has a sustainability index improved by 9%. Moreover, in the case study, this assessment proved that active alternatives should improve implementation-related issues in the teachers’ teaching materials and dedication outside class. Nevertheless, this approach has room for improvement with future steps such as: (a) implementing, monitoring and assessing the outcomes of the case study; (b) improving the approach considering other case studies and its automation.