nach oben

International Journal of Technology and Design Education

Erschienen in:

02.03.2019

Identifying design values across countries through adaptive comparative judgment

verfasst von: Scott Ronald Bartholomew, Emily Yoshikawa Ruesch, Eva Hartell, Greg J. Strimel

Erschienen in: International Journal of Technology and Design Education | Ausgabe 2/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Adaptive comparative judgment (ACJ) has proven to be a valid, reliable, and feasible method for assessing student performance in open-ended design scenarios. In addition to the use of ACJ for purely assessment and evaluation, research has demonstrated an opportunity to identify the design values of judges involved with the ACJ process. The potential for ACJ, as a tool for understanding cultural design values, and potentially facilitating international collaboration, is intriguing. Therefore, this study established three panels of judges, from countries around the world, to assess one body of student work using the ACJ method. The similarities, differences, and findings from these assessment results were analyzed, revealing distinct design values, preferences, and differences for each group of judges from the different locations.

Vorheriger Artikel Finding the connection between research and design: the knowledge development of STEM teachers in a professional learning community

Nächster Artikel Participation and achievement in technology education: the impact of school location and socioeconomic status on senior secondary technology studies

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

Alamir, A. (2015). Comparative Judgment Update. Retrieved February 23, 2018, from http://digitalassess.com/comparative-judgement-update/.

Bartholomew, S. R. (2017). Assessing open-ended design problems. Technology and Engineering Education Teacher,76(6), 13–17.

Bartholomew, S., Hartell, E., & Strimel, G. (2017a). ACJ: A tool for international assessment collaboration. In PATT34 Millersville University, Pennsylvania, USA 10–14 July, 2017.

Bartholomew, S. R., Reeve, E., Veon, R., Goodridge, W., Stewardson, G., Lee, V., et al. (2017b). Mobile devices, self-directed learning, and achievement in Technology and Engineering Education classrooms during a STEM activity. Journal of Technology Education,29(1), 2–24.CrossRef

Bartholomew, S. R., Strimel, G. S., & Jackson, A. (2017c). A comparison of traditional and adaptive comparative judgment assessment techniques for freshmen engineering design projects. International Journal of Engineering,34(1), 20–33. (Manuscript submitted for publication).

Bartholomew, S. R., Strimel, G. S., & Yoshikawa, E. (2018). Using adaptive comparative judgment for student formative feedback and learning during a middle school open-ended design challenge. International Journal of Technology & Design Education. https://doi.org/10.1007/s10798-018-9442-7.CrossRef

Bartholomew, S. R., Zhang, L., Garcia Bravo, E., & Strimel, G. J. (2019). A tool for formative assessment and learning in a graphics design course: Adaptive comparative judgment. The Design Journal. https://doi.org/10.1080/14606925.2018.1560876.CrossRef

Bouncken, R., Brem, A., & Kraus, S. (2016). Multi-cultural teams as sources for creativity and innovation: The role of cultural diversity on team performance. International Journal of Innovation Management,20(01), 1650012.CrossRef

Bramley, T. (2015). Investigating the reliability of adaptive comparative judgment (Vol. 36). Cambridge: Cambridge Assessment.

Charmaz, K., & Belgrave, L. (2012). Qualitative interviewing and grounded theory analysis. The SAGE Handbook of Interview Research: The Complexity of the Craft, 2, 347–365.

Cunningham, C. M. (2009). Engineering is elementary. The Bridge,30(3), 11–17.

de la Paz, K. B. (2017). Who are we? (special issue). Technology and Engineering Teacher, 76(4), 8.

De Vries, M. J. (2015). Research challenges for the future. In P. Williams, A. Jones, & C. Buntting (Eds.), The future of technology education: Contemporary issues in technology education (pp. 253–269). Singapore: Springer.

Denson, C. D., Buelin, J. K., Lammi, M. D., & D’Amico, S. (2015). Developing instrumentation for assessing creativity in engineering design. Journal of Technology Education,27(1), 23–40.

Department of Education. (2013). National curriculum in England: Design and technology programmes of study. Retrieved from https://www.gov.uk/government/publications/nationalcurriculum-in-england-design-and-technology-programmes-of-study. Accessed 25 Feb 2019.

Design and Technology Association. (2017). Designed and made in Britain…? Retrieved from https://www.data.org.uk/campaign/. Accessed 25 Feb 2019.

Diefes-Dux, H. A., Moore, T., Zawojewski, J., Imbrie, P. K., & Follman, D. (2004). A framework for posing open-ended engineering problems: Model-eliciting activities. In 34th Annual frontiers in education, 2004. FIE 2004 (pp. F1A–3). IEEE.

DigitalAssess. (2018). What we do: CompareAssess. Retrieved from http://digitalassess.com/what-wedo/#compareassess.

Discover McDonald’s Around the Globe: McDonald’s (2010). Retrieved August 25, 2017, from http://corporate.mcdonalds.com/mcd/country/map.html. Accessed 25 Feb 2019.

Dutson, A. J., Todd, R. H., Magleby, S. P., & Sorensen, C. D. (1997). A review of literature on teaching engineering design through project-oriented capstone courses. Journal of Engineering Education,86(1), 17–28.CrossRef

Edens, K. M. (2000). Preparing problem solvers for the 21st century through problem-based learning. College Teaching,48(2), 55–60.CrossRef

Friedman, T. L. (2005). The world is flat: A brief history of the twenty-first century. London: Macmillan.

Gay, G. (2002). Preparing for culturally responsive teaching. Journal of teacher education,53(2), 106–116.CrossRef

Getto, G., & Sun, H. (2017). Localizing user experience: Strategies, practices, and techniques for culturally sensitive design. Technical Communication,64(2), 89–94.

Gilpin, R. (2016). The political economy of international relations. Princeton: Princeton University Press.

Guidry Lacina, J. (2002). Preparing international students for a successful social experience in higher education. New Directions for Higher Education,2002(117), 21–28.CrossRef

Hagberg, J. E., & Hultén, M. (2005). Skolans undervisning och elevers lärande i teknik: svensk forskning i internationell kontext. Stockholm: Vetenskapsrådet.

Hallström, J. (2009). Technical knowledge in a technical society: Elementary school technology education in Sweden, 1919–1928. History of Education,38(4), 455–474. https://doi.org/10.1080/00467600801995427.CrossRef

Hartell, E. (2015). Assidere necesse est: Necessities and complexities regarding teachers’ assessment practices in technology education. Doctoral thesis. KTH Royal Institute of Technology. Retrieved from http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A788413&dswid=-3318. Accessed 25 Feb 2019.

Hartell, E., & Skogh, I. B. (2015). Criteria for success: A study of primary technology teachers’ assessment of digital portfolios. Australasian Journal of Technology Education,2(1), 1–17.CrossRef

Hultén, M. (2013). Technology for all: Turning a keyword into a school subject in post-war Sweden. History of Education,42(5), 622–637. https://doi.org/10.1080/0046760X.2013.832410.CrossRef

Idler, S. (2013). How to design for a cross-cultural user experience. Retrieved August 25, 2017, from https://usabilla.com/blog/designing-for-a-cross-cultural-user-experience-part1/. Accessed 25 Feb 2019.

International Technology and Engineering Educators Association (ITEA/ITEEA). (2000/2002/2007). Standards for technological literacy: Content for the study of technology. Reston, VA: Author.

Jones, I., & Adcock, L. (2012). Summative peer assessment of undergraduate calculus using adaptive comparative judgement. In P. Iannone & A. Simpson (Eds.), Mapping university mathematics assessment practices. Norwich: University of East Anglia.

Jones, I., & Wheadon, C. (2015). Peer assessment using comparative and absolute judgement. Studies in Educational Evaluation, 47, 93–101.CrossRef

Katehi, L., Pearson, G., & Feder, M. (2009). Engineering in K-12 education, Committee on K-12 Engineering Education, National Academy of Engineering and National Research Council of the National Academies.

Khan, T., Pitts, M., & Williams, M. A. (2016). Cross-cultural differences in automotive HMI design: A comparative study between UK and Indian users’ design preferences. Journal of Usability Studies,11(2), 45–65.

Kimbell, R. (2007). E-assessment in project e-scape. Design & Technology Education: An International Journal,12(2), 66–76.

Kimbell, R. (2012a). Evolving project e-scape for national assessment. International Journal of Technology and Design Education,22, 135–155.CrossRef

Kimbell, R. (2012b). The origins and underpinning principles of e-scape. International Journal of Technology and Design Education,22, 123–134.CrossRef

Kimbell, R., & Stables, K. (2007). Researching design learning: Issues and findings from two decades of research and development (Vol. 34). Berlin: Springer.CrossRef

Kimbell, R., Wheeler, T., Stables, K., Shepard, T., Martin, F., Davies, D., et al. (2009). e-scape portfolio assessment: A research & development project for the Department of Children, Families and Schools, phase 3 report (p. 169). London: Goldsmiths, University of London.

Lee, J. J., & Sayed, S. (2008). Culturally sensitive design. In Y. Levanto, P. Sivenius, & S. Vihma (Eds.), Design connections: Knowledge, value and involvement through design (pp. 54–63). Finland: UIAH Press.

Lewis, T., & Zuga, K. (2005). A conceptual framework of ideas and issues in technology education. Washington, DC: National Science Foundation.

Lindsay, B., & Simeon, E. J. (2016). University social and public engagement: Creative nexuses for STEM research and international relations. In Globalisation and higher education reforms (pp. 59–75). Springer.

Montgomery, C., & McDowell, L. (2009). Social networks and the international student experience: An international community of practice? Journal of Studies in International Education,13(4), 455–466.CrossRef

Moye, J. J., Jones, V. R., & Dugger, W. E. (2015). Status of technology and engineering education in the United States: A fifth report of the findings from the states (2014-15). Technology and Engineering Teacher,74(7), 30–36.

NAE. (2011/2016). The Swedish National Agency for Education Curriculum for the compulsory school, preschool class and the recreation center, 2011 (2011). Sweden. Retrieved from http://www.skolverket.se/publikationer?id=2687. Accessed 25 Feb 2019.

National Academy of Engineering (NAE) & National Research Council (NRC). (2014). STEM integration in K–12 education: Status, prospects, and an agenda for research. Washington, DC: National Academies Press.

Newhouse, C. P. (2014). Using digital representations of practical production work for summative assessment. Assessment in Education: Principles, Policy & Practice,21(2), 205–220.

Nikander, P. (2008). Working with transcripts and translated data. Qualitative Research in Psychology,5(3), 225–231.CrossRef

Norström, P. (2014). Technological knowledge and technology education. Doctoral thesis. KTH Royal Institute of Technology. Retrieved from http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A715009&dswid=62. ISBN: 978-91-7595-078-5. Accessed 25 Feb 2019.

National Research Council (NRC). (2009). Engineering in K–12 education: Understanding the status and improving the prospects. Washington, DC: National Academies Press.

National Research Council (NRC). (2012). Education for life and work: Developing transferable knowledge and skills in the 21st Century. Washington: National Academies Press.

Partnership for 21st Century Skills. (2017). P21 framework definitions. Washington, DC: Author. Retrieved from http://www.p21.org/our-work/p21-framework.

Pollitt, A. (2004). Let’s stop marking exams. Retrieved from http://www.cambridgeassessment.org.uk/images/109719-let-s-stop-marking-exams.pdf. Accessed 25 Feb 2019.

Pollitt, A. (2012). The method of adaptive comparative judgment. Assessment in Education: Principles, Policy & Practice,19(3), 281–300.

Pollitt, A. (2015). On ‘Reliability’ bias in ACJ. Cambridge Exam Research. Retrieved 2 Feb 2018. https://www.researchgate.net/publication/283318012_On_'Reliability'_bias_in_ACJ.

Pollitt, A., & Crisp, V. (2004). Could comparative judgments of script quality replace traditional marking and improve the validity of exam questions? Paper presented at the BERA annual conference, UMIST Manchester, England.

Qualifications and Curriculum Authority. (2000). Design and technology in the national curriculum. London: HMSO. Retrieved from http://www.qca.org.uk. Accessed 25 Feb 2019.

Rangel, C., & Lynch, D. (2018). Addressing the issue of bias in the measurement of reliability in the method of Adaptive Comparative Judgment. Athlone, Ireland: Paper presented at the 36th Pupils’ Attitudes towards Technology Conference.

Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research. (Reprinted as 2nd ed., 1980, Chicago: University of Chicago Press).

Rasinen, A. (2003). An analysis of the technology education curriculum of six countries. Journal of Technology Education, 15. Retrieved from http://scholar.lib.vt.edu/ejournals/JTE/v15n1/rasinen.html. Accessed 25 Feb 2019.

Regeringskansliet. (2017). Stärkt digital kompetens i skolans styrdokument. Promemoria 20170309. Stockholm, Sweden. Retrieved from https://www.regeringen.se/pressmeddelanden/2017/03/starkt-digital-kompetens-i-laroplaner-ochkursplaner/. Accessed 25 Feb 2019.

Rowsome, P., Seery, N., & Lane, D. (2013). The development of pre-service design educator’s capacity to make professional judgments on design capability using adaptive comparative judgment. American Society for Engineering Education.

Saldaña, J. (2009). The coding manual for qualitative researchers. Thousand Oaks, CA: SAGE.

Schilling, K., & Applegate, R. (2012). Best methods for evaluating educational impact: A comparison of the efficacy of commonly used measures of library instruction. Journal of the Medical Library Association,100(4), 258–269. https://doi.org/10.3163/1536-5050.100.4.007.CrossRef

Seery, N., & Canty, D. (2017). Assessment and learning: The proximal and distal effects of comparative judgment. In Handbook of technology education (pp. 1–14).

Seery, N., Canty, D., & Phelan, P. (2012). The validity and value of peer assessment using adaptive comparative judgement in design driven practical education. International Journal of Technology and Design Education,22(2), 205–226.CrossRef

Seery, N., Delahunty, T., Canty, D., & Buckley, J. (2017). Illustrating educational development through ipsative performance in design based education. In PATT2017: Technology & engineering education–fostering the creativity of youth around the globe.

Seery, N., Delahunty, T., Sorby, S., & Sadowski, M., (2018). Adaptive Comparative Judgment as an alternative to the Delphi method for establishing a concept inventory for graphics. Paper presented at the ASEE EDGD Annual Conference, Kingston, Jamaica.

Shah, N. (2013). Cross cultural consideration for user interface design. Retrieved August 25, 2017, from http://www.humanfactors.com/newsletters/cross-cultural-considerations-for-user-interface-design.asp. Accessed 25 Feb 2019.

Skolinspektionen. (2014). (Swedish School Inspectorate). Teknik – gör det osynliga synligt Om kvaliteten i grundskolans teknikundervisning. Stockholm.

Skolverket/The Swedish National Agency for Education. (2018). Curriculum for the compulsory school, preschool class, and the recreation centre. Retrieved from https://www.skolverket.se/sitevision/proxy/publikationer/svid12_5dfee44715d35a5cdfa2899/55935574/wtpub/ws/skolbok/wpubext/trycksak/Blob/pdf3984.pdf?k=3984. Accessed 25 Feb 2019.

Starkweather, K. N. (2015). Politics and policy. In P. Williams, A. Jones, & C. Buntting (Eds.), The future of technology education: Contemporary issues in technology education (pp. 239–252). Singapore: Springer.

Steedle, J. T., & Ferrara, S. (2016). Evaluating comparative judgment as an approach to essay scoring. Applied Measurement in Education,29(3), 211–223.CrossRef

Strimel, G. (2014). Authentic education by providing a situation for student-selected problem-based learning. Technology and Engineering Teacher, 73(7), 8–18.

Strimel, G., & Grubbs, M. E. (2016). Positioning technology and engineering education as a key force in STEM education. Journal of Technology Education,27(2), 21–36.CrossRef

Strimel, G. J., Grubbs, M. E., & Wells, J. G. (2016). Engineering education: A clear decision. Technology and Engineering Teacher,76(4), 18–24.

Tarricone, P., & Newhouse, C. P. (2016). Using comparative judgement and online technologies in the assessment and measurement of creative performance and capability. International Journal of Educational Technology in Higher Education,13(1), 16. https://doi.org/10.1186/s41239-016-0018-x.CrossRef

Teknikföretagen, & Cetis, (2012). Teknikämnet i träda: teknikföretagens och CETIS rapport om teknikundervisningen i grundskolan. Stockholm: Teknikföretagen.

Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review,34, 273–286.CrossRef

van Daal, T., Lesterhuis, M., Coertjens, L., Donche, V., & De Maeyer, S. (2016). Validity of comparative judgement to assess academic writing: examining implications of its holistic character and building on a shared consensus. Assessment in Education: Principles, Policy & Practice, 26(1), 59–74. https://doi.org/10.1080/0969594X.2016.1253542.CrossRef

Whitehouse, C., & Pollitt, A. (2012). Using adaptive comparative judgement to obtain a highly reliable rank order in summative assessment. Retrieved on February 23, 2018 from https://cerp.aqa.org.uk/sites/default/files/pdf_upload/CERP_RP_CW_20062012_2.pdf. Accessed 25 Feb 2019.

Williams, P. J., & Kimbell, R. (2012). E-scape (special issue). International Journal of Technology and Design Education, 22(2), 265–270.CrossRef

Titel: Identifying design values across countries through adaptive comparative judgment
verfasst von: Scott Ronald Bartholomew
Emily Yoshikawa Ruesch
Eva Hartell
Greg J. Strimel
Publikationsdatum: 02.03.2019
Verlag: Springer Netherlands
Erschienen in: International Journal of Technology and Design Education / Ausgabe 2/2020
Print ISSN: 0957-7572
Elektronische ISSN: 1573-1804
DOI: https://doi.org/10.1007/s10798-019-09506-8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 2/2020

Participation and achievement in technology education: the impact of school location and socioeconomic status on senior secondary technology studies

Identity development in school makerspaces: intentional design

Hotspots and trends of technology education in the International Journal of Technology and Design Education: 2000–2018

Technological knowledge in early childhood education: provision by staff of learning opportunities

Designing with LEGO: exploring low fidelity visualization as a trigger for student behavior change toward idea fluency

An exploration of the variables contributing to graphical education students’ CAD modelling capability

Premium Partner