Top

Education and Information Technologies

Published in:

22-07-2015

A design science research methodology for developing a computer-aided assessment approach using method marking concept

Authors: Hussein Genemo, Shah Jahan Miah, Alasdair McAndrew

Published in: Education and Information Technologies | Issue 6/2016

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Assessment has been defined as an authentic method that plays an important role in evaluating students’ learning attitude in acquiring lifelong knowledge. Traditional methods of assessment including the Computer-Aided Assessment (CAA) for mathematics show limited ability to assess students’ full work unless multi-step questions are sub-divided into sub questions. This issue persisted significant drawback especially within the notion of method marking approach. To address this issue, the aim of the study is to develop a methodological framework that will create an information and communications technology (ICT) artefact prototype. The prototype (termed as method marking assessment (MMA) artefact) implements a method-marking assessment concept to assess through multi-step questions. Extensive literature reviews have revealed that there are features in common between complex-problem solution characteristics and multi-steps questions assessment using ICT; therefore complex problems paradigm is used in the study for developing the MMA prototype.

previous article A naive bayes approach for converging learning objects with open educational resources

next article Eye tracking system for enhanced learning experiences

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Angeli, C. (2010). Diagnostic expert systems: from expert’s knowledge to real—time systems. Advanced Knowledge Based Systems: Model, Applications & Search, 1, 50–73.

Ashton, H. S., Beevers, C. E., Korabinski, A. A., & Youngson, M. A. (2006). Incorporating partial credit in computer-aided assessment of mathematics in secondary education. British Journal of Educational Technology, 37(1), 93–119. doi:10.1111/j.1467-8535.2005.00512.x.CrossRef

Balcombe, A., Brennan, M. & Everiss, L. (2011). Formative Learning: A Description of an Assessment Framework in Initial Teacher Education. In: M. Hodis & S. Kaiser (Eds.), Proceedings of the Symposium on Assessment and Learner Outcomes, Victoria University, (pp. 25–40). Wellington, New Zealand.

Bayazit, N. (2004). Investigating design: a review of forty years of design research. Design Issues, 20(1), 16–29.CrossRef

Beevers, C., Wild, D., McGuire, G., Fiddes, D., & Youngson, M. (1999). Issues of partial credit in mathematical assessment by computer. Research in Learning Technology, 7(1), 26–32.

Black, P., & Wiliam, D. (2005). Lessons from around the world: how policies, politics and cultures constrain and afford assessment practices. Curriculum Journal, 16(2), 249–261. doi:10.1080/09585170500136218.CrossRef

Bloxham, S., & Boyd, P. (2007). Developing Effective Assessment In Higher Education: A Practical Guide: A Practical Guide, McGraw-Hill Education, UK.

Borgman, C. L. (2012). The conundrum of sharing research data. Journal of the American Society for Information Science and Technology, 63(6), 1059–1078.CrossRef

Boyd, N. S. (2010). Domain vocabulary. Retrieved 2013/12/19, from http://educery.com/educe/patterns/domain-vocabulary.html.

Chatterjee, S. (2010). Design research in information systems: theory and practice (Vol. 22). Springer.

Creswell, J. (2003). Research design: Qualitative, quantitative, and mixed methods approaches. Boston: SAGE Publications.

Cross, N. (1984). Developments in design methodology. Chichester: John Wiley & Sons Ltd.

Davis, D. (2007). Complex software problem solving by means of abstractive techniques. Paper presented at the Proceedings of the 11th international conference on Computer aided systems theory, Las Palmas de Gran Canaria, Spain.

de Jong, T. (2014). Emerging representation technologies for problem solving. In: Spector, J. M., Merrill, M. D., Elen, J. & Bishop, M. J. (Eds.), Handbook of Research on Educational Communications and Technology (pp. 809–816). New York: Springer.

Faste, T., & Faste, H. (2012). Demystifying “design research”: design is not research, research is design, IDSA Education Symposium.

Fischer, A., Greiff, S., & Funke, J. (2012). The process of solving complex problems. The Journal of Problem Solving, 4(1), 19–42. doi:10.7771/1932-6246.1118.

Funke, J. (2010). Complex problem solving: a case for complex cognition? Gov’t review. Cognitive Processing, 11(2), 133–142.

Gears, C. (2005). Classroom assessment: minute by minute, day by day. Assessment, 63(3), 19–24.

Grosan, C., & Abraham, A. (2011). Rule-based expert systems. Intelligent Systems. Springer.

Hevner, A. R. (2007). A three cycle view of design science research. Scandinavian journal of information systems, 19(2), 87–92.

Hevner, A., & Chatterjee, S. (2010). Design research in information systems: Theory and practice. New York: Springer Science & Business Media.CrossRef

Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75–105.

Hwang, G.-J., Chen, C.-Y., Tsai, P.-S., & Tsai, C.-C. (2011). An expert system for improving web-based problem-solving ability of students. Expert Systems with Applications, 38, 8664–8672.CrossRef

Jaques, P. A., Seffrin, H., Rubi, G., Morais, F. D., Guilardi, C., Bittencourt, I. I., & Isotani, S. (2013). Rule-based expert systems to support step-by-step guidance in algebraic problem solving: the case of the tutor PAT2Math. Expert Systems with Applications, 40(14), 5456–5465.CrossRef

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: a research paradigm whose time has come. Educational Researcher, 33(7), 14–26.CrossRef

Jones, I. S. (2008). Computer-aided assessment questions in engineering mathematics usingMapleTA®. International Journal of Mathematical Education in Science and Technology, 39(3), 341–356. doi:10.1080/00207390701734523.CrossRef

Kaidar, S. M., Hussain, R. I., Bohani, F. A., Sahran, S., Binti Zainuddin, N., Ismail, F., Thanabalan, J., Kalimuthu, G., & Abdullah, S. N. H. S. (2013). Brain tumor treatment advisory system. Soft Computing Applications and Intelligent Systems. Springer.

Kamel Boulos, M. N. (2012). Expert system shells for rapid clinical decision support module development: an ESTA demonstration of a simple rule-based system for the diagnosis of vaginal discharge. Healthcare Information Research, 18, 252–258.CrossRef

Khan, F. S., Razzaq, S., Irfan, K., Maqbool, F., Farid, A., Illahi, I., & Ul Amin, T. (2008). Wheat: A Web-based expert system for diagnosis of diseases and pests in Pakistani wheat. Proceedings of the World Congress on Engineering, Oct 2008, 2–4.

Kirsh, D. (2010). Thinking with external representations. AI & SOCIETY, 25(4), 441–454. doi:10.1007/s00146-010-0272-8.CrossRef

Lawson, D. (2012). Computer-aided assessment in mathematics: panacea or propaganda? International Journal of Innovation in Science and Mathematics Education (formerly CAL-laborate International), 9(1), 6–12.

Livne, N. L., Livne, O. E., & Wight, C. A. (2007). Can automated scoring surpass hand grading of students’ constructed responses and error patterns in mathematics. MERLOT Journal of Online Learning and Teaching, 3(3), 295–306.

March, S. T., & Smith, G. F. (1995). Design and natural science research on information technology. Decision Support Systems, 15(4), 251–266.CrossRef

Miah, S. J., Kerr, D., & Gammack, J. (2009). A methodology to allow rural extension professionals to build target specific expert systems for Australian rural business operators. Expert Systems with Applications, 36(1), 735–744.CrossRef

Miah, S. J., Kerr, D., & Von-Hellens, L. (2014). A collective artefact design of decision support systems: design science research perspective. Information Technology & People, 27(3), 259–279.CrossRef

Offermann, P., Blom, S., Schönherr, M., & Bub, U. (2010). Artifact types in information systems design science–a literature review. In: Winter, R., Zhao, J. L. & Aier, S. (Eds.), Global perspectives on design science research (pp. 77–92). Springer Berlin Heidelberg.

Österle, H., Becker, J., Frank, U., Hess, T., Karagiannis, D., Krcmar, H., et al. (2011). Memorandum on design-oriented information systems research. European Journal of Information Systems, 20(1), 7–10.CrossRef

Palm, T. (2008). Performance assessment and authentic assessment: a conceptual analysis of the literature. Practical Assessment, Research & Evaluation, 13(4), 1–11.

Passmore, T., Brookshaw, L., & Butler, H. (2011). A flexible, extensible online testing system for mathematics. Australasian Journal of Educational Technology, 27(6), 896–906.CrossRef

Pereira, R., Almeida, R., & da Silva, M. M. (2013). How to generalize an information technology case study. In: Vom Brocke, J., Hekkala, R., Ram, S. & Rossi, M. (Eds.), Design science at the intersection of physical and virtual design (pp. 150–164). Springer Berlin Heidelberg.

Price, M., Carroll, J., O’Donovan, B., & Rust, C. (2011). If I was going there I wouldn’t start from here: a critical commentary on current assessment practice. Assessment & Evaluation in Higher Education, 36(4), 479–492.CrossRef

Sangwin, C. (2012). Computer aided assessment of mathematics using STACK. Paper presented at the Proceedings of ICME.

Sangwin, C., Cazes, C., Lee, A., & Wong, K. L. (2010). Micro-level automatic assessment supported by digital technologies. Hoyles, C. & Lagrange, J.-B. (Eds.), Mathematics Education and Technology-Rethinking the Terrain, (pp. 227-250). Springer USA. 13, 227–250.

Shepard, L. A. (2005). Linking formative assessment to scaffolding. Educational Leadership, 63(3), 66–70.

Sonnleitner, P., Keller, U., Martin, R., & Brunner, M. (2013). Students’ complex problem-solving abilities: their structure and relations to reasoning ability and educational success. Intelligence, 41(5), 289–305. doi:10.1016/j.intell.2013.05.002.CrossRef

Vaishnavi, V. K., & Kuechler Jr, W. (2007). Design science research methods and patterns: innovating information and communication technology. CRC Press.

Van den Broeck, J., Brestoff, J. R., & Chhagan, M. (2013). Maintaining data integrity. In: Van den Broeck, J. & Brestoff, J. R. (Eds.), Epidemiology: principles and practical guidelines (pp. 379–388). UK: Springer.

Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3–14. doi:10.1016/j.stueduc.2011.03.001.CrossRef

Wüstenberg, S., Greiff, S., & Funke, J. (2012). Complex problem solving—more than reasoning? Intelligence, 40(1), 1–14. doi:10.1016/j.intell.2011.11.003.CrossRef

Yeo, R. K., & Marquardt, M. J. (2012). Complex problem solving through action learning: implications for human resource development. International Journal of Human Resources Development and Management, 12(4), 258–273.CrossRef

Title: A design science research methodology for developing a computer-aided assessment approach using method marking concept
Authors: Hussein Genemo
Shah Jahan Miah
Alasdair McAndrew
Publication date: 22-07-2015
Publisher: Springer US
Published in: Education and Information Technologies / Issue 6/2016
Print ISSN: 1360-2357
Electronic ISSN: 1573-7608
DOI: https://doi.org/10.1007/s10639-015-9417-1

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 6/2016

Relationship between gender identity, perceived social support for using computers, and computer self-efficacy and value beliefs of undergraduate students

Developing online learning resources: Big data, social networks, and cloud computing to support pervasive knowledge

Student observations: Introducing iPads into university classrooms

Blending an Android development course with software engineering concepts

Social and emotional learning around technology in a cross-cultural, elementary classroom

Potential use of Augmented Reality in LIS education

Premium Partner