Top

Education and Information Technologies

Published in:

08-05-2020

A critical deconstruction of computer-based test application in Turkish State University

Author: Ömer Gökhan Ulum

Published in: Education and Information Technologies | Issue 6/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Artificial Intelligence (AI) is growing – as can be clearly observed not only from the rising recognition of assistance tools such as Siri (Apple) but also from the newly introduced Google Voiced Translator. Yet, some crucial benchmarks still have to be supplied before it can act as a proxy for a real instructor: imagination, creativity, and spontaneity. Automated assessment containing the use of AI is one of the recent education practices. It accelerates the time for exam grading, eliminates human prejudice, and is as precise as human assessors. However, it has encountered many criticisms in education community, in our case, English as foreign language (EFL) learning community. Therefore, this phenomenological inquiry examined Turkish EFL students’ and instructors’ conceptions on the Versant English Test (VET), an automated test of spoken and written language functioning by means of an AI software. Using semi-structured interview questions and a focus-group discussion, the study adopted a qualitative research design in order to collect the required data. The findings show that EFL university students developed negative attitudes towards VET and that VET is not a reliable and valid test because same questions were observed to have appeared in the computer-based test. In addition, copying and pasting similar sentences produced better results, which decreased the validity and reliability of the test. Another important finding was that the test was reported to have measured only their memory skills but not their language skills. Besides, the curriculum was totally incongruent with the content of the test, which caused a severe washback in EFL learners.

previous article Reflection through drama and concept maps for preservice teacher education in information communication technologies

next article “I felt like I was missing out on something”: an evaluation of using remote technology in the classroom

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Au, W. (2013). What’sa nice test like you doing in a place like this. Rethinking Schools, 27(4), 22–27.

Bach, J. (2020). When artificial intelligence becomes general enough to understand itself. Commentary on Pei Wang’s paper “on defining artificial intelligence”. Journal of Artificial General Intelligence, 11(2), 15–18.

Barr, A., & Feigenbaum, E. A. (Eds.). (2014). The handbook of artificial intelligence (Vol. 2). Burlington: Morgan Kaufmann Publications.

Bernstein, J., & Cheng, J. (2007). Logic and validation of fully automatic spoken English test. In V. M. Holland & F. P. Fisher (Eds.), The path of speech technologies in computer assisted language learning: From research toward practice (pp. 174–194). New York: Routledge.

Bettis, R. A., & Hitt, M. A. (1995). The new competitive landscape. Strategic Management Journal, 16(S1), 7–19.

Bin, Y., & Mandal, D. (2019). English teaching practice based on artificial intelligence technology. Journal of Intelligent & Fuzzy Systems, (preprint), 37(3), 1–11.

Booth, S. (1997). On phenomenography, learning and teaching. Higher Education Research & Development, 16(2), 135–158.

Bowden, J. (2000). Phenomenography. Melbourne: RMIT University Press.

Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment (Vol. 17). New York: Sage publications.

Chapelle, C. A., & Douglas, D. (2006). Assessing language through computer technology. Stuttgart: Ernst Klett Sprachen.

Chapelle, C. A. (2010). Research and practice: A look at issues in technology for second language learning. Language Learning & Technology, 14(3), 27–30.

Cheng, L., Klinger, D., Fox, J., Doe, C., Jin, Y., & Wu, J. (2014). Motivation and test anxiety in test performance across three testing contexts: The CAEL, CET, and GEPT. Tesol Quarterly, 48(2), 300–330.

Clark, P., & Etzioni, O. (2016). My computer is an honor student—But how intelligent is it? Standardized tests as a measure of AI. AI Magazine, 37(1), 5–12.

Dascalu, M., Chioasca, E. V., & Trausan-Matu, S. (2008). ASAP-an advanced system for assessing chat participants. In International Conference on Artificial Intelligence: Methodology, Systems, and Applications (pp. 58–68). Berlin, Heidelberg: Springer.

Devedžić, V. (2004). Web intelligence and artificial intelligence in education. Educational Technology & Society, 7(4), 29–39.

Downey, R., Farhady, H., Present-Thomas, R., Suzuki, M., & Van Moere, A. (2008). Evaluation of the usefulness of the versant for English test: A response. Language Assessment Quarterly, 5(2), 160–167.

Duarte, M., Shiplacoff, D., Simon, G., & Raju, R. (2012). U.S. Patent No. 8,291,348. Washington, DC: U.S. Patent and Trademark Office.

Eagly, A. H., & Chaiken, S. (1993). The psychology of attitudes. San Diego: Harcourt Brace Jovanovich College Publishers.

Elder, C., Iwashita, N., & McNamara, T. (2002). Estimating the difficulty of oral proficiency tasks: what does the test-taker have to offer? Language Testing, 19(4), 347–368.

Fan, J., & JI, P. (2014). Test candidates' attitudes and their test performance: The case of the Fudan English Test. University of Sydney Papers in TESOL, 9(1–35).

Freedman, R. A., Klein, R. A., & Lederman, J. (1994). Artificial intelligence in the capital markets: State-of-the-art applications for institutional investors, bankers and traders. New York: McGraw-Hill Publishing.

Gartner Group (2016). Gartner’s top 10 strategic technology trends for 2017. Retreived on the 20^th of 2019 from http://www.gartner.com/smarterwithgartner/gartners-top-10-technology-trends-2017/.

Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature, 521(7553), 452–459.

Ginther, A., & McIntosh, K. (2018). Language testing and assessment. In The Palgrave handbook of applied linguistics research methodology (pp. 845–867). London: Palgrave Macmillan.

Giroux, H. (2002). Neoliberalism, corporate culture, and the promise of higher education: The university as a democratic public sphere. Harvard Educational Review, 72(4), 425–464.

Giroux, H. A. (2020). On critical pedagogy. London: Bloomsbury Publishing.

Goldberg, Y. (2017). Neural network methods for natural language processing. Synthesis Lectures on Human Language Technologies, 10(1), 1–309.

Greene, D., Hoffmann, A. L., & Stark, L. (2019). Better, nicer, clearer, fairer: A critical assessment of the movement for ethical artificial intelligence and machine learning. In Proceedings of the 52nd Hawaii International Conference on System Sciences.

Hausser, R. (2012). Computation of language: An essay on syntax, semantics and pragmatics in natural man-machine communication. Berlin: Springer-Verlag Publications.

Heckman, J. J., & Rubinstein, Y. (2001). The importance of noncognitive skills: Lessons from the GED testing program. American Economic Review, 91(2), 145–149.

Hirschberg, J., & Manning, C. D. (2015). Advances in natural language processing. Science, 349(6245), –261, 266.

Jones, M. (1985). Applications of artificial intelligence within education. Computers & Mathematics with Applications, 11(5), 517–526.

Kang, O., Thomson, R., & Moran, M. (2019). The effects of international accents and shared first language on listening comprehension tests. TESOL Quarterly, 53(1), 56–81.

Kasabov, N. (2015). Evolving connectionist systems: From Neuro-fuzzy-, to spiking-and Neuro-genetic. In Springer handbook of computational intelligence (pp. 771–782). Berlin, Heidelberg: Springer.

Keerthiwansha, N. B. S. (2018). Artificial intelligence education (AIEd) in English as a second language (ESL) classroom in Sri Lanka. Artificial Intelligence, 6(1), 31–36.

Kim, N. Y. (2019). A study on the use of artificial intelligence Chatbots for improving English grammar skills. Journal of Digital Convergence, 17(8), 37–46.

Kovács, G., & Kot, S. (2016). New logistics and production trends as the effect of global economy changes. Polish Journal of Management Studies, 14, 115–126.

Landauer, T. K. (2003). Automatic essay assessment. Assessment in education: Principles, policy & practice, 10(3), 295–308.

Letier, E., & Van Lamsweerde, A. (2002). Deriving operational software specifications from system goals. ACM SIGSOFT Software Engineering Notes, 27(6), 119–128.

Lu, X. (2019). An empirical study on the artificial intelligence writing evaluation system in China CET. Big Data, 7(2), 121–129.MathSciNet

Luckin, R. (2017). Towards artificial intelligence-based assessment systems. Nature Human Behaviour, 1(28), 1–3.

Ma, Z. (2019). Application and practice of artificial intelligence in maker education and teaching. Advances in Higher Education, 3(2), 105–108.

McTear, M. F. (2016). The rise of the conversational interface: A new kid on the block? In International workshop on future and emerging trends in language technology (pp. 38–49). Cham: Springer.

Nichols, J., Myers, B. A., & Litwack, K. (2004). Improving automatic interface generation with smart templates. In Proceedings of the 9th international conference on intelligent user interfaces (pp. 286-288). ACM.

Nilsson, N. J. (2014). Principles of artificial intelligence. Massachusetts: Morgan Kaufmann Publications.

Ockey, G. J. (2009). Developments and challenges in the use of computer-based testing for assessing second language ability. The Modern Language Journal, 93, 836–847.

Ordem, E. (2017). Participatory approach in EFL classes. Electronic Turkish Studies, 12(3), 505–516.

Ordem, E., & Yükselir, C. (2017). Views of Turkish EFL instructors on critical pedagogy. Electronic Turkish Studies, 12(14), 285–294.

Patton, M. Q. (2002). Two decades of developments in qualitative inquiry: A personal, experiential perspective. Qualitative Social Work, 1(3), 261–283.

Plonsky, L. (2015). Quantitative considerations for improving Replicability in CALL and applied linguistics. CALICO Journal, 32(2), 232–244.

Pollach, I. (2012). Taming textual data: The contribution of corpus linguistics to computer-aided text analysis. Organizational Research Methods, 15(2), 263–287.

Poorsoti, S., & Asl, H. D. (2015). Iranian Candidates' attitudes toward TOEFL iBT. Journal of Applied Linguistics and Language Research, 2(8), 263–272.

Quesada, J. F., Mateos, F. J. M., & Soto, T. L. (Eds.). (2017). Future and emerging trends in language technology. Machine learning and big data: Second International Workshop, FETLT 2016, Seville, Spain, November 30–December 2, 2016, revised selected papers (Vol. 10341). Springer.

Reiter, E., & Dale, R. (1997). Building applied natural language generation systems. Natural Language Engineering, 3(1), 57–87.

Reiter, E., & Dale, R. (2000). Building natural language generation systems. Cambridge: Cambridge University Press.

Riggio, R. E. (1986). Assessment of basic social skills. Journal of Personality and Social Psychology, 51(3), 649.

Russell, S. J., & Norvig, P. (2016). Artificial intelligence: a modern approach. New York: Pearson Education Limited.

Sang, E. F. T. K., & Nerbonne, J. (1999, June). Learning the logic of simple phonotactics. In International conference on learning language in logic (pp. 110–124). Springer, Berlin, Heidelberg.

Shin, M. H. (2018). How to use artificial intelligence in the English language learning classroom. Indian Journal of Public Health Research & Development, 9(9), 557–561.

Sproat, R. W. (1997). Multilingual text-to-speech synthesis. New York: KLUWER academic publishers.

Underwood, J. H. (1984). Linguistics, computers, and the language teacher. A communicative approach. New York: Newbury House Publishers.

Underwood, J. (2017). Exploring AI language assistants with primary EFL students. CALL in a climate of change: Adapting to turbulent global conditions–short papers from EUROCALL, 317-321.

Van der Spek, R., & Spijkervet, A. (1997). Knowledge management: dealing intelligently with Knowledge. In J. Liebowitz & L. C. Wilcox (Eds.), Knowledge management and its integrative elements (pp. 31-58). NewYork: CRC Press.

Vicente, M., & Lloret, E. (2016). Exploring flexibility in natural language generation through discursive analysis of new textual genres. In International workshop on future and emerging trends in language technology (pp. 98–109). Cham: Springer.

Wacker, P., & Joscelyne, A. (2016). Improving collaboration between the European language technology industry and research: A new framework for supply and demand. In international workshop on future and emerging trends in language technology (pp. 61-69). Springer, Cham.

Wu, C. W. (2009). Decision-making in testing process performance with fuzzy data. European Journal of Operational Research, 193(2), 499–509.MathSciNetMATH

Title: A critical deconstruction of computer-based test application in Turkish State University
Author: Ömer Gökhan Ulum
Publication date: 08-05-2020
Publisher: Springer US
Published in: Education and Information Technologies / Issue 6/2020
Print ISSN: 1360-2357
Electronic ISSN: 1573-7608
DOI: https://doi.org/10.1007/s10639-020-10199-z

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 6/2020

The use of ICT in teaching geometry in primary school

Fostering intuitive competence in L2 for a better performance in EAP writing through fraze.it in aTurkish context

Information technology teachers’ perceptions of the benefits and efficacy of using online communities of practice when teaching computer skills classes

Identifying quality educational apps: Lessons from ‘top’ mathematics apps in the Apple App store

Correction to: Classroom dialogue and digital technologies: a scoping review

An outcome of expert-oriented digitalization of university processes

Premium Partner