Clarificative evaluation of elementary level grammar program of a tertiary level preparatory class in Turkey

Highlights

• The Delphi Technique helped to identify the implicit program outcomes and activities.

• Fall Semester short-term outcomes were perceived as “highly achievable”.

• Fall Semester long-term outcomes were perceived as “achievable”.

• The instructors were not sure about the achievability of Spring Semester outcomes.

• The unstructured activities implemented scarcely were found to be the most effective.

Abstract

Although the general goals of the elementary level grammar program in a Language Preparatory Class at a Turkish university are stated, the outcomes bridging those goals and activities are not explicit, which hinders evaluation. Accordingly, this mixed-method case study, which includes several stakeholders, aims to investigate (1) underlying outcomes of the elementary level grammar program regarding assisting writing skills and learning activities, and (2) perceived achievability of the outcomes and the effectiveness of the learning activities to realize the outcomes. Utilizing the Delphi technique, an opinionnaire based on the Delphi technique and interview methods, the underlying long-term and short-term outcomes, activities, perceived achievability of the long-term and short-term outcomes, effectiveness of activities and other factors in the elementary level grammar program were identified and reported.

Introduction

Developments in recent decades of language teaching propose that language programs be prepared for a certain group of teachers and learners with specific goals in a particular context (Kumaravadivelu, 2001). This makes program evaluation, providing context-bound answers to common questions in the field, much more important in language education.

The position of grammar programs in language curricula is one of the issues that has been questioned for a long time. Some argue that provision of enough comprehensible input, which means giving a manageable amount of new knowledge based on what is already known, makes it possible to reach higher levels of language proficiency without explicit grammar instruction. Thus, language learning that occurs in a natural setting, where what is new is learnt through the filter of what is known or apparent from background information, should be channeled into the classroom to generate similar results (Rodrigo, Krashen, & Gribbons, 2004). On the other hand, some research reveals that when grammar instruction is given explicitly, the speed of learning and the proficiency level attained differ positively in input-poor settings, even though the sequence of learning does not change (Ellis, 2001).

No matter what type of grammar instruction is adopted, the rationale for including grammar instruction in input-poor settings, where the exposure to L2 is often limited to the classroom environment, is supported by several factors. Firstly, grammar instruction accelerates learning. In other words, form-focused instruction, which means planning classes in a way that students pay attention to grammatical structures, influences language learning positively (Ellis, 2001; Lightbown & Spada, 2006; Spada & Lightbown, 2008; Loewen et al., 2009).

Age is another factor that affects processing of explicit instruction. While younger children can acquire the language input in their environment easily, isolated grammar instruction facilitates the learning of older children, teenagers, and adults (DeKeyser, 2000), which may also affect young adults’ attitude toward grammar instruction positively (Brown, 2009; Manley & Calk, 1997). Therefore, designing isolated grammar programs or focusing on form for longer periods of time may be appropriate in higher education (HE). Yet it should also be remembered that grammar instruction should cater to the communicative needs of learners rather than imposing metalinguistic structures without practice (Muncie, 2002).

Thus, striking a balance in language curricula is quite significant in the program design and evaluation process. However, how this balance is provided and managed is often determined by several factors including student characteristics, time limitations, and syllabus load. For example, in HE, where the medium of instruction is English, the low entry proficiency levels of students may create a burden (Kilickaya, 2006; Dearden & Macaro, 2016; Ashraf, 2018). In such cases, explicit instruction of form may be favorable since it addresses the needs of adult language learners better and accelerates the speed of learning, which is usually a priority for language preparatory programs in Turkish HE because of time limitations. Yet, it should also be remembered that cooperation between different staff executing the program (Richards, 2010) and the collection of achievement data (Hamilton et al., 2009; Kane et al., 2011) gain importance in building a balanced and strong curriculum involving a grammar component.

Whether isolated or integrated, grammar instruction seems to have its own benefits (Spada & Lightbown, 2008). Thus it may be more rational to leave the decision to the program's stakeholders when choosing an appropriate form of grammar instruction, rather than imposing a certain dogma. Doing this, on the other hand, requires undertaking program evaluation (PE) projects, since only then is it possible to fully reveal the understandings and preferences of the stakeholders that design, deliver, and receive the program. It was with this objective in mind that the researchers involved in this study conducted the current evaluation.

Evaluation is associated with identification of the internal merit, relative value and significance of a program, which are usually contingent on its context and stakeholders (Scriven, 1999). Stake (1967) claims that the purpose of evaluation is to discover the stakeholders’ perspective on the program's values and merits without any pressure or censorship. Besides unveiling different perspectives without searching for consensus, PE should help institutions with their decision-making by contributing to the program's development and maintenance rather than assessing programs against prescribed criteria (Preskill & Torres, 1999). From that aspect, PE should assist institutions in avoiding erroneous decisions by determining the significance of a program (Fitzpatrick, Sanders, & Worthen, 1997), set well-described goals that are realistic for the context (Bjerke & Renger, 2017), reveal the level of consistency between different stakeholders’ assumptions and the actual outcomes of the program (El Soufi & See, 2019; Tang, 2011; Zvoch, 2019), and indicate the program's fidelity by inquiring about the effect of the implementation on the achievement of the program's logic (Holliday, 2014; Newton, 2016).

In Turkish HE, the departments where at least 30 % of the departmental courses are taught through the medium of English are obliged to provide students failing to pass the exemption exam with a one-year English Preparatory Class (PC) (Yabancı Dilde Eğitim, 2017), which aims at increasing the students’ English proficiency level at least to the Common European Framework (CEFR) B1 proficiency level (CEFR, 2017), whereby they are supposed to be able to cope with the academic language requirements of their department. Although the aim of the PC curricula at different universities in Turkey helps their students reach B1 level of proficiency, the decision of whether to add separate grammar or vocabulary programs to curricula, and how to teach skills or structures, varies. Thus, evaluation of each of these programs can guide critical decisions that need to be taken regarding how the PC curricula should be designed.

Several PC language curricula in Turkish HE have been evaluated focusing either on the whole curriculum (Coşkun, 2013; Gerede, 2005; Karataş, 2007; Köse, 2012; Özkanal & Hakan, 2010; Tunc, 2010) or on only one program component (Yücel, 2009; Çetinavcı & Topkaya, 2012). In Tunç’s study, the evaluation of a PC curriculum conducted through the lenses of 406 students and 12 instructors using the CIPP model involved the evaluation of context, input, process, and product of a program in order to improve it through aiding the decision-making processes (Stufflebeam, 2003). Results indicated that the students and instructors disagreed about the sufficiency of the materials but agreed that the students were not as competent in listening-speaking skills as they were in reading–writing skills. In Coşkun’s PE of a PC at a Turkish state university conducted with 381 students and 22 instructors using mixed-methodology, both of the participant groups were reported to have complained about the grammar instruction overwhelming the four skills and a deficiency in communicative activities because of the syllabus load. Coşkun even suggested lessening the time allocated for grammar instruction. As results suggest, the grammar programs under investigation did not cater to communicative needs, as emphasized by Muncie (2002), despite their dominance in the PC curricula. In another study conducted at the same Turkish university PC as this study, with 36 students and five instructors mainly using qualitative methods, Çetinavcı and Topkaya (2012) compared two regimes of grammar instruction accompanying the main course, which integrated the four skills. The first regime had its own syllabus independent of the main course content; while the second regime involved a program completely contingent on the main course. The results indicated that the program's stakeholders found the integrated grammar instruction more effective in bringing about the intended learning outcomes. Köse’s study, which was conducted in the same institution as Çetinavcı and Topkaya’s at about the same time, highlighted the students’ weak writing skills and inability to implement their grammar and vocabulary knowledge in productive skills, which pointed to a need to improve the PC language curriculum regarding synchrony and integration.

A close examination of the above studies reveals that most of them evaluated grammar programs in relation to other courses but only in terms of their relative importance in the whole curriculum. Only Çetinavcı and Topkaya’s study directly inquired into the effects of asynchrony between the main course and the grammar program. However, none of these studies questioned the direct relationship between grammar programs and productive skills, which creates a gap in the PC evaluation literature of Turkish HE. Also, two aforementioned PE studies conducted on the site of the current study indicated the poor impact of the grammar program on productive skill accuracy and the language variety of the students, because no formal PE had been conducted there for about six years.

In the light of the considerations above, this study evaluated an elementary grammar program (EGP) run in a tertiary level English PC where the purpose of the program is to support productive skills, specifically writing. Thus, the following research questions guided the study:

• What outcomes is the EGP supposed to generate regarding assisting students’ writing skills?

• What learning activities are utilized to bring about the intended outcomes?

• Are the outcomes of the EGP achievable with the learning activities? If so, which learning activities are perceived to be effective in achieving the learning outcomes?

• What other factors are considered to influence the success of the EGP?