Skip to main content
Erschienen in: Educational Assessment, Evaluation and Accountability 1/2020

13.12.2019

Enhanced use of educational accountability data to monitor educational progress of Australian students with focus on Indigenous students

verfasst von: Joy Cumming, Harvey Goldstein, Kirstine Hand

Erschienen in: Educational Assessment, Evaluation and Accountability | Ausgabe 1/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In Australia, under the National Assessment Plan, educational accountability testing in literacy and numeracy (NAPLAN) is undertaken with all students in Years 3, 5, 7 and 9 to monitor student achievement and inform policy. However, the extent to which these data have been analysed to report student progress is limited. This article reports a study analysing Year-3 and Year-5 NAPLAN reading and numeracy data, school and student information for a single student cohort from Queensland, Australia, to examine student achievement and progress. The analyses use longitudinal multilevel modelling, incorporating an enhanced approach for missing data imputation, given that such data frequently involve large amounts of missing data and failure to account properly for such missing data may bias interpretations of analyses. Further, statistical adjustments to deal with the impact of measurement error, an aspect not previously addressed in such analyses of data, are undertaken. A special focus of analyses is achievement of Australian Indigenous and non-Indigenous students. International and national data demonstrate a considerable achievement gap between these students. “Closing the gap” is a core Australian education equity policy, with NAPLAN data used as a primary indicator of policy impact. Overall, analyses indicate greater understanding of student progress for all students is available from Australian data if appropriate analyses are undertaken. However, analyses also demonstrate not only that the gap between Australian Indigenous and non-Indigenous student progress increases as they move through school but also diversity of achievement within the Indigenous student cohort. Implications for policy are considered.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
1
Students who have significant disability or are unable to “access” the NAPLAN tests may be exempted. Students who are ill are noted as absent. Students may be withdrawn if parents or students do not wish to participate. Reported statistics (means and percentages) include statistical imputation for absent or withdrawn students; exempt students are not included in the population or calculation of results (ACARA 2017). In 2013 for Year 5 Reading, Queensland, participation rate was 95.5% of students, with 2.3% absent, 2.7% withdrawn and 1.7% exempt (ACARA 2013), with similar rates for Numeracy.
 
2
Australian school years are referred to as Year levels, rather than Grades, as students progress through schooling with their age cohort, rather than a “grade” level of performance.
 
3
Approximately two thirds of Australian students attend government schools and one third attend non-government schools (20% Catholic schools, 10% other independent schools).
 
4
Australia is a federation of six states (New South Wales, Queensland, South Australia, Tasmania, Victoria, Western Australia) and two territories (Australian Capital Territory, Northern Territory).
 
5
For example, the Action Plan (2010–2014) stated: “Reform directions are detailed in the National Indigenous Reform Agreement, the National Education Agreement, the Early Childhood Education National Partnership, the Indigenous Early Childhood Development National Partnership, Remote Service Delivery National Partnership, the Smarter Schools - Improving Teacher Quality National Partnership, the Smarter Schools - Low Socio-economic Status School Communities National Partnership and the Smarter Schools - Literacy and Numeracy National Partnership, the Youth Achievement and Transitions National Partnership and other agreements. Commitments made in these national partnerships and agreements are brought together in the Plan with a number of new and continuing complementary measures to close the gap between the educational outcomes of Aboriginal and Torres Strait Islander students and other students.” (MCEECDYA 2009, p. 4)
 
6
In 2013, relevant to the data analysed, the proportion of the Australian population of 23 million in each state and territory was NSW, 32%; Vic, 25%; Qld, 20%; WA, 11%; SA, 7%; Tas, 2%; ACT, 2%; and NT, 1% (ABS, 2013a).
 
7
Between Years 3 and 5, approximately 20% of students change schools. Unfortunately, we have no information about when these changes take place so that accounting for exposure to two or, indeed, more schools is not possible and we do not pursue this further. Goldstein et al. (2007) suggest that while a failure to take account of movement between schools affects the relative proportions of variance at school and student level, the effects on the remaining coefficients are small.
 
8
While NAPLAN includes full cohorts in testing, rather than sampling, of students, NAPLAN technical reports note that sampling error is included at student and school level in order “to make inference about the educational systems each year and not about the specific student cohorts” within a year, and also to take account of “a certain amount of non-response” (ACARA 2016b).
 
9
As noted, some 20% of students change school between Year 3 and Year 5 but this does not appear to be associated with progress between these ages. Fitting a cross-classified model to study the relative effect of the Year 3 school and Year 5 school on progress is of interest and a topic for further research.
 
10
Both for Reading and Numeracy, the estimated Level 2 residuals uj, the “school effects”, are little changed after adjustment for measurement error. The adjusted and unadjusted estimates are correlated 0.99 with similar standard errors. We have also fitted models where the coefficients of the Year 3 score and Indigenous status vary across schools. There is some evidence that the (random) coefficients do indeed vary across schools. However, the corresponding model fixed effects do not change markedly when such a model is fitted. Thus, while when comparing schools it would be important to take such effects into account, for present purposes, we do not present this analysis.
 
Literatur
Zurück zum Zitat ACIL Allen Consulting with La Trobe University, & KPA, P. (2014). Evaluation of the Aboriginal and Torres Strait Islander Education Action Plan 2010–2014: Final evaluation report. Brisbane: ACIL Allen Consulting. ACIL Allen Consulting with La Trobe University, & KPA, P. (2014). Evaluation of the Aboriginal and Torres Strait Islander Education Action Plan 2010–2014: Final evaluation report. Brisbane: ACIL Allen Consulting.
Zurück zum Zitat Australian Curriculum, Assessment and Reporting Authority. (2013). National Assessment Program—Literacy and Numeracy. Achievement in Reading, Persuasive Writing, Language Conventions and Numeracy: National report for 2013. Sydney, NSW: ACARA. Australian Curriculum, Assessment and Reporting Authority. (2013). National Assessment Program—Literacy and Numeracy. Achievement in Reading, Persuasive Writing, Language Conventions and Numeracy: National report for 2013. Sydney, NSW: ACARA.
Zurück zum Zitat Australian Curriculum, Assessment and Reporting Authority (2014). National Assessment Program—Literacy and Numeracy. Technical report. 2013. Sydney, NSW: ACARA. Australian Curriculum, Assessment and Reporting Authority (2014). National Assessment Program—Literacy and Numeracy. Technical report. 2013. Sydney, NSW: ACARA.
Zurück zum Zitat Australian Curriculum, Assessment and Reporting Authority. (2016b). NAPLAN 2016 technical report. Sydney: ACARA. Australian Curriculum, Assessment and Reporting Authority. (2016b). NAPLAN 2016 technical report. Sydney: ACARA.
Zurück zum Zitat Australian Curriculum, Assessment and Reporting Authority (2017). National Assessment Program—Literacy and Numeracy. Achievement in reading, persuasive writing, language conventions and numeracy: National report for 2017. Sydney, NSW: ACARA. Australian Curriculum, Assessment and Reporting Authority (2017). National Assessment Program—Literacy and Numeracy. Achievement in reading, persuasive writing, language conventions and numeracy: National report for 2017. Sydney, NSW: ACARA.
Zurück zum Zitat Commonwealth of Australia, Department of the Prime Minister and Cabinet. (2018). Closing the gap: Prime Minister’s report 2018. Canberra: DPMC. Commonwealth of Australia, Department of the Prime Minister and Cabinet. (2018). Closing the gap: Prime Minister’s report 2018. Canberra: DPMC.
Zurück zum Zitat Cumming, J., & Goldstein, H. (2016). From school to employment: analysing the careers of year 9 students using the Longitudinal Surveys of Australian Youth. Longitudinal and Life Course Studies Journal, 7, 53–63. https://doi.org/10.14301/llcs.v7i1.342 . Cumming, J., & Goldstein, H. (2016). From school to employment: analysing the careers of year 9 students using the Longitudinal Surveys of Australian Youth. Longitudinal and Life Course Studies Journal, 7, 53–63. https://​doi.​org/​10.​14301/​llcs.​v7i1.​342 .
Zurück zum Zitat Cumming, J., Wyatt-Smith, C., & Colbert, P. (2016). Students ‘at risk’ and the National Assessment Program Literacy and Numeracy (NAPLAN): The ‘collateral damage’, in R. Lingard, G. Thompson & S. Sellar (Eds.), National testing and its effects: Evidence from Australia (pp. 126–138) (Abingdon, Routledge). Cumming, J., Wyatt-Smith, C., & Colbert, P. (2016). Students ‘at risk’ and the National Assessment Program Literacy and Numeracy (NAPLAN): The ‘collateral damage’, in R. Lingard, G. Thompson & S. Sellar (Eds.), National testing and its effects: Evidence from Australia (pp. 126–138) (Abingdon, Routledge).
Zurück zum Zitat Goldstein, H. (2011). Multilevel statistical models (Fourth ed.). Chichester: Wiley. Goldstein, H. (2011). Multilevel statistical models (Fourth ed.). Chichester: Wiley.
Zurück zum Zitat Goldstein, H., Carpenter, J. R., & Browne, W. J. (2014). Fitting multilevel multivariate models with missing data in responses and covariates that may include interactions and non-linear terms. Journal of the Royal Statistical Society: Series A (statistics in society), 177(2), 553–564. https://doi.org/10.1111/rssa.12022. Goldstein, H., Carpenter, J. R., & Browne, W. J. (2014). Fitting multilevel multivariate models with missing data in responses and covariates that may include interactions and non-linear terms. Journal of the Royal Statistical Society: Series A (statistics in society), 177(2), 553–564. https://​doi.​org/​10.​1111/​rssa.​12022.​
Zurück zum Zitat Goldstein, H., & French, R. (2015). Differential educational progress and measurement error. Journal of Longitudinal and Lifecourse and Studies, 6, 351–358. Goldstein, H., & French, R. (2015). Differential educational progress and measurement error. Journal of Longitudinal and Lifecourse and Studies, 6, 351–358.
Zurück zum Zitat Goldstein, H., Rasbash, J., Yang, M., Woodhouse, G., Pan, H., Nuttall, D., & Thomas, S. (1993). A multilevel analysis of school examination results. Oxford Review of Education, 19(4), 425–433.CrossRef Goldstein, H., Rasbash, J., Yang, M., Woodhouse, G., Pan, H., Nuttall, D., & Thomas, S. (1993). A multilevel analysis of school examination results. Oxford Review of Education, 19(4), 425–433.CrossRef
Zurück zum Zitat Goldstein, H., & Wood, R. (1989). Five decades of item response modelling. British Journal of Mathematical and Statistical Psychology, 42, 139–167.CrossRef Goldstein, H., & Wood, R. (1989). Five decades of item response modelling. British Journal of Mathematical and Statistical Psychology, 42, 139–167.CrossRef
Zurück zum Zitat Klenowski, V., Tobias, S., Funnell, B., Vance, F., & Kaesehagen, C. (2010). Culture-fair assessment: Challenging Indigenous students through effortful mathematics teaching. Paper presented at the AARE annual conference, Melbourne, 28 November to 2 December 2010. Klenowski, V., Tobias, S., Funnell, B., Vance, F., & Kaesehagen, C. (2010). Culture-fair assessment: Challenging Indigenous students through effortful mathematics teaching. Paper presented at the AARE annual conference, Melbourne, 28 November to 2 December 2010.
Zurück zum Zitat Lingard, B., Thompson, G., & Sellar, S. (Eds.). (2016). National testing in schools. An Australian assessment. Oxon: Routledge. Lingard, B., Thompson, G., & Sellar, S. (Eds.). (2016). National testing in schools. An Australian assessment. Oxon: Routledge.
Zurück zum Zitat Marks, G. (2015). Students, schools and socioeconomic background. Independence, 40(1), 4, 6, 8, 10, 12. Marks, G. (2015). Students, schools and socioeconomic background. Independence, 40(1), 4, 6, 8, 10, 12.
Zurück zum Zitat Ministerial Council on Education, Employment, Training and Youth Affairs. (2006). Australian directions in Indigenous education 2005–2008. Carlton South, Vic: MCEETYA). Ministerial Council on Education, Employment, Training and Youth Affairs. (2006). Australian directions in Indigenous education 2005–2008. Carlton South, Vic: MCEETYA).
Zurück zum Zitat Rice, S., Dulfer, N., Polesol, J., & O’Hanlon, C. (2016). NAPLAN and student wellbeing: teacher perceptions of the impact of NAPLAN on students. In R. Lingard, G. Thompson, & S. Sellar (Eds.), National testing and its effects: Evidence from Australia (pp. 72–85). Abingdon: Routledge. Rice, S., Dulfer, N., Polesol, J., & O’Hanlon, C. (2016). NAPLAN and student wellbeing: teacher perceptions of the impact of NAPLAN on students. In R. Lingard, G. Thompson, & S. Sellar (Eds.), National testing and its effects: Evidence from Australia (pp. 72–85). Abingdon: Routledge.
Zurück zum Zitat Senate Standing Committee on Education and Employment [SSCEE]. (2014). Effectiveness of the National Assessment Program–Literacy and Numeracy. Final report. Canberra: SSCEE. Senate Standing Committee on Education and Employment [SSCEE]. (2014). Effectiveness of the National Assessment Program–Literacy and Numeracy. Final report. Canberra: SSCEE.
Zurück zum Zitat Steering Committee for the Review of Government Service Provision. (2014). Overcoming Indigenous disadvantage: Key indicators 2014. Canberra: Productivity Commission. Steering Committee for the Review of Government Service Provision. (2014). Overcoming Indigenous disadvantage: Key indicators 2014. Canberra: Productivity Commission.
Zurück zum Zitat Thompson, G., Adie, L., & Klenowski, V. (2017). Validity and participation: Implications for school comparison of Australia’s national assessment program. Journal of Education Policy, 33(6), 759–777. https://doi.org/10.1080/02680939.2017.1373407 . Thompson, G., Adie, L., & Klenowski, V. (2017). Validity and participation: Implications for school comparison of Australia’s national assessment program. Journal of Education Policy, 33(6), 759–777. https://​doi.​org/​10.​1080/​02680939.​2017.​1373407 .
Zurück zum Zitat Thomson, S., De Bortoli, L., & Underwood, C. (2017a). PISA 2015: Reporting Australia’s results. Camberwell: ACER. Thomson, S., De Bortoli, L., & Underwood, C. (2017a). PISA 2015: Reporting Australia’s results. Camberwell: ACER.
Zurück zum Zitat Thomson, S., Hillman, K., Schmid, M., Rodrigues, S., & Fullarton, J. (2017b). Reporting Australia’s results PIRLS 2016. Camberwell: ACER. Thomson, S., Hillman, K., Schmid, M., Rodrigues, S., & Fullarton, J. (2017b). Reporting Australia’s results PIRLS 2016. Camberwell: ACER.
Zurück zum Zitat Thomson, S., Wernert, N., O’Grady, E., & Sima Rodrigues, S. (2017c). TIMSS 2015: Reporting Australia’s results. Camberwell: ACER. Thomson, S., Wernert, N., O’Grady, E., & Sima Rodrigues, S. (2017c). TIMSS 2015: Reporting Australia’s results. Camberwell: ACER.
Metadaten
Titel
Enhanced use of educational accountability data to monitor educational progress of Australian students with focus on Indigenous students
verfasst von
Joy Cumming
Harvey Goldstein
Kirstine Hand
Publikationsdatum
13.12.2019
Verlag
Springer Netherlands
Erschienen in
Educational Assessment, Evaluation and Accountability / Ausgabe 1/2020
Print ISSN: 1874-8597
Elektronische ISSN: 1874-8600
DOI
https://doi.org/10.1007/s11092-019-09310-x

Weitere Artikel der Ausgabe 1/2020

Educational Assessment, Evaluation and Accountability 1/2020 Zur Ausgabe