Skip to main content
SpringerLink
Log in

Sampling and Weighting Cohort Samples in Institutional Contexts

The National Educational Panel Study cohort samples of Kindergarten children, students in Grade 5 and in Grade 9

  • Originalveröffentlichung
  • Published:
AStA Wirtschafts- und Sozialstatistisches Archiv Aims and scope Submit manuscript

Abstract

The National Educational Panel Study surveys, among others, cohort samples of Kindergarten children, students in Grade 5 and students in Grade 9. This paper gives details on the applied sampling designs to realize these samples. The implemented designs cover indirect sampling procedures, stratification, and two-stage cluster sampling designs. Details on the derivation of design weights and their successive adjustments yielding nonresponse adjusted weights are presented. The considered adjustments refer to institutional and individual nonresponse and take into account clustering on institutional level by specifying random effects within the corresponding regression models. For Kindergarten children, the empirical results show that a child’s living conditions (with both or only one parent present) influence the participation propensity. For students in secondary schools, a language other than German spoken at home as well as competencies in math and German influence the decision to participate in the panel study. A discussion of strategies to provide cross-sectional and longitudinal weights is provided.

Zusammenfassung

Das Nationale Bildungspanel erhebt unter anderem Kohortenstichproben von Kindergartenkindern, Schülern in der Klasse 5 und Schülern in der Klasse 9. Dieser Beitrag beschreibt detailliert die Stichprobenpläne dieser Kohorten. Die eingesetzten Stichprobenverfahren umfassen dabei indirekte Stichprobenziehung, Schichtung und zweistufige Klumpenstichproben. Im Rahmen der Erstellung der Gewichte werden im ersten Schritt Designgewichte hergeleitet. Im zweiten Schritt werden diese adjustiert, um für Teilnahmeverweigerungen innerhalb der Bruttostichprobe zu kompensieren. Diese Anpassungen kommen sowohl bei der Teilnahmeverweigerung auf institutioneller als auch auf individueller Ebene zur Anwendung. Die Klumpung von Individuen in Institutionen wird durch Zufallseffekte in den Regressionen berücksichtigt. Die empirischen Ergebnisse zeigen, dass die Teilnahme von Kindergartenkindern signifikant dadurch beeinflusst wird, ob das Kind mit beiden Eltern oder nur mit einem Elternteil zusammenlebt. Bei Schülern hingegen wird die Teilnahmebereitschaft durch die zu Hause gesprochene Sprache (Deutsch oder eine andere Sprache), sowie Kompetenzen in Mathematik und Deutsch maßgeblich beeinflusst. Abschließend werden Möglichkeiten zur Bereitstellung von Quer- und Längsschnittgewichten für Folgewellen der Panelerhebungen dargestellt.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. First versions of corresponding Scientific Use Files (SUF) for these starting cohorts were released in September and October 2012. See doi:10.5157/NEPS:SC2:1.0.0, doi:10.5157/NEPS:SC3:1.0.0 and doi:10.5157/NEPS:SC4:1.0.0.

  2. The replacement strategy established for Kindergarten institutions was very effective. That is, all Kindergartens refusing to participate could be replaced, see Sect. 2.4 of this paper.

  3. Regular schools are all “allgemeinbildende Schulen”, that is, schools of general education according to the definition of Kultusministerkonferenz (2014). Thus, regular or mainstream schools include all school types listed here with the exception of the school type of Förderschulen (FS).

  4. Note that the scientific use files thus so far are not including a Federal State variable and provide the school type based on reported answers.

  5. In most special schools, students in Grade 7, 8, and 9 are educated together. That is, in the majority of cases the number of Grade 9 students is hard to report, whereas the total number of students in Grades 7 to 9 is mostly available. Therefore, the number of Grade 9 students is approximated by one third of the reported number of students in Grades 7–9.

  6. Here, the design weights for Kindergarten institutions of SC2 are an exception. Due to the indirect sampling approach applied, design weights for Kindergarten institutions cannot directly be derived as the inverse of the inclusion probabilities of sampling units and are computed differently. For details see Sect. 2.2.

  7. Note that some of the schools participating in SC3 also participate in SC4.

  8. In SC3, 5191 students belong to regular schools, 584 students to special schools, and 290 to the migrants supplement. In SC4, 15327 students belong to regular schools and 1286 students to special schools. Differences between these numbers and the case numbers reported in the SUF data are due to the different time points at which panel content was obtained and Wave 1 surveys were conducted. Furthermore, panel consent could be withdrawn during later survey stages so that subsequently complete cases had to be deleted form the SUF data.

  9. All models below were estimated in R (R Core Team 2014) using the glmer function provided by lme4 (Bates et al. 2012) for estimating random intercept models. Exporting functions for tables were provided by memisc (Elff 2012).

  10. The models were also controlled for the following variables: strata (\(h=1,\ldots,7\)), nationality (German, non-German), dyslexia (yes, no), attention deficit hyperactivity disorder (yes, no).

  11. To account for the multilevel structure, differences in probabilities resulting from changing X 1 to X 2 are estimated via simulation (with simulation sample size \(S = 10^6\)) as follows

    $$\frac{1}{S}\sum_{s=1}^S \Phi\left(X_1\beta^{(s)}+\alpha^{(s)}\right)-\Phi\left(X_2\beta^{(s)}+\alpha^{(s)}\right)\quad,$$

    where \(\beta^{(s)}\) and \(\alpha^{(s)}\), \(s=1,\ldots,S\) denote a sample from the estimated asymptotic distribution. The corresponding quantiles of the trajectory \(\{\Phi(X_1\beta^{(s)}+\alpha^{(s)})-\Phi(X_2\beta^{(s)}+\alpha^{(s)})\}_{s=1}^S\) serve as estimates for 95 % confidence intervals as well.

  12. Note that no adjustments are necessary for the group of never-participating individuals.

  13. The NEPS sample SC2 of Kindergarten children is not stratified. Thus, weights and adjustment factors are here independent of the subindex h.

References

  • Arora A, Foy P, Martin MO, Mullis IVS (eds) (2009) TIMSS Advanced 2008 Technical Report. TIMSS & PIRLS International Study Center Lynch School of Education Boston College, Chestnut Hill

  • Aßmann C, Steinhauer HW, Kiesl H, Koch S, Schönberger B, Müller-Kuller A, Rohwer G, Rässler S, Blossfeld HP (2011) Sampling designs of the National Educational Panel Study: challenges and solutions. In: Blossfeld HP, Roßbach HG, von Maurice J (eds) Education as a lifelong process: Zeitschrift für Erziehungswissenschaft (Vol. 14). VS Verlag für Sozialwissenschaften, Wiesbaden, pp 51–65

  • Aßmann C, Steinhauer HW, Rässler S (2012) Aspekte der Stichprobenziehung in der erziehungswissenschaftlichen Forschung. In: Maschke S, Stecher L (eds) Enzyklopädie der Erziehungswissenschaft Online (EEO), Fachgebiet Methoden der empirischen erziehungswissenschaftlichen Forschung, Beltz Juventa, Weinheim und Basel, pp 1–15

  • Baker R, Brick JM, Bates NA, Battaglia M, Couper MP, Dever JA, Gile KJ, Tourangeau R (2013) Summary Report of the AAPOR task force on non-probability sampling. J Surv Stat Methodol 1(2):90–143

  • Bates D, Maechler M, Bolker B (2012) lme4: Linear mixed-effectsmodels using S4 classes. http://CRAN.R-project.org/package=lme4. Accessed 3 June 2014

  • Blossfeld HP, Roßbach HG, von Maurice J (eds) (2011) Education as a lifelong process: The German National Educational Panel Study (NEPS) [Special Issue]: Zeitschrift für Erziehungswissenschaft (Vol 14). VS Verlag für Sozialwissenschaften, Wiesbaden

  • Brick JM (2013) Unit Nonresponse and weighting adjustments: a critical review. J Off Stat 29(3):329–353

  • Brick JM, Kalton G (1996) Handling missing data in survey research. Stat Methods Med Res 5(3):215–238

  • Elff M (2012) memisc: Tools for management of survey data, graphics, programming, statistics, and simulation. http://CRAN.R-project.org/package=memisc. Accessed 3 June 2014

  • Elliott MR (2009) Combining data from probability and non-probability samples using pseudo-weights. Surv Pract 2(6):1–7

  • Kalton G (1986) Handling wave nonresponse in panel surveys. J Off Stat 2(3):303–314

  • Kalton G, Flores-Cervantes I (2003) Weighting methods. J Off Stat 19(2):81–97

  • Kalton G, Kasprzyk D (1986) The treatment of missing survey data. Surv Methodol 12:1–16

  • Kiesl H (2010) Selecting kindergarten children by three stage indirect sampling. In: American Statistical Association (ed) Proceedings of the survey research methods section, Alexandria, pp 2730–2738. http://www.amstat.org/sections/srms/proceedings/y2010/Files/307400_63544.pdf. Accessed 9 Jan 2012

  • Kish L (1990) Weighting: Why, when, and how? In: American Statistical Association (ed) Proceedings of the survey research methods section, pp 121–130. https://www.amstat.org/sections/SRMS/Proceedings/papers/1990_018.pdf. Accessed 3 April 2014

  • Kish L (1992) Weighting for unequal P i . J Off Stat 8(2):183–200

  • Kultusministerkonferenz (2014) Definitionenkatalog zur Schulstatistik 2014. http://www.kmk.org/fileadmin/pdf/Statistik/Dokumentationen/Defkat2014.pdf. Accessed 6 May 2014

  • LaRoche S, Zuehlke O, Joncas M (2009) TIMSS Advanced 2008 Samping. In: Arora A, Foy P, Martin MO, Mullis IVS (eds) TIMSS Advanced 2008 Technical Report, TIMSS & PIRLS International Study Center Lynch School of Education Boston College, Chestnut Hill, MA, pp 51–90

  • Lavallée P (2007) Indirect sampling. Springer, New York and London

  • Lepkowski JM (1989) Treatment of Wave Nonresponse in Panel Surveys. In: Kasprzyk D, Duncan GJ, Kalton G, Singh M (eds) Panel surveys, Wiley series in probability and mathematical statistics/Applied probability and statistics, Wiley, New York, pp 348–374

  • Little R, Vartivarian S (2003) On weighting the rates in non-response weights. Stat Med 22(9):1589–1599

  • Lynn P, Kaminska O (2010) Weighting strategy for understanding society: understanding society working paper no. 2010–05. http://research.understandingsociety.org.uk/publications/working-paper/2010-05.pdf. Accessed 5 June 2012

  • Martin MO, Mullis IVS, Kennedy AM (2007) PIRLS 2006 technical report. TIMSS & PIRLS, International Study Center, Lynch School of Education, Boston College, Chestnut Hill

  • OECD (2012) PISA 2009 Technical Report. OECD Publishing, Paris

  • Olson JF, Martin MO, Mullis IV (eds) (2008) TIMSS 2007 technical report. TIMSS & PIRLS International Study Center Lynch School of Education, Boston College, Chestnut Hill

  • Pfeffermann D (1996) The use of sampling weights for survey data analysis. Statistical Methods in Medical Research 5(3):239–261

  • R Core Team (2014) R: A language and environment for statistical computing. http://www.R-project.org/. Accessed 5 June 2014

  • Rosenbaum PR, Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70(1):41–55

  • Schnell R, Gramlich T, Bachteler T, Reiher J, Trappmann M, Smid M, Becher I (2013) Ein neues Verfahren für namensbasierte Zufallsstichproben von Migranten. Methoden-Daten-Analysen 7(1):5–33

  • Skinner CJ, D’Arrigo J (2011) Inverse probability weighting for clustered nonresponse. Biometrika 98(4):953–966

  • Statistisches Bundesamt (2010a) Fachserie 1 Reihe 1.3: Bevölkerung und Erwerbstätigkeit: Bevölkerungsfortschreibung. https://www.destatis.de/DE/Publikationen/Thematisch/Bevoelkerung/Bevoelkerungsfortschreibung2010130107004.pdf. Accessed 10 June 2014

  • Statistisches Bundesamt (2010b) Fachserie 11 Reihe 1: Bildung und Kultur: Allgemeinbildende Schulen: Schuljahr 2008/09. https://www.destatis.de/GPStatistik/servlets/MCRFileNodeServlet/DEHeft_derivate_00006815/2110100097004.pdf. Accessed 10 Feb 2014

  • Statistisches Bundesamt (2011) Statistiken der Kinder- und Jugendhilfe: Kinder und tätige Personen in Tageseinrichtungen und in öffentlich geförderter Kindertagespflege am 01.03.2011. https://www.destatis.de/DE/Publikationen/Thematisch/Soziales/KinderJugendhilfe/TageseinrichtungenKindertagespflege5225402117004.pdf. Accessed 10 June 2014

  • Wolter K (2007) Introduction to variance estimation. Springer, New York

  • Zinn S (2013) Replication Weights for the Cohort Samples of Students in Grade 5 and 9 in the National Educational Panel Study: NEPS Working Paper 27. https://www.neps-data.de/Portals/0/Working. Accessed 13 Aug 2014

Download references

Acknowledgement

This paper uses data from the National Educational Panel Study (NEPS). From 2008 to 2013, NEPS data were collected as part of the Framework Programme for the Promotion of Empirical Educational Research funded by the German Federal Ministry of Education and Research (BMBF). As of 2014, the NEPS survey is carried out by the Leibniz Institute for Educational Trajectories (LIfBi) at the University of Bamberg in cooperation with a nationwide network.

The authors would like to thank two anonymous reviewers and the editor for adding valuable comments and suggestions improving the paper considerably.

Author information

Authors and Affiliations

  1. Lehrstuhl für Statistik und Ökonometrie, Otto-Friedrich-Universität Bamberg, 96045, Bamberg, Germany

    Christian Aßmann, Solange Goßmann & Susanne Rässler

  2. Leibniz-Institut für Bildungsverläufe e. V., Wilhelmsplatz 3, 96047, Bamberg, Germany

    Hans Walter Steinhauer, Christian Aßmann, Sabine Zinn, Solange Goßmann & Susanne Rässler

  3. Max Planck Institute for Demographic Research, Konrad-Zuse-Straße 1, 18057, Rostock, Germany

    Sabine Zinn

Authors
  1. Hans Walter Steinhauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Aßmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sabine Zinn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Solange Goßmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Susanne Rässler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hans Walter Steinhauer.

Appendices

Appendix A

Table 1 Population of regular schools by school type (school year 2008/2009)
Full size table
Table 2 Population of Students by school type (school year 2008/2009)
Full size table
Table 3 Population sizes (\(M_h^5\) and \(M_h^9\)), sample sizes (\(m_h^5\) and \(m_h^9\)), and total measures of size (\(MOS_h^5\) and \(MOS_h^9\)) for schools by strata (\(h = 1, \ldots, 7\))
Full size table
Table 4 Results of random intercept models for school participation (by strata)
Full size table
Table 5 Models estimating the individual participation propensity used to derive adjustment factors for sample weighting adjustment of the initial sample
Full size table

Appendix B

Fig. 1
figure 1

Decision processes involved for specific units of interest

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Steinhauer, H., Aßmann, C., Zinn, S. et al. Sampling and Weighting Cohort Samples in Institutional Contexts. AStA Wirtsch Sozialstat Arch 9, 131–157 (2015). https://doi.org/10.1007/s11943-015-0162-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11943-015-0162-0

Keywords

Schlüsselwörter

Search

Navigation