The drop and collect survey among industrial populations: theory and empirical evidence

Abstract

This paper examines the effectiveness of drop-and-collect-survey (DCS) method in improving response rates among organisations. It proposes a theory for its effectiveness and advances a number of propositions relating to its usage among organisations. Subsequent analyses, using empirical evidence from two separate DCS-based studies in Nigeria and Germany, suggest that the DCS results in significantly higher response rates among organisational respondents compared to mail questionnaires. These improved response rates are more likely to be achieved among smaller organisations and through direct, face to face, contact with the target key informant. Combining DCS with prenotification showed only a small, but positive effect. Support was also found for the hypothesised greater effectiveness of the DCS among high-technology firms relative to their low-technology counterparts. Based on the proposed theory and the empirical findings, a DCS decision and implementation framework is presented and directions for future research suggested.

Introduction

Collecting, analysing and interpreting data in a methodologically sound way is at the heart of science Hunt, 1991, Popper, 1994. While a myriad ways of collecting these data are available to the researcher, one of the dominant means in researching organisations is the paper-and-pencil questionnaire (Stablein, 1996), itself dominated by the mailed questionnaire (Cavusgil & Elvey-Kirk, 1998). However, despite its many advantages, the mail questionnaire technique is plagued by a major problem: nonresponse. Or, more specifically, low response rates Baruch, 1999, Chrisnall, 1997, Delener, 1995, Kinnear & Taylor, 1991. This low response problem appears to be greater Mehta & Sivadas, 1995, Mendelson & Pillai, 1999, Tse, 1998 with the increasingly adopted but less representative online survey approaches Dommeyer & Moraiarty, 1999/2000, Taylor, 1999–2000. There is little doubt that this untoward trend, with its implications for nonresponse bias (Schlegelmilch & Diamantopoulos, 1991), questions the validity and generalisability of research findings.

The nature of this problem is further complicated by the fact that response behaviour among organisations is contingent upon difficult-to-control organisational characteristics (Tomaskovic-Devey, Leiter, & Thompson, 1994). Although several studies into the determinants of nonresponse- and response-inducing techniques Baldauf et al., 1999, Cavusgil & Elvey-Kirk, 1998, Diamantopoulos et al., 1991, Greer et al., 2000, Jobber, 1986, Jobber & O'Reilly, 1998, Harvey, 1987, Karimbady & Brunn, 1991 have suggested the importance of measures like prenotification, follow-ups and the use of response incentives, the application of these measures does not appear to have resulted in any significant overall improvement in response rates reported in academic publications, especially among organisational populations Baruch, 1999, Taylor, 1999–2000, and even more so in international research contexts (Harzing, 1997).

Two related explanations may account for the decline in (or low) response rates among organisational populations. One is that the mail questionnaire technique has become a victim of its own success, that is, its many advantages (low cost, ability to reach a widely dispersed sample, lack of interviewer bias and relatively easy implementation (e.g., Diamantopoulos et al., 1991, Hart, 1987, Schlegelmilch & Diamantopoulos, 1991) have resulted in such widespread usage that the average organisational respondent is left feeling swamped Baldauf et al., 1999, Diamantopoulos et al., 1991. The observed research fatigue in oversampled organisational populations Brock, 2000, Cooper, 2000 appears to support this. It may also be the case that the ‘impersonal’ mail questionnaire approach—including its online variant—is perceived, to an extent, as a ‘put-off’ among organisational populations, which generally favour more personal methods (Roth & BeVier, 1998). There may, indeed, be a link between the perceived impersonality of the mail questionnaire method and the increasing level of respondent fatigue and antipathy to survey instruments.

One data collection technique which appears to share most of the best characteristics of the mail questionnaire method without the associated impersonality is the drop-and-collect-survey (DCS) method (Brown, 1984). As the name implies, the DCS method involves the researcher(s) and/or properly trained field assistants in personally delivering—and later collecting—the survey instrument (the questionnaire) either directly to the target respondent or indirectly via a gatekeeper (e.g., a secretary).

It is arguable that this interpersonal feature of the DCS plays a large part in the very high response rates reported in studies that employed this method Baker, 1991, Brown, 1987, Fox et al., 1998, Lovelock et al., 1976, Stover & Stone, 1974, e.g., 50–70% (Kinnear & Taylor, 1991), 77% (Brown, 1987), up to 80% (Lin & Germain, 1998) and 90% (Brown, 1984). In addition to supporting Jobber and O' Reilly's (Jobber & O'Reilly, 1998) (p. 104) views that “[a] key issue in survey research is the choice of delivery system,” the foregoing evidence raises the more fundamental question of why the DCS delivery system is virtually unexplored in organisational research (as opposed to consumer research), and why it is usually mentioned only implicitly (e.g., Yavas, 1987). It is, indeed, a matter of concern that nearly a decade and half after Brown's (1987) observation that the DCS has attracted little serious discussion, the situation has not really improved, even in the face of the worryingly low response rates associated with mail (and online) questionnaires. The extent to which the identified neglect is caused by the general lack of specific guidelines on the ‘whens,’ ‘hows’ and ‘whys’ of DCS usage (including potential demerits) is unclear. What is obvious, however, is the urgent need to facilitate greater and more effective use of the DCS technique through the provision of appropriate operational guidelines.

This paper seeks to assist in this process. Based on insights from relevant literature schemes, a theory explaining the effectiveness of the DCS is developed, with some five research propositions pertaining to its usage among organisations advanced. These are subsequently tested using empirical data from two separate DCS-based investigations undertaken among industrial populations in two different countries, Nigeria and Germany. The paper further presents a DCS decision and implementation framework, drawing from the present researchers' findings, as well as relevant insights from the previous DCS literature: pre-, since and including Brown (1987). It is envisaged that this best practice guide should assist future researchers in considering and implementing the DCS technique.

The rest of this paper is structured as follows. First, we present the theoretical underpinning of DCS, replete with a number of research propositions. Second, we present the methodology employed in the two DCS-based investigations. This is followed by the analysis of the empirical findings and tests of the research propositions. The penultimate section discusses the findings and presents the DCS decision and implementation framework. The paper concludes with some remarks, including relevant comments on the study's limitations and future research directions.