A field study of the software design process for large systems

Reviewer: David Noel Card

Because of the difficulty, expense, and time involved in collecting data and performing quantitative analyses, many recent investigators of critical software engineering problems have adopted field research or case study methods. Instead of using discrete attributes of software projects, these researchers rely on interviews with software developers and observations of them at work. The authors of this paper demonstrate the power of this approach. Curtis, Krasner, and Iscoe study seventeen software projects and identify three common design problems: limited application domain knowledge, fluctuating requirements, and poor technical communication. They explore the causes and consequences of these problems at five levels—individual, team, project, company, and milieu—and observe that problems caused at one level may have repercussions on another. While these problems will be familiar to most software practitioners, a view of them through this organizational model may not. Consider, for example, the problem of limited application knowledge at the team level. The authors observe that most design teams are dominated by one leader or project guru. The characteristics of such a designer include substantial knowledge of the application domain, interdisciplinary background and experience, and good communication skills. The exceptional designer is often, as the authors note, not a good programmer. Lack of application knowledge would thus present more of a difficulty to design teams than lack of computer science knowledge. The authors provide an interesting and well-written exposition of their approach and its results. Use of the term “ecological” in the first subheading of the paper is somewhat distracting, though. This term is not defined until the very last section, where the authors explain that their work is not really an ecological study anyway. (Ecological studies tend to be quantitative analyses of such things as the carrying capacity of an environment.) While the field research/case study method provides an important tool that will improve our understanding of human activity, we must remember that it is not conclusive in itself. The history of anthropology provides several examples in which researchers obtained grossly distorted views of cultures from interviews with informants and observations of activities. The authors point out that in the early stages of problem identification and definition, however, when quantitative studies are often impossible, this method of study works well.