Innovations for Requirement Analysis. From Stakeholders’ Needs to Formal Designs

This paper is an extended abstract of an invited talk at the workshop that I put together using material from other talks and from papers that I and colleagues have written. The purposes of this extended abstract are to summarize the talk and to allow the reader to find the source materials for the talk directly.

The overall goal is to discuss some issues concerning the dependencies at the discourse level and at the sentence level. However, first I will briefly describe the Penn Discourse Treebank (PDTB)*, a corpus in which we annotate the discourse connectives (explicit and implicit) and their arguments together with “attributions” of the arguments and the relations denoted by the connectives, and also the senses of the connectives. I will then focus on the complexity of dependencies in terms of (a) the elements that bear the dependency relations, (b) graph theoretic properties of these dependencies such as nested and crossed dependencies, dependencies with shared arguments, and (c) attributions and their relationship to the dependencies, among others. I will compare these dependencies with those at the sentence level and discuss some issues that relate to the transition from the sentence level to the level of ”immediate discourse” and propose some conjectures.

An increasing interest in moving human language technology beyond the level of the sentence in text summarization, question answering, and natural language generation , among others, has recently led to the development of several resources that are richly annotated at the discourse level. Among these is the Penn Discourse TreeBank. (PDTB), a large-scale resource of annotated discourse relations and their arguments over the one million word Wall Street Journal (WSJ) Corpus. Since the sentence-level syntactic annotations of the Penn Treebank [2] and the predicate-argument annotations of the Propbank [4] have been done over the same target corpus, the PDTB thus provides a richer substrate for the development and evaluation of practical algorithms while supporting the extraction of useful features pertaining to syntax, semantics and discourse all at once. The PDTB is the first to follow a lexically - grounded approach to the annotation of discourse relations. Discourse relations, when realized explicitly in the text, are annotated by marking the necessary lexical items – called discourse connectives - expressing them, thus supporting their automatic identification.

PDTB adopts a theory-neutral approach to the annotation, making no commitments to what kinds of high-level structures may be created from the low level annotations of relations and their arguments. This approach has the appeal of allowing the corpus to be useful for researchers working within different frameworks. This theory neutrality also permits investigation of the general question of how structure at the sentence level relates to structure at the discourse level, at least that part of the discourse structure that is “parallel” to the sentence structure [6]. In addition to the argument structure of discourse relations, the PDTB provides sense labels for each relation following a hierarchical classification scheme. Annotation of senses highlights the polysemy of connectives, making the PDTB useful for sense disambiguation tasks [3]. Finally, the PDTB separately annotates the attribution of each discourse relation and of each of its two arguments. While attribution is a relation between agents and abstract objects and thus not a discourse relation, it has been annotated in the PDTB because (a) it is useful for applications such as subjectivity analysis and multi-perspective QA [5], and (b) it exhibits an interesting and complex interaction between sentence-level structure and discourse structure [1]. The first preliminary release of the PDTB was in April 2006. A significantly extended version was released as PDTB-2.0 in February 2008, through the Linguistic Data Consortium (LDC), see http://www.seas.upenn.edu/ pdtb, for the annotation manual, published papers, tutorial slides and a link to LDC.

Requirement engineering usually involves repeated refinements of the requirements specifications, starting with high-level systems goals and constraints, to more precise and measurable specifications of intended behavior, to detailed, focused statements that provide the basis for formal reasoning. We refer to these more detailed, mathematically rigorous specifications as property specifications. Although care must be taken when defining requirements at all these levels of abstraction, it is particularly difficult to accurately capture all the subtle details associated with property specifications. To help understand these decisions, the PROPEL (PROPerty ELicitation) system [1, 2] provides templates for commonly occurring property patterns [3] in which the options that need to be considered for each pattern are explicitly represented. PROPEL currently provides three views of each template and its associated options: natural language phrases to be selected, a set of hierarchical questions to be answered, or a finite-state automaton with optional labels, transitions, and accepting states to be selected. After all the options have been selected for a template, the finite-state automaton view provides a mathematically precise property specification.

The Defect Detection and Prevention (DDP) decision support process, developed at JPL, has over the last 8 years been applied to assist in making a variety of spacecraft decisions. It was originally conceived of as a means to help select and plan hardware assurance activities (inspections, tests, etc) [1], generally late in the development lifecycle. However, since then it has been used predominantly in early phase of system design, when information is scarce, yet many critical decisions are made. Its range of application has extended to encompass a wide variety of kinds of systems and technologies. Its predominant role has been to assist in planning the maturation of promising new technologies to help guide the next steps in their development as they emerge from the laboratory and seek to mature sufficiently to become acceptable to spacecraft missions [2]. Although this may at first glance seem far removed from terrestrial considerations, the factors that come into play in this kind of decision-making are universal - unclear and inconsistent perceptions about requirements and capabilities, uncertainty of what are the driving concerns that should be addressed and how best to address them, challenges of gathering and combining information from experts of multiple difference disciplines, and inevitably the lack of sufficient resources (money, time, CPU, power, ...) to do everything one would wish. Other significant applications of DDP have been as the risk management tool for entire spacecraft projects in their early phases of development, as an aid to planning portfolios of mission activities (e.g., [3]), and as a means to help guide R&D decisions (e.g., [4], [5]).

The lions’s share of the software faults can be traced to requirements and specification errors, so improvements in requirements engineering can have a large impact on the effectiveness of the overall system development process. A weak link in the chain is the transition from the vague and informal needs of system stakeholders to the formal models that support theoretical analysis and software tools.

This paper explains the context for the 2007 Monterey workshop that was dedicated to this problem. It provides the case study that participants were asked to use to illustrate their new methods, and summarizes the discussion and conclusions of the workshop.