A Practical Approach to Testing a Natural Language System: Tools and Procedures

Most software development projects follow some type of clearly defined process, refined over many years of practical experience. Different phases of development are carefully tracked and documented according to strict guidelines. Until now the majority of natural language systems have been developed as research projects, that is they have been developed without such rigid conditions. However, now that the market for natural language systems is beginning to open up, there is a growing number of commercially available products. It therefore seems timely to start to discuss how the more traditional approaches to software development can be applied to such technologies. This paper concentrates on one vital stage in any development process, namely the test phase. We will show how current approaches need to be adapted to suit the needs of this type of software and, conversely, how much can be gained by looking at the experiences of testers of more conventional software systems.