Skip to main content
SpringerLink
Log in

Software requirements and acceptance testing

  • Published:
Annals of Software Engineering

Abstract

In this paper we 1) review industry acceptance testing practices and 2) present a systematic approach to scenario analysis and its application to acceptance testing with the aim to improve the current practice. It summarizes the existing practice into categories and identifies the serious weakness. Then, a new approach based on the formal scenario analysis is presented. It is systematic, and easily applicable to any software or system. A simple, yet realistic example is used to illustrate its effectiveness. Finally, its benefits and its applicability are summarized.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alford, M. (1977), “A Requirements Engineering Methodology for Real-Time Systems,” IEEE Transactions on Software Engineering 3,1, 60–69.

    Article  MathSciNet  Google Scholar 

  • Andriole, S.J. (1989), Storyboard Prototyping — A New Approach to User Requirements Analysis, QED Information Systems, Wellesley, MA.

    Google Scholar 

  • Bauer, J., S. Faasse, A. Finger, and W. Goodhue (1978), “The Automatic Generation and Execution of Function Test Plans for Electronic Switching Systems,” In Proceedings of the Software Quality Assurance Workshop, ACM, New York, NY, pp. 92–100.

    Google Scholar 

  • Bauer, J. and A. Finger (1979), “Test Plan Generation Using Formal Grammars,” In Proceedings of the Fourth International Conference on Software Engineering, IEEE Computer Society Press, Los Alamitos, CA, pp. 425–432.

    Google Scholar 

  • Beizer, B. (1984), Software System Testing and Quality Assurance, Van Nostrand Reinhold, New York, NY.

    Google Scholar 

  • Chandrasekharan, M., B. Dasarathy, and Z. Kishimoto (1989), “Requirements-Based Testing of Real-Time Systems: Modeling for Testability,” IEEE Computer 18,4, 71–80.

    Google Scholar 

  • Chow, T. (1977), “Testing Software Design Modeled by Finite State Machines,” In Proceedings COMPSAC'77, IEEE Computer Society Press, Washington, DC, pp. 58–64; also in IEEE Transactions on Software Engineering 4, 3, 178-187.

    Google Scholar 

  • Compton, H., P. Giloth, M. Haverty, and B. Niedfeldt (1977), “System Integration and Early Office Experience,” The Bell System Technical Journal 56,7, 1279–1294.

    Google Scholar 

  • Dasarathy, B. and M. Chandrasekharan (1982), “Test Generation for Functional Validation of Real-Time Systems,” Sixth IEEE International Conference on Software Engineering, Unpublished poster session.

  • Davis, A. (1980), “Automating the Requirements Phase: Benefits to Later Phases of the Software Life Cycle,” In Proceedings COMPSAC'80, IEEE Computer Society Press, Washington, DC, pp. 42–48.

    Google Scholar 

  • Davis, A. (1990), “System Testing: Implications of Requirements Specifications,” Information and Software Technology 32,6, 407–414.

    Article  Google Scholar 

  • Davis, A. (1993), Software Requirements, Objects, Function & States, Prentice Hall, Englewood Cliffs, NJ.

    Google Scholar 

  • Davis, A. and T. Ranscher (1978), “Formal Techniques and Automatic Processing to Ensure Correctness in Requirements Specifications,” In Proceedings of the Specifications of Reliable Software Conference, IEEE Computer Society Press, Los Alamitos, CA, pp. 15–35.

    Google Scholar 

  • Deutsch, M. (1982), Software Verification and Validation, Prentice Hall, Englewood Cliffs, NJ.

    MATH  Google Scholar 

  • Dunn, R. and R. Ullman (1982), Quality Assurance for Computer Software, McGraw-Hill, New York, NY.

    MATH  Google Scholar 

  • Ferguson, R. and B. Korel (1996), “The Chaining Approach for Software Test Data Generation,” ACM Transactions on Software Engineering and Methodology 5,1, 63–86.

    Article  Google Scholar 

  • Gomaa, H. and D. Scott (1981), “Prototype as a Tool in the Specification of User Requirements,” In Proceedings of the 5th International Conference Software Engineering, IEEE Computer Society Press, Los Alamitos, CA, pp. 333–342.

    Google Scholar 

  • Hetzel, W. (1984), The Complete Guide to Software Testing, QED Information Systems, Wellesley, MA.

    Google Scholar 

  • Hsia, P., J. Gao, J. Samuel, D. Kung, Y. Toyoshima, and C. Chen (1994a), “A Formal Approach to Scenario Analysis,” IEEE Software 11,2, 33–41.

    Article  Google Scholar 

  • Hsia, P., J. Gao, J. Samuel, D. Kung, Y. Toyoshima, and C. Chen (1994b), “Behavior-Based Integration Testing of Software Systems: A Formal Scenario Approach,” In Proceedings of the Third International Conference on System Integration, IEEE Computer Society Press, Los Alamitos, CA, pp. 1138–1147.

    Google Scholar 

  • Hsia, P., J. Gao, J. Samuel, D. Kung, Y. Toyoshima, and C. Chen (1994c), “Behavior-Based Acceptance Testing of Software Systems: A Formal Scenario Approach,” In Proceedings of COMPSAC'94, IEEE Computer Society Press, Los Alamitos, CA, pp. 293–298.

    Google Scholar 

  • Hsia, P., J. Samuel, D. Kung, L. Li, C.T. Hsu, C. Chen, and Y. Toyoshima (1995), “A Usage-Model Based Approach to Test the Therac-25,” Safety and Reliability in Emerging Control Technologies, IFAC Workshop Report, Pergamon Press, Daytona Beach, FL, pp. 55–63.

    Google Scholar 

  • Huebner, D. (1979), “Systems Validation through Automated Requirements Verification,” In Proceedings of COMPSAC'79, IEEE Computer Society Press, Washington, DC, pp. 222–227.

    Google Scholar 

  • Janowiak, T. (1990), “A Simulation for Combat Systems Development and Acceptance Testing,” In Proceedings of the 1990 Winter Simulation Conference, IEEE, New York, NY, pp. 210–213.

    Chapter  Google Scholar 

  • Kaner, C., J. Falk, and H. Nguyen (1988), Testing Computer Software, Second Edition, Van Nostrand Reinhold, New York, NY.

    MATH  Google Scholar 

  • Kleinstein, D. (1988), “System Acceptance Testing in the Real World: A Case Study,” In Proceedings of the Third Israel Conference on Computer Systems and Software Engineering, Tel-Aviv, Israel, pp. 40–46.

    Chapter  Google Scholar 

  • Krause, K. and L. Diamant (1978), “A Management Methodology for Testing Software Requirements,” In Proceedings of COMPSAC'78, IEEE Computer Society Press, Washington, DC, pp. 749–754.

    Google Scholar 

  • Lea, R., S. Chen, and C. Chung (1991), “On Generating Test Data From Prototypes,” In Proceedings of the 15th International Computer Software and Applications Conference, IEEE Computer Society Press, Los Alamitos, CA, pp. 345–350.

    Google Scholar 

  • Linehan, T., Jr. (1988), “Application Software Configuration Management and Testing in a Pharmaceutical Laboratory Automation Environment,” In Proceedings of the Conference on Software Maintenance, IEEE Computer Society Press, Los Alamitos, CA, pp. 178–182.

    Chapter  Google Scholar 

  • Lipow, M. and T. A. Thayer (1977), “Prediction of Software Failures,” In Proceedings of the Annual Symposium on Reliability and Maintainability, IEEE, New York, NY, pp. 489–494.

    Google Scholar 

  • McCabe, T. (1983), “Structured Requirements and Testing,” Software Outlook Newsletter, McCabe and Associates, Columbia, MD.

    Google Scholar 

  • McCabe, T. and G. Schulmeyer (1985), “System Testing aided by Structured Analysis — A Practical Experience,” IEEE Transactions on Software Engineering 11,9, 917–921.

    Article  Google Scholar 

  • Meyers, G. (1979), The Art of Software Testing, Wiley, New York, NY.

    Google Scholar 

  • Musa, J. and A. Ackerman (1989), “Quantifying Software Validation: When to Stop Testing,” IEEE Software, 19–27.

  • Nardi, B. (1992), “The Use of Scenarios in Design,” SIGCHI Bulletin 24,4, 13–14.

    Article  Google Scholar 

  • NBS (1984), Guideline for Lifecycle Validation, Verification, and Testing of Computer Software (1984), US Department of Commerce, National Bureau of Standards, Springfield, VA.

    Google Scholar 

  • Osder, S. and D. Decker (1982), “Software Validation for Flight-Critical Systems,” In Proceedings of the Phoenix Conference on Computers and Communications, IEEE Computer Society Press, Washington, DC, pp. 193–198.

    Google Scholar 

  • Protzel, P. (1988), “Automatically Generated Acceptance Test: A Software Reliability Experiment,” In Proceedings of the 2nd Workshop on Software Testing, Verification, and Analysis, IEEE Computer Society Press, Washington, DC, pp. 196–203.

    Chapter  Google Scholar 

  • Quirk, W., Ed. (1985), Verification and Validation of Real-Time Software, Springer, Berlin, Germany.

  • Ramamoorthy, C., S. Ho, and W. Chen (1976), “On the Automated Generation of Program Test Data,” IEEE Transactions on Software Engineering 2,4, 293–300.

    Article  Google Scholar 

  • Ray, S. (1989), “Acceptance Tests of Distributed Processing Oriented Supervisory Control and Data Acquisition (SCADA) Systems for Offshore Platforms,” In Proceedings of the First European Test Conference, IEEE Computer Society Press, Washington, DC, pp. 221–228.

    Chapter  Google Scholar 

  • Rumbaugh, J. (1991), Object-Oriented Modeling and Design, Prentice-Hall, Englewood Cliffs, NJ.

    Google Scholar 

  • Samson, D. (1990), “Test Planning from System Requirements: A Classification Approach,” Jet Propulsion Laboratory Quarterly Report 12, 33–37.

    Google Scholar 

  • Szaboky, G. (1996), “A Systematic Approach for Testing GUI Applications,” Masters Thesis, Computer Science and Engineering Department, University of Texas at Arlington, Arlington, TX.

    Google Scholar 

  • Tausworthe, R. (1979), Standardized Development of Computer Software, Part II, Standards, Prentice-Hall, Englewood Cliffs, NJ.

    Google Scholar 

  • Wallace, D. and J. Cherniavsky (1986), Report on the NBS Software Acceptance Test Workshop April 1–2, 1986, Natl. Bur. Stand. US Department of Commerce, Washington, DC.

    Google Scholar 

  • Wallace, D. (1990), Guide to Software Acceptance, US Department of Commerce, National Institute of Standards and Technology, Gaithersburg, MD.

    Google Scholar 

  • Wardle, P. (1991), “Methodology and Tools for Requirements Capture, Traceability, and Verification,” In Proceedings of the Third International Conference on Software Engineering for Real Time Systems (Conf. Publication No. 344), The Institute, London, UK, pp. 46–50.

    Google Scholar 

  • Worrest, R. (1982), “Using an Automatic Test Executor to Support Parallel System and Test Development,” In Proceedings of the Phoenix Conference on Computers and Communications, IEEE Computer Society Press, Washington, DC, pp. 204–208.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Texas at Arlington, Box 19015, Arlington, Texas, 76019–0015, USA E‐mail:

    Pei Hsia & David Kung

  2. Lockheed/Martin, Fort Worth, TX, 76101, USA

    Chris Sell

Authors
  1. Pei Hsia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Kung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chris Sell
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hsia, P., Kung, D. & Sell, C. Software requirements and acceptance testing. Annals of Software Engineering 3, 291–317 (1997). https://doi.org/10.1023/A:1018938021528

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018938021528

Keywords

Search

Navigation