Skip to main content
Erschienen in: Empirical Software Engineering 1/2013

01.02.2013

A study of methods for textual satisfaction assessment

verfasst von: Elizabeth Ashlee Holbrook, Jane Huffman Hayes, Alex Dekhtyar, Wenbin Li

Erschienen in: Empirical Software Engineering | Ausgabe 1/2013

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Software projects requiring satisfaction assessment are often large scale systems containing hundreds of requirements and design elements. These projects may exist within a high assurance domain where human lives and millions of dollars are at stake. Satisfaction assessment can help identify unsatisfied requirements early in the software development lifecycle, when issues can be corrected with less impact and lower cost. Manual satisfaction assessment is expensive both in terms of human effort and project cost. Automated satisfaction assessment assists requirements analysts during the satisfaction assessment process to more quickly determine satisfied requirements and to reduce the satisfaction assessment search space. This paper introduces two new automated satisfaction assessment techniques and empirically demonstrates their effectiveness, as well as validates two previously existing automated satisfaction assessment techniques. Validation shows that automatically generated satisfaction assessments have high accuracy, thus reducing the workload of the analyst in the satisfaction assessment process.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
1
The above scenario examines the case of a developer, but in many mission- or safety-critical projects, an IV&V agent is responsible for performing satisfaction assessment.
 
2
This threshold was determined based on discussions with IV&V analysts (not involved in the assessment study). Also, when other thresholds were used, it was the case that the number of requirements labeled as satisfied was unrealistically small or large (no requirements satisfied, all requirements satisfied).
 
3
We did not ask the participants to review every ‘not satisfied” requirement due to participant time constraints. The participants were assigned review items so as to achieve coverage while also ensuring as much overlap as possible (for example, having reviewer 1 look at the first 30 items and reviewer 2 look at items 20 – 40).
 
4
We consulted with a statistics professor during the answer set creation process.
 
Literatur
Zurück zum Zitat Altman DG (1991) Practical statistics for medical research. Chapman & Hall Altman DG (1991) Practical statistics for medical research. Chapman & Hall
Zurück zum Zitat Antoniol G (2002) Recovering traceability links between code and documentation. IEEE Trans on Software Engineering 28(10):970–983CrossRef Antoniol G (2002) Recovering traceability links between code and documentation. IEEE Trans on Software Engineering 28(10):970–983CrossRef
Zurück zum Zitat Asplaugh TA, Antón AI (2008) Scenario support for effective requirements. Inf Softw Technol 50(3):198–220CrossRef Asplaugh TA, Antón AI (2008) Scenario support for effective requirements. Inf Softw Technol 50(3):198–220CrossRef
Zurück zum Zitat Baeza-Yates R, Ribeiro-Neto B (2003) Modern information retrieval. Addison-Wesley Baeza-Yates R, Ribeiro-Neto B (2003) Modern information retrieval. Addison-Wesley
Zurück zum Zitat Cleland-Huang J (2002) Automating speculative queries through event-based requirements traceability. Joint Conference on Requirements Engineering Cleland-Huang J (2002) Automating speculative queries through event-based requirements traceability. Joint Conference on Requirements Engineering
Zurück zum Zitat Cuddeback D, Dekhtyar A, Hayes JH (2010) Automated requirements traceability: the study of human analysts. In Proc. 18th International Conference on Requirements Engineering, Sydney, Australia Cuddeback D, Dekhtyar A, Hayes JH (2010) Automated requirements traceability: the study of human analysts. In Proc. 18th International Conference on Requirements Engineering, Sydney, Australia
Zurück zum Zitat Di Lucca GA, Di Penta M, Antoniol G, Casazza G (2001) An approach for reverse engineering of web-based applications. BP - 231, Dipartimento di Informatica e Sistemistica Di Lucca GA, Di Penta M, Antoniol G, Casazza G (2001) An approach for reverse engineering of web-based applications. BP - 231, Dipartimento di Informatica e Sistemistica
Zurück zum Zitat Diallo MH, Naslavsky L, Ziv H, Alspaugh TA, Richardson DA (2007) Evaluating software architectures against requirements-level scenarios. Workshop on the Role of Software Architecture for Testing and Analysis Diallo MH, Naslavsky L, Ziv H, Alspaugh TA, Richardson DA (2007) Evaluating software architectures against requirements-level scenarios. Workshop on the Role of Software Architecture for Testing and Analysis
Zurück zum Zitat Durán A, Ruiz A, Toro M (2001) An automated approach for verification of software requirements. Jornadas de Ingeniería de Requisitos Aplicada, Seville, Spain Durán A, Ruiz A, Toro M (2001) An automated approach for verification of software requirements. Jornadas de Ingeniería de Requisitos Aplicada, Seville, Spain
Zurück zum Zitat Gotel OCZ, Finkelstein ACW (1996) Extended requirements traceability: a framework for changing requirements. Workshop on Requirements Engineering in a Changing World Gotel OCZ, Finkelstein ACW (1996) Extended requirements traceability: a framework for changing requirements. Workshop on Requirements Engineering in a Changing World
Zurück zum Zitat Greenspan S, Mylopoulos J, Borgida J (1994) On formal requirements modeling languages: RML revisited. Proc. 16th International Conference on Software Engineering, p 135–147, Sorrento, Italy Greenspan S, Mylopoulos J, Borgida J (1994) On formal requirements modeling languages: RML revisited. Proc. 16th International Conference on Software Engineering, p 135–147, Sorrento, Italy
Zurück zum Zitat Hayes JH, Dekhtyar A (2005) Humans in the traceability loop: can't live with 'em, can't live without 'em. Proc. of the 3rd International Workshop on Traceability in Emerging Forms of Software Engineering, Long Beach, California. TEFSE '05. ACM, New York, NY, 20–23 Hayes JH, Dekhtyar A (2005) Humans in the traceability loop: can't live with 'em, can't live without 'em. Proc. of the 3rd International Workshop on Traceability in Emerging Forms of Software Engineering, Long Beach, California. TEFSE '05. ACM, New York, NY, 20–23
Zurück zum Zitat Hayes JH, Dekhtyar A, Osbourne J (2003) Improving requirements tracing via information retrieval. International Conference on Requirements Engineering Hayes JH, Dekhtyar A, Osbourne J (2003) Improving requirements tracing via information retrieval. International Conference on Requirements Engineering
Zurück zum Zitat Hayes JH, Dekhtyar A, Sundaram S (2006a) Advancing requirements tracing: the study of methods. IEEE Trans Software Engineering 32(1):4–19CrossRef Hayes JH, Dekhtyar A, Sundaram S (2006a) Advancing requirements tracing: the study of methods. IEEE Trans Software Engineering 32(1):4–19CrossRef
Zurück zum Zitat Hayes JH, Dekhtyar A, Sundaram S (2006) Advances in dynamic generation of traceability links. Tech Report, (TR 451–06) Hayes JH, Dekhtyar A, Sundaram S (2006) Advances in dynamic generation of traceability links. Tech Report, (TR 451–06)
Zurück zum Zitat Hayes JH, Dekhtyar A, Sundaram S, Holbrook A, Vadlamudi S, April A (2007) REquirements TRacing On target (RETRO): improving software maintenance through traceability recovery. Innovations in Systems and Software Engineering: A NASA Journal 3(3):193–202 Hayes JH, Dekhtyar A, Sundaram S, Holbrook A, Vadlamudi S, April A (2007) REquirements TRacing On target (RETRO): improving software maintenance through traceability recovery. Innovations in Systems and Software Engineering: A NASA Journal 3(3):193–202
Zurück zum Zitat Holbrook EA (2009) Satisfaction assessment of textual software engineering artifacts, PhD dissertation, Dept. of Computer Science., University of Kentucky, Lexington, KY Holbrook EA (2009) Satisfaction assessment of textual software engineering artifacts, PhD dissertation, Dept. of Computer Science., University of Kentucky, Lexington, KY
Zurück zum Zitat Holbrook EA, Hayes JH, Dekhtyar A (2009) Toward automating requirements satisfaction assessment. Requirements Engineering Conference, 2009. RE '09. 17th IEEE International, vol., no., pp 149–158 Holbrook EA, Hayes JH, Dekhtyar A (2009) Toward automating requirements satisfaction assessment. Requirements Engineering Conference, 2009. RE '09. 17th IEEE International, vol., no., pp 149–158
Zurück zum Zitat ISO9000:2000, Quality Standard, International Organization for Standardization ISO9000:2000, Quality Standard, International Organization for Standardization
Zurück zum Zitat Lecceuche R (2000) Finding comparatively important concepts between texts. Automated Software Engineering (ASE’00). Washington, DC, 55 Lecceuche R (2000) Finding comparatively important concepts between texts. Automated Software Engineering (ASE’00). Washington, DC, 55
Zurück zum Zitat Letier E, van Lamsweerde A (2004) Reasoning about partial goal satisfaction for requirements and design engineering. SIGSOFT Softw Eng Notes 29(6):53–62CrossRef Letier E, van Lamsweerde A (2004) Reasoning about partial goal satisfaction for requirements and design engineering. SIGSOFT Softw Eng Notes 29(6):53–62CrossRef
Zurück zum Zitat Marcus A, Maletic JI (2003) Recovering documentation-to-source code traceability links using latent semantic indexing. In: Proceedings of the 25th International Conference on Software Engineering (ICSE '03). IEEE Computer Society, Washington, DC, pp 125–135 Marcus A, Maletic JI (2003) Recovering documentation-to-source code traceability links using latent semantic indexing. In: Proceedings of the 25th International Conference on Software Engineering (ICSE '03). IEEE Computer Society, Washington, DC, pp 125–135
Zurück zum Zitat Marcus MP, Santorini B, Marcinkiewicz MA (1993) Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics 19:313–330 Marcus MP, Santorini B, Marcinkiewicz MA (1993) Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics 19:313–330
Zurück zum Zitat Marcus A, Maletic JI, Sergeyev A (2005) Recovery of traceability links between software documentation and source code. Int J Softw Eng Knowl Eng, World Scientific 15(5):811–836 Marcus A, Maletic JI, Sergeyev A (2005) Recovery of traceability links between software documentation and source code. Int J Softw Eng Knowl Eng, World Scientific 15(5):811–836
Zurück zum Zitat Porter A, Votta L (1998) Comparing detection methods for software requirements inspections. Empir Softw Eng 3(4):355–379CrossRef Porter A, Votta L (1998) Comparing detection methods for software requirements inspections. Empir Softw Eng 3(4):355–379CrossRef
Zurück zum Zitat Robinson WN (2009) Seeking quality through user-goal monitoring. IEEE Software, pp 58–65 Robinson WN (2009) Seeking quality through user-goal monitoring. IEEE Software, pp 58–65
Zurück zum Zitat Robinson WN, Pawlowski S (1999) Managing requirements inconsistency with development goal monitors. IEEE Trans. on Software Eng Robinson WN, Pawlowski S (1999) Managing requirements inconsistency with development goal monitors. IEEE Trans. on Software Eng
Zurück zum Zitat Salton G (1983) Introduction to modern information retrieval. McGraw-Hill Salton G (1983) Introduction to modern information retrieval. McGraw-Hill
Zurück zum Zitat Shull G, Rus I, Basili VR (2000) How perspective-based reading can improve requirements inspections. IEEE Computer 33(7):73–79CrossRef Shull G, Rus I, Basili VR (2000) How perspective-based reading can improve requirements inspections. IEEE Computer 33(7):73–79CrossRef
Zurück zum Zitat Spanoudakis G, Zisman A (2005) Software traceability: a roadmap. In: Chang SK (ed) Handbook of Software Engineering and Knowledge Engineering, vol. 3: Recent Advancements. World Scientific Publishing Spanoudakis G, Zisman A (2005) Software traceability: a roadmap. In: Chang SK (ed) Handbook of Software Engineering and Knowledge Engineering, vol. 3: Recent Advancements. World Scientific Publishing
Zurück zum Zitat Spanoudakis G, d’Avila A, Garcez A, Zisman A (2003) Revising rules to capture requirements traceability relations: a machine learning approach. In: Proceedings of the 15th International Conference in Software Engineering and Knowledge Engineering (SEKE 2003), pp 570–577, San Francisco Spanoudakis G, d’Avila A, Garcez A, Zisman A (2003) Revising rules to capture requirements traceability relations: a machine learning approach. In: Proceedings of the 15th International Conference in Software Engineering and Knowledge Engineering (SEKE 2003), pp 570–577, San Francisco
Zurück zum Zitat Spanoudakis G, Zisman A, Pérez-Miñana E, Krause P (2004) Rule-based generation of requirements traceability relations. J Syst Softw 2(72):105–127CrossRef Spanoudakis G, Zisman A, Pérez-Miñana E, Krause P (2004) Rule-based generation of requirements traceability relations. J Syst Softw 2(72):105–127CrossRef
Zurück zum Zitat Spivey J (1988) Understanding Z. Cambridge Spivey J (1988) Understanding Z. Cambridge
Metadaten
Titel
A study of methods for textual satisfaction assessment
verfasst von
Elizabeth Ashlee Holbrook
Jane Huffman Hayes
Alex Dekhtyar
Wenbin Li
Publikationsdatum
01.02.2013
Verlag
Springer US
Erschienen in
Empirical Software Engineering / Ausgabe 1/2013
Print ISSN: 1382-3256
Elektronische ISSN: 1573-7616
DOI
https://doi.org/10.1007/s10664-012-9198-8

Weitere Artikel der Ausgabe 1/2013

Empirical Software Engineering 1/2013 Zur Ausgabe

Premium Partner