nach oben

Automated Software Engineering

Erschienen in:

12.06.2019

An NLP approach for cross-domain ambiguity detection in requirements engineering

verfasst von: Alessio Ferrari, Andrea Esuli

Erschienen in: Automated Software Engineering | Ausgabe 3/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

During requirements elicitation, different stakeholders with diverse backgrounds and skills need to effectively communicate to reach a shared understanding of the problem at hand. Linguistic ambiguity due to terminological discrepancies may occur between stakeholders that belong to different technical domains. If not properly addressed, ambiguity can create frustration and distrust during requirements elicitation meetings, and lead to problems at later stages of development. This paper presents a natural language processing approach to identify ambiguous terms between different domains, and rank them by ambiguity score. The approach is based on building domain-specific language models, one for each stakeholders’ domain. Word embeddings from each language model are compared in order to measure the differences of use of a term, thus estimating its potential ambiguity across the domains of interest. We evaluate the approach on seven potential elicitation scenarios involving five domains. In the evaluation, we compare the ambiguity rankings automatically produced with the ones manually obtained by the authors as well as by multiple annotators recruited through Amazon Mechanical Turk. The rankings produced by the approach lead to a maximum Kendall’s Tau of 88%. However, for several elicitation scenarios, the application of the approach was unsuccessful in terms of performance. Analysis of the agreement among annotators and of the observed inaccuracies offer hints for further research on the relationship between domain knowledge and natural language ambiguity.

Vorheriger Artikel Requirements-driven evolution of sociotechnical systems via probabilistic reasoning and hill climbing

Nächster Artikel Heterogeneous defect prediction with two-stage ensemble learning

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

https://doi.org/10.5281/zenodo.1476902.

https://en.wikipedia.org/wiki/Category:Computer_science.

In the pseudo-code, array indexes start from 1.

When n = 2, the variance is equivalent to the mean squared error.

https://pypi.org/project/wikipedia/.

https://radimrehurek.com/gensim/.

https://spacy.io.

spaCy is used also to identify nouns through POS tagging. The authors of spaCy (Honnibal and Montani 2017) report a 97% accuracy for the pretrained POS tagger for English, which is in line with the state of the art (Manning 2011). Given the high accuracy, we consider the influence of POS tagger errors as negligible.

https://doi.org/10.5281/zenodo.1476902.

In this preliminary task, the authors annotated a group of 60 sentences for the (I3) Medical Software (CS, MED) and 60 sentences for the (M2) Medical Robot scenario (CS, EEN, MEN, MED)—the sentence sets were not included in the evaluation presented in this work. The annotation was followed by a discussion on the annotated data and evident cases of disagreement, which allowed the authors to have an initial common ground to select the examples.

The original Kendall’s Tau measure does not provide a policy to handle ties.

https://arxiv.org.

https://www.qualicen.de/en/.

https://qracorp.com.

https://wordnet.princeton.edu.

Preliminary experiments were performed in this direction, to obtain the currently adopted parameters’ values. However, we reckon that a systematic evaluation campaign is required, as there may be an optimal parameters’ selection for each domain group.

Agirre, E., Edmonds, P.: Word Sense Disambiguation: Algorithms and Applications, vol. 33. Springer, Berlin (2007). https://doi.org/10.1007/978-1-4020-4809-8

Agresti, A.: Analysis of Ordinal Categorical Data, vol. 656. Wiley, Hoboken (2010)MATHCrossRef

Ambriola, V., Gervasi, V.: On the systematic analysis of natural language requirements with CIRCE. ASE 13(1), 107–167 (2006). https://doi.org/10.1007/s10515-006-5468-2

Arora, C., Sabetzadeh, M., Briand, L., Zimmer, F.: Automated checking of conformance to requirements templates using natural language processing. TSE 41(10), 944–968 (2015). https://doi.org/10.1109/TSE.2015.2428709 CrossRef

Avramidis, E.: Rankeval: open tool for evaluation of machine-learned ranking. Prague Bull. Math. Linguist. 100, 63–72 (2013). https://doi.org/10.2478/pralin-2013-0012 CrossRef

Berry, D.M., Kamsties, E.: Ambiguity in requirements specification. In: do Prado Leite, J.C.S., Doorn, J.H. (eds) Perspectives on Software Requirements. The Springer International Series in Engineering and Computer Science, vol. 753, pp. 7–44. Springer, Boston, MA (2004). https://doi.org/10.1007/978-1-4615-0465-8_2

Berry, D.M., Kamsties, E.: The syntactically dangerous all and plural in specifications. IEEE Softw. 22(1), 55–57 (2005). https://doi.org/10.1109/MS.2005.22 CrossRef

Berry, D.M., Kamsties, E., Krieger, M.M.: From contract drafting to software specification: linguistic sources of ambiguity. https://cs.uwaterloo.ca/~dberry/handbook/ambiguityHandbook.pdf (2003). Accessed 10 June 2019

Blei, D.M.: Probabilistic topic models. Commun. ACM 55(4), 77–84 (2012). https://doi.org/10.1145/2133806.2133826 CrossRef

Camacho-Collados, J., Pilehvar, T.: From word to sense embeddings: a survey on vector representations of meaning. arXiv preprint arXiv:1805.04032 (2018)

Casamayor, A., Godoy, D., Campo, M.: Functional grouping of natural language requirements for assistance in architectural software design. KBS 30, 78–86 (2012). https://doi.org/10.1016/j.knosys.2011.12.009 CrossRef

Chantree, F., Nuseibeh, B., De Roeck, A.N., Willis, A.: Identifying nocuous ambiguities in natural language requirements. In: RE’06, pp. 56–65 (2006). https://doi.org/10.1109/RE.2006.31

Chen, X., Liu, Z., Sun, M.: A unified model for word sense representation and disambiguation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1025–1035 (2014). https://doi.org/10.3115/v1/D14-1110

Cleland-Huang, J.: Mining domain knowledge [requirements]. IEEE Softw. 32(3), 16–19 (2015). https://doi.org/10.1109/MS.2015.67 CrossRef

Nattoch Dag, J., Gervasi, V., Brinkkemper, S., et al.: A linguistic-engineering approach to large-scale requirements management. IEEE Softw. 1, 32–39 (2005). https://doi.org/10.1109/MS.2005.1 CrossRef

Dalpiaz, F., van der Schalk, I., Lucassen, G.: Pinpointing ambiguity and incompleteness in requirements engineering via information visualization and NLP. In: International Working Conference on Requirements Engineering: Foundation for Software Quality. Springer, pp. 119–135 (2018). https://doi.org/10.1007/978-3-319-77243-1_8

Dieste, O., Juristo, N.: Systematic review and aggregation of empirical studies on elicitation techniques. IEEE Trans. Softw. Eng. 37(2), 283–304 (2011). https://doi.org/10.1109/TSE.2010.33 CrossRef

Evans, M.C., Bhatia, J., Wadkar, S., Breaux, T.D.: An evaluation of constituency-based hyponymy extraction from privacy policies. In: Requirements Engineering Conference (RE), 2017 IEEE 25th International. IEEE, pp. 312–321 (2017). https://doi.org/10.1109/RE.2017.87

Falessi, D., Cantone, G., Canfora, G.: Empirical principles and an industrial case study in retrieving equivalent requirements via natural language processing techniques. IEEE Trans. Softw. Eng. 39(1), 18–44 (2013). https://doi.org/10.1109/TSE.2011.122 CrossRef

Femmer, H., Kučera, J., Vetrò, A.: On the impact of passive voice requirements on domain modelling. In: ESEM. ACM, p. 21 (2014). https://doi.org/10.1145/2652524.2652554

Femmer, H., Fernández, D.M., Wagner, S., Eder, S.: Rapid quality assurance with requirements smells. JSS 123, 190–213 (2017). https://doi.org/10.1016/j.jss.2016.02.047 CrossRef

Fernández, D.M., Wagner, S., Kalinowski, M., Felderer, M., Mafra, P., Vetrò, A., Conte, T., Christiansson, M.T., Greer, D., Lassenius, C., Männistö, T., Nayabi, M., Oivo, M., Penzenstadler, B., Pfahl, D., Prikladnicki, R., Ruhe, G., Schekelmann, A., Sen, S., Spinola, R., Tuzcu, A., de la Vara, J.L., Wieringa, R.: Naming the pain in requirements engineering. Empir. Softw. Eng. 22(5), 2298–2338 (2017). https://doi.org/10.1007/s10664-016-9451-7 CrossRef

Ferrari, A., Gnesi, S.: Using collective intelligence to detect pragmatic ambiguities. In: RE’12. IEEE, pp. 191–200 (2012). https://doi.org/10.1109/RE.2012.6345803

Ferrari, A., Spoletini, P., Gnesi, S.: Ambiguity and tacit knowledge in requirements elicitation interviews. REJ 21(3), 333–355 (2016). https://doi.org/10.1007/s00766-016-0249-3 CrossRef

Ferrari, A., Dell’Orletta, F., Esuli, A., Gervasi, V., Gnesi, S.: Natural language requirements processing: a 4D vision. IEEE Softw. 6, 28–35 (2017a). https://doi.org/10.1109/MS.2017.4121207 CrossRef

Ferrari, A., Donati, B., Gnesi, S.: Detecting domain-specific ambiguities: an NLP approach based on Wikipedia crawling and word embeddings. In: 2017 IEEE 25th International Requirements Engineering Conference Workshops (REW). IEEE, pp. 393–399 (2017b). https://doi.org/10.1109/REW.2017.20

Ferrari, A., Esuli, A., Gnesi, S.: Identification of cross-domain ambiguity with language models. In: Groen, E.C., Harrison, R., Murukannaiah, P.K., Vogelsang, A. (eds) 5th International Workshop on Artificial Intelligence for Requirements Engineering, AIRE@RE 2018, Banff, AB, Canada, 21 Aug 2018. IEEE, pp. 31–38 (2018a). https://doi.org/10.1109/AIRE.2018.00011

Ferrari, A., Gori, G., Rosadini, B., Trotta, I., Bacherini, S., Fantechi, A., Gnesi, S.: Detecting requirements defects with NLP patterns: an industrial experience in the railway domain. Empir. Softw. Eng. 23(6), 3684–3733 (2018b). https://doi.org/10.1007/s10664-018-9596-7 CrossRef

Firth, J.R.: Selected Papers of JR Firth, 1952–59. Indiana University Press, Bloomington (1968)

Flekova, L., Gurevych, I.: Supersense embeddings: a unified model for supersense interpretation, prediction, and utilization. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 2029–2041 (2016). https://doi.org/10.18653/v1/P16-1191

Gacitua, R., Sawyer, P., Gervasi, V.: On the effectiveness of abstraction identification in requirements engineering. In: Proceedings of the 18th IEEE International Requirements Engineering Conference (RE’10). IEEE, pp. 5–14 (2010). https://doi.org/10.1109/RE.2010.12

Gleich, B., Creighton, O., Kof, L.: Ambiguity detection: Towards a tool explaining ambiguity sources. In: REFSQ’10, vol. 6182. Springer, LNCS, pp. 218–232 (2010). https://doi.org/10.1007/978-3-642-14192-8_20

Guo, J., Cheng, J., Cleland-Huang, J.: Semantically enhanced software traceability using deep learning techniques. In: 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE, pp. 3–14 (2017). https://doi.org/10.1109/ICSE.2017.9

Harris, Z.S.: Distributional structure. Word 10(2–3), 146–162 (1954). https://doi.org/10.1080/00437956.1954.11659520 CrossRef

Hoffman, R.R., Shadbolt, N.R., Burton, A.M., Klein, G.: Eliciting knowledge from experts: a methodological analysis. Organ. Behav. Hum. Decis. Process. 62(2), 129–158 (1995). https://doi.org/10.1006/obhd.1995.1039 CrossRef

Honnibal, M., Montani, I.: spaCy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing (2017) (to appear)

Jha, N., Mahmoud, A.: Mining user requirements from application store reviews using frame semantics. In: International Working Conference on Requirements Engineering: Foundation for Software Quality. Springer, pp. 273–287 (2017). https://doi.org/10.1007/978-3-319-54045-0_20

Kendall, M.G.: The treatment of ties in ranking problems. Biometrika 33(3), 239–251 (1945). https://doi.org/10.2307/2332303 MathSciNetMATHCrossRef

Kof, L.: From requirements documents to system models: a tool for interactive semi-automatic translation. In: 2010 18th IEEE International Requirements Engineering Conference, pp. 391–392 (2010). https://doi.org/10.1109/RE.2010.53

Krisch, J., Houdek, F.: The myth of bad passive voice and weak words an empirical investigation in the automotive industry. In: 2015 IEEE 23rd International Requirements Engineering Conference (RE). IEEE, pp. 344–351 (2015). https://doi.org/10.1109/RE.2015.7320451

Lami, G., Gnesi, S., Fabbrini, F., Fusani, M.: The linguistic approach to the natural language requirements quality: Benefit of the use of an automatic tool. In: Proceedings 26th Annual NASA Goddard Software Engineering Workshop(SEW), vol. 00, p. 97 (2001). https://doi.org/10.1109/SEW.2001.992662

Landis, J.R., Koch, G.G.: An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 33(2), 363–374 (1977). https://doi.org/10.2307/2529786 MATHCrossRef

Lee, Y.K., Ng, H.T.: An empirical evaluation of knowledge sources and learning algorithms for word sense disambiguation. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10. Association for Computational Linguistics, pp. 41–48 (2002). https://doi.org/10.3115/1118693.1118699

Levy, O., Goldberg, Y.: Neural word embedding as implicit matrix factorization. In: Advances in Neural Information Processing Systems, pp. 2177–2185. http://papers.nips.cc/paper/5477-neuralword-embedding-as-implicit-matrix-factorization.pdf (2014). Accessed 10 June 2019

Lian, X., Rahimi, M., Cleland-Huang, J., Zhang, L., Ferrai, R., Smith, M.: Mining requirements knowledge from collections of domain documents. In: 2016 IEEE 24th International Requirements Engineering Conference (RE). IEEE, pp. 156–165 (2016). https://doi.org/10.1109/RE.2016.50

Maalej, W., Nabil, H.: Bug report, feature request, or simply praise? on automatically classifying app reviews. In: Proceedings of the 23rd IEEE International Requirements Engineering Conference, (RE’15). IEEE, pp. 116–125 (2015). https://doi.org/10.1109/RE.2015.7320414

Manning, C.D.: Part-of-speech tagging from 97% to 100%: is it time for some linguistics? In: International Conference on Intelligent Text Processing and Computational Linguistics. Springer, pp. 171–189 (2011)

Massey, A.K., Rutledge, R.L., Anton, A.I., Swire, P.P.: Identifying and classifying ambiguity for regulatory requirements. In: RE’14. IEEE, pp. 83–92 (2014). https://doi.org/10.1109/RE.2014.6912250

Mavin, A., Wilkinson, P., Harwood, A., Novak, M.: Easy approach to requirements syntax (EARS). In: RE’09. IEEE, pp. 317–322 (2009). https://doi.org/10.1109/RE.2009.9

Mich, L.: NL-OOPS: from natural language to object oriented requirements using the natural language processing system LOLITA. NLE 2(2), 161–187 (1996). https://doi.org/10.1017/S1351324996001337 CrossRef

Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013). arXiv:1310.4546

Miller, G.: WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)MATH

Navigli, R.: Word sense disambiguation: A survey. ACM Comput. Surv. (CSUR) 41(2), 10 (2009). https://doi.org/10.1145/1459352.1459355 CrossRef

Pedersen, T.: A simple approach to building ensembles of naive bayesian classifiers for word sense disambiguation. In: Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference, pp. 63–69. Association for Computational Linguistics. http://www.aclweb.org/anthology/A00-2009 (2000). Accessed 10 June 2019

Pedersen, T.: Unsupervised corpus-based methods for WSD. In: Agirre, E., Edmonds, P. (eds) Word Sense Disambiguation. Text, Speech and Language Technology, vol. 33, pp. 133–166. Springer, Dordrecht (2007). https://doi.org/10.1007/978-1-4020-4809-8_6

Pohl, K., Rupp, C.: Requirements Engineering Fundamentals. Rocky Nook Inc, San Rafael (2011)

Quirchmayr, T., Paech, B., Kohl, R., Karey, H.: Semi-automatic software feature-relevant information extraction from natural language user manuals. In: Proceedings of the 23rd International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ’17), pp. 255–272, Springer (2017). https://doi.org/10.1007/978-3-319-54045-0_19

Raganato, A., Bovi, C.D., Navigli, R.: Neural sequence learning models for word sense disambiguation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1156–1167 (2017). https://doi.org/10.18653/v1/D17-1120

Robeer, M., Lucassen, G., van der Werf, J.M.E., Dalpiaz, F., Brinkkemper, S.: Automated extraction of conceptual models from user stories via nlp. In: Proceedings of the 24th IEEE International Requirements Engineering Conference (RE’16), pp. 196–205. IEEE (2016). https://doi.org/10.1109/RE.2016.40

Rodriguez, D.V., Carver, D.L., Mahmoud, A.: An efficient wikipedia-based approach for better understanding of natural language text related to user requirements. In: 2018 IEEE Aerospace Conference, pp. 1–16. IEEE (2018) https://doi.org/10.1109/AERO.2018.8396645

Rosadini, B., Ferrari, A., Gori, G., Fantechi, A., Gnesi, S., Trotta, I., Bacherini, S.: Using NLP to detect requirements defects: an industrial experience in the railway domain. In: REFSQ, pp. 344–360. Springer (2017) https://doi.org/10.1007/978-3-319-54045-0_24

Sleimi, A., Sannier, N., Sabetzadeh, M., Briand, L., Dann, J.: Automated extraction of semantic legal metadata using natural language processing. In: 2018 IEEE 26th International Requirements Engineering Conference (RE), pp. 124–135. IEEE (2018). https://doi.org/10.1109/RE.2018.00022

Sultanov, H., Hayes, J.H.: Application of reinforcement learning to requirements engineering: requirements tracing. In: Proceedings of the 21st IEEE International Requirements Engineering Conference (RE’13), pp. 52–61. IEEE (2013). https://doi.org/10.1109/RE.2013.6636705

Taghipour, K., Ng, H.T.: Semi-supervised word sense disambiguation using word embeddings in general and specific domains. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 314–323 (2015) https://doi.org/10.3115/v1/N15-1035

Tjong, S., Berry, D.: The design of SREE a prototype potential ambiguity finder for requirements specifications and lessons learned. In: REFSQ’13, LNCS, vol. 7830, pp. 80–95 (2013). https://doi.org/10.1007/978-3-642-37422-7_6

Wang, W., Niu, N., Liu, H., Niu, Z.: Enhancing automated requirements traceability by resolving polysemy. In: 2018 IEEE 26th International Requirements Engineering Conference (RE), pp. 40–51. IEEE (2018). https://doi.org/10.1109/RE.2018.00-53

Yang, H., De Roeck, A.N., Gervasi, V., Willis, A., Nuseibeh, B.: Analysing anaphoric ambiguity in natural language requirements. Requir. Eng. 16(3), 163–189 (2011). https://doi.org/10.1007/s00766-011-0119-y CrossRef

Yuan, D., Richardson, J., Doherty, R., Evans, C., Altendorf, E.: Semi-supervised word sense disambiguation with neural models. arXiv preprint arXiv:1603.07012 (2016)

Yue, T., Briand, L.C., Labiche, Y.: aToucan: an automated framework to derive UML analysis models from use case models. ACM Trans. Softw. Eng. Methodol. (TOSEM) 24(3), 13 (2015). https://doi.org/10.1145/2699697 CrossRef

Zowghi, D., Coulin, C.: Requirements elicitation: a survey of techniques, approaches, and tools. In: Engineering and Managing Software Requirements, pp. 19–46. Springer (2005). https://doi.org/10.1007/3-540-28244-0_2

Titel: An NLP approach for cross-domain ambiguity detection in requirements engineering
verfasst von: Alessio Ferrari
Andrea Esuli
Publikationsdatum: 12.06.2019
Verlag: Springer US
Erschienen in: Automated Software Engineering / Ausgabe 3/2019
Print ISSN: 0928-8910
Elektronische ISSN: 1573-7535
DOI: https://doi.org/10.1007/s10515-019-00261-7

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 3/2019

Requirements-driven evolution of sociotechnical systems via probabilistic reasoning and hill climbing

Automatic B-model repair using model checking and machine learning

Heterogeneous defect prediction with two-stage ensemble learning

Guest editorial: special section on artificial intelligence for requirements engineering

Premium Partner