Contents for a Model-Based Software Engineering Body of Knowledge

A Body of Knowledge (BoK) is a set of concepts, terminology, and tasks that constitute a professional domain. Often, a BoK is developed by a professional association (e.g., ACM, IEEE) and captures the knowledge that is inherent, sometimes tacit, and often explicit in the interactions and literature that occur in that professional domain. The main goals of a BoK on a given discipline are:A BoK should also provide concrete deliverables, more specifically:A BoK should always be descriptive, but not prescriptive: Intentionally, it should not impose any particular method or tool, nor specific practices.

With respect to what should be considered as “generally recognized” or as a “good practice” of a discipline, the Project Management BoK (PMBOK) [16] offers precise definitions. First, “generally recognized” means that the knowledge and practices described are generally applicable to multiple kinds of diversified projects in various situations, and there is consensus about their value and usefulness. In turn, “good practice” means that there is general agreement that the application of skills, tools, and techniques can enhance the chances of success over a wide range of projects. It does not mean, however, that the precise knowledge or practice should always be applied to any project; the organization and/or project management team should ultimately be responsible for determining what practices are more appropriate for a given project in a certain situation.

Currently, there are a number of BoKs for various software-related disciplines; some are mature documents with a rigorous review and revision process (e.g., SWEBOK), while others are evolving (e.g., SLEBOK):

In 2004, the IEEE Computer Society established for the first time a baseline for the BoK of the field of software engineering. It was the outcome of a joint effort with ACM, whose mission was “to establish the appropriate set(s) of criteria and norms for professional practice of software engineering upon which industrial decisions, professional certification, and educational curricula can be based.”

The SWEBOK was developed as an international collective effort, in order to achieve the goal of providing a consistent global view of software engineering. The committee appointed two chief editors, several co-editors to support them, and editors for each of the Knowledge Areas. All chapters were openly reviewed, in an editing process that engaged approximately 150 reviewers from 33 countries. Professional and scientific societies, as well as public agencies from all over the world involved in software engineering, were contacted, made aware of this project, and invited to participate in the review process too. Presentations on the project were made at various international venues. The 2004 edition was revised in 2014, using the same editing process, giving birth in 2014 to the current version (v3) of the SWEBOK [4]. The SWEBOK has been adopted by ISO and IEC as ISO/IEC TR 19759:2005.²

It should be noted that the SWEBOK does not aim at describing the entire BoK for software engineering. Instead, it provides a “guide” to the existing BoK that has been developed since the start of the discipline—generally agreed to have happened in 1964, in the NATO conference held in Germany to discuss, for the first time, the software crisis.

One complete KA of the SWEBOK is devoted to Software Engineering Models and Methods (Chapter 9). As stated in the SWEBOK, software engineering models and methods impose structure on software engineering with the goal of making that activity systematic, repeatable, and ultimately more success-oriented. The use of models provides an approach to problem solving, a notation, and procedures for model construction and analysis. Methods provide an approach to the systematic specification, design, construction, test, and verification of the end-item software and associated work products.

Figure 1 shows the breakdown of topics for the SE Models and Methods Knowledge Area (Chapter 9):In addition to these topics, the SWEBOK lists the International Standards related to “Software Engineering Models and Methods” in its Annex B. There are three groups of standards, depending on their scope: modeling notations (such as IDEF, UML, OCL, or KDM); tools (IEEE Stds. 14102, 14471, and 1175, which apply to all CASE tools and their interoperability); and the environments of the systems (such as ISO/IEC 26515, on developing information in agile environments, and 15940 on Software Engineering environment services). One additional standard about terminology (ISO/IEC 24765) is common to all Knowledge Areas.

By looking at the concepts and related standards, we see that the coverage of MBE concepts and mechanisms is generally appropriate, but some essential concepts of MBE that should be part of the education of MBE practitioners—such as model transformations, executable models, or code generation, for instance—have not been included. Furthermore, although the SWEBOK provides definitions for some MBE concepts, not all of them are precisely defined, and in some cases the given definitions miss important features and characteristics of the defined concepts that have been later identified by the modeling community. In this respect, providing precise definitions for all the main MBE terms is another goal of the MBEBOK.

The field of Software Language Engineering (SLE) has emerged to connect and integrate different research disciplines such as compiler construction, reverse engineering, software transformation, model-driven engineering, and ontologies, in order to identify the principles and practices of engineering software languages—i.e., languages that may ultimately be implemented on a machine. SLEBOK is an ongoing initiative to capture a BoK for SLE. A Dagstuhl seminar [8] was held to capture a preliminary set of artifacts, definitions, methods, best practices, open challenges, etc. The intent was for these to be consolidated into the SLEBOK, which is currently evolving. While SWEBOK and several other BoKs have a mature process for their continued evolution and development, SLEBOK does not yet have such a process. However, SLEBOK is noteworthy in the fact that it is very open, and anyone can contribute to the revision process via its Git repository [19].

MBE principles and techniques can be used for Software Language Engineering (e.g., metamodels can be used to define the abstract syntax of software languages). As such, there are relationships between SLEBOK and MBEBOK. Although these relationships are still evolving, due to the relative immaturity of both BoKs, we can make some key observations:

The list of topics proposed for the contents of the MBEBOK is shown in Figs. 2 and 3. This list was produced in two stages. An initial proposal was presented at the MODELS Educators’ Symposium in October 2018 and discussed during the event with the audience. A poster with the proposal was also prepared and displayed during the conference, and the participants were asked to comment on its contents. Apart from the suggestions made directly to the poster presenters, sticky notes and pens were available for attendees to annotate the poster with comments at any time during the three days of the conference. A shared document was also available online after the conference so that any interested person could comment on it. We received numerous interesting and valuable comments and suggestions and refined the list of topics in November 2018.

The next step was to reach the wider modeling audience to get a feedback from the interested community. For this, we decided to conduct a survey study. A questionnaire was developed to sound out the opinion of the community about three main aspects: the topics included in the list, the importance assigned to each of them, and the coverage of the topics in the courses taught at the respondents’ institutions, at both Bachelor and Master levels. The outcomes are summarized in the following sections and depicted in Figs. 2, 3, and 4.

The proposed list contains the topics that were considered to be relevant for the MBEBOK, i.e., that should be part of the knowledge that any MBE practitioner should possess. The list proposed in the survey, which is shown in the first column of Figs. 2 and 3, is structured into the following nine sections.

The survey³ was conducted in December 2018. An invitation to participate was sent to all major Software Engineering and Modeling distribution lists, with a questionnaire where participants were asked to grade their perceived importance for each topic and whether the topic was covered in any of the courses taught at their institution. The detailed instructions given to the participants were the following:

“By assigning importance to each topic, you are defining the minimum set of concepts that an average MBS engineer should know, according to your own view of what an ‘average’ model-based software engineer is. In the questions below, ‘Not important at all’ means that this topic should not be part of the basic curriculum. “Covered” means that the topic is already covered in any of the courses taught at your institution, at Bachelor or Master level, or both. Respond indicating the level of coverage of the topic in the course(s).”

A 4-point Likert scale was used to record the importance assigned to each topic: (1) “Not at all important,” (2) “Slightly important,” (3) “Moderately important,” and (4) “Important.” An additional option “No opinion” was also included. Similarly, coverage was captured by a 4-value Likert scale: (1) “No,’ (2) “Slightly,” (3) “Enough,” and (4) “Well,” with an additional value “Do not know” to improve the reliability of the responses.

A total of 101 responses were recorded, from 23 different countries, distributed across continents as follows: Europe 85%, America 11%, Oceania 2%, Asia 1%, and Africa 1%. The results of the survey, including the raw responses from the participants, are available at http://​bit.​ly/​MBEBOK-2018SurveyResult​s.

Figures 2 and 3 display the results in tabular form. Each row represents a specific topic, with the average score and standard deviation of the assigned importance and coverage at BSc and MSc levels. The last two columns show the difference between the importance assigned to a topic and how it is covered in the curricula of the institution, in order to identify possible decompensations. Cells shaded in green color highlight the highest scores, while red-shaded cells identify the lowest. The highest standard deviations are highlighted in yellow; they identify the topics with less consensus.

For clarity, Fig. 4 shows the results of the survey graphically. The X-axis lists the topics (they are the same as those listed in Figs. 2 and 3, respectively), while the Y-axis plots the resulting scores of the answers: topic importance and topic coverage at MSc and BSc levels.

The results of the survey provided a very interesting feedback about our proposed list of topics.

(a) Topics with insufficient support. Some of the proposed topics received little support:(b) Topics with low coverage. Interestingly, there is a clear correlation between importance and coverage: In general, more important topics show higher coverage in existing curricula (the Pearson coefficient for the correlation between the assigned importance and the coverage at BSc level is 0.73 and 0.82 for the coverage at MSc level). There are significant exceptions, such as model maintenance and execution; despite being considered fairly important, their coverage is rather low.

(c) Relative importance of subtopics. There are some topics—namely 3.1 (structural model analysis), 8.1 (model transformation languages), and 9.1 (MBE application domains)—classified as Basic due to their importance, but the selected subtopics are, however, classified as Intermediate or even Advanced. This means that the topic as a whole is considered essential, but the subtopics themselves are slightly less relevant.

After analyzing the results of the survey, our proposal is summarized in the last column of the tables shown in Figs. 2 and 3, where we have classified each topic as: Basic, Intermediate, or Advanced. Topics marked with two hyphens (--) received little support, and hence, they have been excluded from the final proposal.

Topics were classified as Basic if their average importance score was above 3.45, and Intermediate if importance was above 3.0, which represent, respectively, an average of 86% and 66% of the max score in the Likert scale [1–4]. Advanced topics were those that scored at least 2.8 in importance (60%). We decided to discard those topics that did not reach a score of 2.8 in importance or did not get sufficient support, as discussed above. This classification is also consistent with the expected coverage of each topic at BSc and MSc levels. Basic topics should be taught at BSc level. Basic and Intermediate topics should be covered by all BSc and MSc courses on MBE. This approach would provide the required homogeneity and interoperability between the foundational contents of separate MBE curricula.

Of course, additional MBE topics (e.g., those not included in the list) could also be taught depending on the experience, industrial demands, and specialized offerings of each institution.

As mentioned earlier, the SWEBOK already provides good coverage of some of the concepts and practices of MBE. However, it misses out some key elements that should be part of the knowledge and skills of any model-based software engineer.

At the beginning of this effort, we considered several options to integrate our extension with the contents of the SWEBOK, from more conservative to more disruptive: (a) adding paragraphs to the existing text of Chapter 9 and new subsections if needed, but respecting its current structure; (b) replacing the contents of the entire Chapter 9; and (c) creating a complete BoK, separating from SWEBOK (this is the strategy that other BoKs, e.g., SLEBOK, adopted).

Based on the suggestions received at EduSymp’18, our proposal is to follow the same strategy as other extensions to BoKs, which are delivered as Appendices to them. Such a strategy is not intrusive nor disruptive and enables both the SWEBOK and the MBE-annex to separately evolve, but still maintaining consistency and minimizing redundancy.

The design, development, and maintenance of modeling languages could be considered as very important for any MBE practitioner. In fact, a number of topics listed in Sect. 3 already cover several aspects related to Software Language Engineering that apply to modeling languages. Although in theory the SLEBOK should address most of these topics, by looking at its current contents it does not seem to cover some MBE topics adequately. For example, simple topics such as syntax and semantics are not covered in their generality, or at least with not enough level of detail for our purposes.

Similarly to what we are proposing here with the SWEBOK, the relationship between the MBEBOK and the SLEBOK should be clarified, stating the scopes of both BoKs, identifying their intersecting concepts and mechanisms, and making sure that they are treated consistently in both guides. This is not a trivial issue, given the current status of the SLEBOK. However, the fact that it is still under development can also be an advantage: now that we have identified the topics that should be part of the MBEBOK, it would be a matter of defining an integration strategy that permits complementing the contents of both BoKs in a successful manner.

Although the closest connections of the MBEBOK are with the SWEBOK and the SLEBOK, overlaps with the other BoKs listed in Sect. 2.1 also exist. The identification and analysis of common and related topics with other BoKs is essential in order to maintain consistency and interoperability with the rest of the engineering disciplines.

In parallel with the development of the basic contents for the MBEBOK, a number of ongoing initiatives are surveying different aspects of how models are currently taught, for example, the experiences of instructors when teaching modeling and MBE topics [6] or the experiences of students with software modeling tools [2]. These initiatives nicely complement our present proposal and could be of significant value for the development of the Guide to the MBEBOK, e.g., to define the MBE skills and practices required to use the concepts identified in this work, and the kinds of tools needed by MBE practitioners and students.