What do class comments tell us? An investigation of comment evolution and practices in Pharo Smalltalk

To better understand class commenting practices of Pharo and achieve reliable results, we analyzed the core libraries of Pharo. We extracted the most recent revision of each major release of Pharo, from Pharo 1 to Pharo 7 (2008 to 2019), using a software analysis platform named Moose (Ducasse et al. 2005). For each version we used Moose (Moose 2020) to extract the class comments and meta details of the classes in the standard image, known as the Pharo core.² This includes classes to work with files, collections, sockets, streams, exceptions, graphical interfaces, unit tests, etc.

Table 1 shows the details of each version with version number, release date, the total number of classes and the total number of classes with comments.

Using this dataset,³ we measured the trend of commenting by calculating the ratio of commented classes to uncommented classes in each version. To investigate whether developers change comments of old classes, we tracked comment changes in already existing classes (old classes). For comment changes, we compared each class in a given version to its previous version to assess added comments, removed comments and changed content. Additionally, we tracked code changes of a class in comparison to the previous version to get an overall summary of the historical changes. To compute code changes we extracted the class definition (instance side and class side), all methods of the class, and source code of all methods of each class for each version.⁴

The result in Fig. 3 shows that the trend of commenting classes increases rapidly for initial Pharo versions, and is then maintained in subsequent versions. Indeed, in the figure, we can see that the percentage of commented classes, in light and dark blue (for old and new classes), increased in initial versions, and then remained constant from the fourth version.

Finding 1: The trend of commenting classes increases rapidly over the first three Pharo versions, from 50% of commented classes in Pharo 1, to 80% commented classes in Pharo 3 and subsequent versions.

Figure 3 also portrays the detailed aspect of classes that have survived from old versions and classes added in the current version. For instance, in version 3, we can see that the number of old classes without comments has decreased (height of the light orange bar segment decreased), and the number of old classes with comments has increased (height of light blue bar segment increased) implying that several old classes are commented in version 3, in addition to commenting new classes. In version 7, we can see a major effort being put into commenting new classes (77% of the new classes were commented) compared to old classes (12% of old classes were commented). In particular, 89% of the old classes from Pharo 6 survived to Pharo 7, of which 20% were uncommented classes and only 12% of the uncommented classes were commented in Pharo 7.

Finding 2: In later versions of Pharo, developers put effort into maintaining the code comment ratio, commenting new classes, and adding comments to old classes.

In addition, we find that developers change comments of old classes as shown in Fig. 4. Changing a class comment includes adding comments to an uncommented class, removing the comment, and updating the content of the comment.

Differentiating this change behavior in Fig. 4 highlights that in versions 2 and 3, developers focused more on adding comments to old classes compared to updating or removing the comment content. Since version 4, the focus of changing comments shifted to updating the content of class comments compared to adding comments to old classes. For example, in Pharo 7, more class comments are changed compared to comments added to old classes. To find the reason behind this behavior, we examine the code changes in old classes, and measure the extent to which developers update comments of old classes when changing their code.

From Fig. 5, we find that in Pharo 7, 52% of the old classes are changed either by changing code, comments, or both, indicating a major refactoring of the old classes. Nearly 44% of old classes are changed without updating their class comment. Specifically, 75% of these changes were related to adding, removing, or updating methods, but we found no corresponding changes in the class comments. We expected these changes to affect the class comments, due to the dedicated section in the class comment template for instance variables and key messages. In contrast, the changes such as renaming a package or changing a method category carry a lower tendency to affect the class comment. Only 7.9% of the old class comments are changed together with the code in Pharo 7, as shown by the dark red bar segment at the bottom of version 7 in Fig. 5. We further explored this segment by analyzing a sample of 15% of the 327 classes where both comments and code changed. We find that 50% of the changes in class comments are related to code changes, confirming the finding from earlier work (Fluri et al. 2009). In our analysis, the most specific types of code changes that triggered comment changes were the deprecation of a class and the addition of new methods. The rest of the code changes e.g., updating a method or class definition changes, triggered comment changes less frequently. In one particular case of code changes where a method is removed from the class, the method code is added to the class comment as an example. In contrast, in another similar case, the method comment is added to the class comment as implementation details. The reason for such a behavior can be the intent to keep the information about the removed method in the system for future tasks even though it is deleted. The remaining 50% of the comment changes are not related to code changes, even though 73% of the code changes in these classes are adding new methods, updating methods, or removing existing methods which can potentially trigger the comment changes, according to previous work (Fluri et al. 2009). These unrelated comment changes are about clarifying details of the class by changing the information types or formatting, improving the grammar, or changing the writing style from third person to first person or vice versa.⁵ We further analyzed which information types are frequently changed in comment changes irrespective of the code changes to find out the importance of specific information types. We found that most specific information changes in the class comments were about adding and updating the intent of the class, warnings, usage examples, and implementation details of the class, thus indicating the importance of these information types. On the other hand, in test classes (10% of the classes where code and class comments changed) the most specific information changes were about removing the bug-related details from the class comments in the next version. Analyzing what factors motivate developers to make such comment changes and at what stages of the project they change is the subject of future work.

Finding 3: In 50% of the cases, the code and class comments of old classes change together, with developers updating comments of the classes to keep them synchronized with the implementation.

Until now we separated the old classes from the new classes, but did not distinguish between the originating versions of old classes and those that survived from a specific version. For example, in Pharo 7, what portion of the classes survived from Pharo 1 or Pharo 2? This information is crucial to gain insight into comment coverage of a particular version in each version, and which class comments developers considered important to refactor in the current version. Furthermore, it helped us to analyze what happened to the old classes in the current version. For example, if the system went through a major refactoring, then which old version’s classes were deleted, re-introduced or modified? We therefore need to keep track of the history of a class, from Pharo 1 to latest version, to get an overall view of the evolution of the system. To answer all these questions, we track the origins of old classes and their survival history to the current version in Fig. 6.

In Fig. 6, each Pharo version is assigned a unique color. The shading indicates the distribution of classes with and without comments. The height of a bar segment in one color represents the classes surviving from a previous version to the new version. The original versions of each class are ordered by age with the oldest version at the bottom and the newest version at the top. Tracking the color of a version allows us to know how long classes are kept in the system. For example, in Pharo 1 the dark shade of green shows the classes with comments and a lighter shade of green shows the classes without comments. We find that until Pharo 6 the classes originating in version 1 still constitute the largest group of all older classes. In Pharo 3, major efforts were devoted to refactoring and re-documenting classes from older versions, 1 and 2. In Pharo 4, we observe that the ratio of adding comments to the new classes is less compared to preceding and succeeding versions except Pharo 1. Pharo 7 shows the effort of documenting old classes and new classes, thus achieving maximum coverage i.e. 80% of classes with comments among past versions.

In addition to showing the overview of a version, we also summarized the major projects that were added, removed and re-documented in each version in Table 2. We observed that the documentation of few projects such as Zinc, and Refactoring were actively updated, but whether it was due to their importance, or discipline of their developers, or both, is the subject of future work. We summarized the projects by grouping the added, removed and recommented classes by their package in each version. To verify our calculated list, we compare our project list to Pharo change logs.⁶ From the aforementioned analysis we collected several observations about Pharo commenting patterns:Finding 4: In Pharo 3, a major effort is put into adding comments to old classes whereas in subsequent versions, more effort is put into updating comments of old classes. Both cases show developers adding and updating comments of old classes.

The investigation performed on commenting trends presents important insights into the commenting habits of Pharo developers. These insights can assist developers and researchers in the following aspects:This investigation helped us to gather the general practices developers follow towards class commenting but does not characterize the content of the comments, nor does it describe how comments adhere to the commenting guidelines of Pharo. We cover these aspects in the rest of this paper.

To investigate the commenting practices, we studied the latest stable version of Pharo, namely Pharo 7. Since each class has one class comment, all the classes with class comments participated in the analysis dataset, resulting in a dataset of 6 324 classes. However, due to the semi-structured nature of comments and the lack of content headers or annotations, a content-wise investigation of comments requires manual effort, making the investigation of the whole dataset a non-trivial task.

We therefore selected a representative subset of comments for manual analysis by defining the required minimum sample size n with the following standard formula (Triola 2006):N is the size of the dataset, e is the margin of error, p is the percentage of picking a comment and the z is selected according to the desired confidence level. We calculated the required sample size from the finite population of 6 324 to reach a confidence level of 95% and error e of 5%. The z-score is 1.96 according to the confidence level and p is 0.5 used for the sample size needed. The resulting dataset should therefore contain a subset of 363 class comments in total. In order to choose 363 representative comments from the dataset, we investigated the distribution of comments based on the number of sentences present in a comment shown in Fig. 7. The sentences were separated using a custom-built Pharo sentence splitter. We found that the number of sentences in the comments varies from 1 to 272. Therefore, we used stratified random sampling approach to ensure that all kinds (or size) of comments are represented in the manual analysis dataset in case of skewed population. This approach divides the whole dataset into smaller strata based on the comment distribution, and allows random samples to be drawn from each stratum.

In order to select the 363 sample comments according to the approach, we used quintiles from the logarithmic distribution based on the number of sentences in each class comment shown in Fig. 7b. Accordingly, we obtained five quintiles as follows 1, 1, 2, 6, and 272. Based on the quintile values, we obtained comment strata, and calculated the comment proportion of each stratum shown in Table 3. We selected from each stratum a number of comments that correspond to the proportion of such comments in the entire dataset, following a random sampling approach without replacement. For example, from a total of 3 040 comments of comment stratum “1-1”, we selected 175 comments i.e. 48% of 363 comments using a random sampling approach without replacement. As the approach facilitates the selection of a random sample from a stratum and not all strata are entirely homogeneous (such as ‘3-6’ compared to ‘1-1’), we observed that the margin of error varies from 7% to 9% within strata (measured using the formula samplesize(n) for each stratum). On the other hand, this approach is known to increase the overall precision instead of that of the individual strata, thus helping us to better select representative comments.

To verify the practices of Smalltalk developers in other projects than the Pharo core, we analyzed the selected comments from seven external projects. We filtered the external projects from GitHub⁸ based on several criteria: (i) the project is not part of the Pharo core, (ii) it has an active project activity since 2019, and the project history spans at least two years with at least 600 commits, (iii) it is not a repository for books, an article, or documentation, (iv) it has more than five contributors, (v) the project does not contain more than 20% code from other programming languages to avoid polyglot projects, e.g., opensmalltalk-vm contains 89% code from C, and SmalltalkCI contains 35% shell scripts,⁹ and (vi) it contains more than 20 000 lines of Smalltalk code, to remove small projects thus the projects MaterialDesignLite,¹⁰ Kendrick,¹¹ and PharoLauncher¹² were removed.

We sorted the projects based on commits and size (based on lines of code), and selected the top seven projects. The projects consequently vary in size, domain, and contributors. For each project we followed the same methodology used for selecting representative Pharo core comments. Depending on the proportion of each project’s comments with respect to the comments of all projects, we selected the sample comments. We extracted 351 comments in total from the selected external projects and analyzed their information types.¹³

We conducted a pilot study to construct initial categories of the content of comments. We selected a sample of 100 classes from Pharo 7 classes with comments (6 324) using a random sampling approach. We used an open card-sorting approach and established the categorization procedure for the next larger-scale study. The study was performed by the first author, and the classification granularity was set to sentence-level. She manually analyzed the selected 100 classes, constructed new categories, and placed the comment sentences into appropriate categories according to the intent of the sentence. Thus, she formed 21 categories, among them seven categories being inspired by the recent Pharo template.

She constructed the category names by looking at the intent of the sentence and type of information, resulting in an initial draft of the Pharo-CTM.¹⁴ Once an initial taxonomy was elicited from the pilot study, we started the taxonomy study on 363 further comments to verify the completeness of the initial taxonomy, and to mitigate the chances of bias due to analysis by a single evaluator.

Our taxonomy study led to the finalization of Pharo-CTM, identifying 23 types of information (categories) present in the class comments The majority of these types, i.e. 21 categories, are taken from the pilot study even though several categories of the pilot study underwent the refinement process (renaming, merging) for the final Pharo-CTM. From these 21 categories, seven belong to the Pharo template while six categories were merged to three categories in the taxonomy study.¹⁵ The rest of the types, such as Subclasses Explanation, TODO comments, and Others, were added during the taxonomy study.

Table 4 presents an overview of this taxonomy. The list of 23 identified information types, with full details and examples is available online.¹⁶ The column Description describes the category, Implicitness level defines the degree to which information is hidden in the text, and keywords lists the keywords and patterns observed during manual analysis for each category. The implicitness level is taken from a five-level Likert scale with items Implicit, Often Implicit, Sometimes Implicit, Often Explicit, and Explicit. A category is marked Implicit when it is either in the same line or paragraph with other categories or without a header in the comment, making it difficult to identify. For example, the category todo is always mentioned in a separate paragraph with a header Todo, which makes it Explicit. On the other hand, a majority of the time the category Intent is combined with Responsibility in one line thus making them Often Implicit, but Collaborator is always combined with other categories in the same paragraph without a header. Based on the formulated criteria, one author evaluated the Implicitness level of each category, and other authors reviewed them and possibly proposed changes. All authors resolved the disagreements by the majority voting mechanism and refined the measurement criteria by mutual discussions. The examples for the categories are present in the respective category of classified comments.¹⁷ We found that in one-line comments developers usually describe the Intent of the class, and a very few times Responsibilities. A substantial number of comments contain warning information of some type (e.g., a note about the code, or behavior of the class, an important point to keep in mind while extending the class). In Others, we observed a few comments having the source code from other languages and following the commenting style of other languages, such as C and Java.

Figure 9 presents the distribution of the comments across all 23 categories. There are seven template-inspired categories, which are colored in blue and the remaining categories are colored in orange. The template-inspired categories contain the details proposed by the recent template. Other categories, composed of 16 definitions, contain comment details that developers deem important to understand their class and therefore mention in the class documentation.

Finding 5: The most recent Pharo class comment template suggests writing seven different types of details, namely Intent, Responsibility, Public API, Example, Instance Variable, Collaborators, and Internal details. Interestingly, developers frequently add other types of details such as Warnings, References to other classes and external docs, Dependencies, and Contracts in the class comments.

In external projects, we found all 23 types of information embedded by developers as shown in Fig. 10, though the frequency of some information types in comments is not as high as in Pharo core comments. For example, Collaborators, Implementation Points, Contracts, and Dependencies are not found so often in the external projects as in the Pharo core. Interestingly, we found that the project domain plays an important role in having a particular type of information. For instance, Roassal, a visualization engine project, contains a large number of Examples in the comments. Most of the examples are small code snippets to create different visualizations using the class. In contrast, we found detailed code examples (tutorials) in GToolkit class comments to explain how the project works. Additionally, we found that template-inspired categories are not used so often as in the Pharo core. On the other hand, some additional information types (not inspired by the template) are used more often than in the Pharo core. A few such information types are Links, Recommendation, Subclasses explanation, and References to other resources. Specifically, we found Links in less than 1% of Pharo core comments whereas nearly 6% of comments from external projects contain Links. This suggests that the Pharo core and external projects contain similar information types (23) but with different frequencies. Padioleau et al. analyzed operating system (OS) and non-OS projects and found similarities and differences in the kinds of details in project comments. However, whether these similarities are due to common developers or coding guidelines, if any, is not investigated (Padioleau et al. 2009). On the other hand, our preliminary investigation found few common developers from the external projects Moose, GToolkit who contributed to the Pharo core projects as well. Whether developers change their commenting practices in core and external projects would be an interesting topic to explore in the future. Additionally, investigating the impact of the template on external projects in addition to Pharo core comments can also highlight the differences in developer commenting practices across projects. In the future, we plan to investigate the impact of the template on external projects.

Discussion: A very few categories are explicit, such as Examples, and Instance variables, and they are generally indicated by a header, such as Usage, and Instance variables respectively. Most of the categories we found are implicit in the text and thus pose a challenge for the automated identification and extraction. However, we observed various patterns for them. Such patterns can help the researchers in designing approaches and heuristics to extract the specific information automatically. For implicit categories mentioned more frequently, we observed that developers mostly use common keywords to indicate the specific types of information in their comments. For instance, developers use a keyword Note while describing any kind of warning, sometimes as a header as shown in Listing 1, or in the first line of the warning shown in Listing 2 whereas in some cases the information is implicit in the text as shown in Listing 3. Similarly to the implicit warnings, instructions for using a class as in Listing 4 are implicit, without any header or specific pattern.

For categories like Intent, we observed that developers mostly mention the intent of the class in the first line of a comment. For Class references, we observed that class names are broken into words and not capitalized, thus making it hard to recognize the class name from the text. Pharo does not provide any language mechanism to support private or public scope for APIs, therefore APIs used by other services are generally marked Public by grouping such APIs in a protocol (interface) named Public, and documenting these in the class comment as a recommended practice. Additionally, we found that not all classes describe their public APIs in the class comments, and not all public APIs of the class are mentioned. The APIs mentioned are those that are considered to be important by the developer who is writing the comment e.g., the class “FTAllItemsStrategy” has eight methods, three of which are public APIs, but not all three are mentioned in the comment, and only one API “realSearch” is mentioned in the comment under the Public API and Key Messages section. Similarly, for other information types, developers follow different commenting practices, and the writing style shown in Table 4.

Finding 7: The top three types of information found in comments are template-inspired categories and these categories are implicitly present in the text, but developers mostly use common patterns or keywords in mentioning them.

All of these information types answer different developer questions in understanding the program, and assist them in various software development activities. LaToza et al. surveyed 179 developers during coding activities and collected the questions perceived as being hard-to-answer by developers (LaToza and Myers 2010). Questions about rationale, intent, and implementation are the topmost categories of those marked hard-to-answer by developers. In our study, we also found that developers mention intent, rationale, and implementation information in their comments with high frequency, indicating that developers find such pieces of information important. However, these information types are implicit in the text, which makes them hard to extract and present to the developers. Better tool support and more studies are needed to address the general problem of identifying information types and highlighting them to assist developers.

Based on our comparison analysis, code commenting practices vary across programming languages and systems. For common information types present in the comments across systems such as summary, links, code examples, we observed that they differ in the way they are located in the system and the way they are written. Hata et al. investigated the Links embedded in the comments and found top three links github.com, stackoverflow.com, and en.wikipedia.com (Hata et al. 2019). In our analysis, none of the links from Pharo core comments or external project comments point to github.com or stackoverflow.com. We did, however, find instances of Links pointing to en.wikipedia.com in Pharo external projects.

Padioleau et al. explored comments in different programming languages by focusing on Eclipse (IDE) written in Java, MySQL (a database server) and Firefox (a web browser) written in C and C++. (Padioleau et al. 2009). We observed similar information types with our taxonomy, such as code relationship, TODO, and deprecated code. In our work, we also observed these information types in both internal and external projects, though with lower frequency compared to Java, C and C++. Indeed, Padioleau et al. found that several projects embed often these specific concerns, which can vary among different domains. For example, OS-related projects contain a higher number of memory management, and lock/synchronization related concerns. In contrast, Eclipse comments include null references, error management, or links to issue tracker services (e.g., Bugzilla). Similar results have been reported by Pascarella et al. and Zhang et al. for code comments in Java and Python (Pascarella and Bacchelli 2017; Zhang et al. 2018). In our study, we find that class comments of Roassal contain a large number of code examples, with PolyMath containing more implementation details compared to other external projects. However, we did not find any error management related information, or links to issue tracker services in Pharo class comments. Similarly to other languages, Pharo class comments contain object-oriented programming guidelines or design pattern details. Hence, our results show a high diversity in commenting practices across various systems and languages. In future we plan to systematically and more precisely compare class commenting practices in other popular languages.

Finding different types of information embedded in class comment can assist developers to quickly find and access information required for various development tasks. In this section, we discuss the need of identifying information types in code comments of various application domains and languages. We then discuss language-independent approaches to organize and identify such information type automatically:

To study the evolution of the template, we extracted the template from each Pharo version since Pharo 1 and compared all template versions to record the differences.

In order to measure the adherence of commenting practices to the template, we extracted the class comment template and a sample of an equal number of classes from each version, then identified the information types they contain. The classes chosen for the study should be the newly added classes of each version, to make sure that the developer got a chance to look at the default template. This is because, in Pharo, the template appears only when developers add a class comment to the class for the first time. For each comment in the sample set (363) used in the RQ2, we therefore identified the original Pharo version when the comment was first added to the class. We then extracted the class comment of that version to compare the comment to the corresponding template in content and writing style aspects. For example, for a class comment added in Pharo 2, we compared the comment to the Pharo 2 template.

This partitioning of 363 comments according to the original Pharo version led to an unequal number of comments for each Pharo version e.g., out of 363 comments version 2 has fewer than 40 comments whereas version 7 has more than 60 comments. Furthermore, to compare the class commenting practices of all versions across each other, we selected an equal number of comments from each version. To balance the equal sample comments from each version, we set a lower threshold of 52 comments for each Pharo version, summing to a total of 364 comments. We extracted more comments from the Pharo versions where there were fewer than 52 comments, mainly Pharo 2 and Pharo 4. For each such version, we selected the sample classes from newly added classes with comments shown in the top dark blue segment of Fig. 3 according to the distribution of comments based on the number of sentences present in a comment. Similarly, we removed the classes from Pharo versions where there were more than 52 comments, mainly Pharo 1, Pharo 6, and Pharo 7, based on the distribution of comments of each version. We followed the same approach to choose representative comments as used in 363 comments from the earlier study (taxonomy study).

Assessing the adherence of comments to the suggested guidelines provides important directions on how to maintain comments and keep them consistent with such guidelines. Based on our study insights, we provide implications for developers and researchers to address the comment quality and consistency with commenting guidelines:

Considering the importance of code comments, several researchers have analyzed comments quantitatively and qualitatively. Woodfield et al. study the usefulness of comments quantitatively, and measure the effects of comments on program comprehension (Woodfield et al. 1981). They find that the groups of programmers who were given a program with comments were able to answer more questions about a program in a quiz than the programmers who were given the program without comments. A few studies focus on the evolution of comments. Schreck et al. qualitatively analyze the evolution of comments over time in the Eclipse project (Schreck et al. 2007), whereas Jiang et al. (Jiang and Hassan 2006) quantitatively examine the evolution of source code comments in PostgreSQL. Their focus is on comments associated with functions while we study the comments associated with classes in Pharo and focus on analyzing the comments quantitatively over Pharo versions.

Fluri et al. analyze the co-evolution of code and comments in Java and discover that changes in comments are triggered by a change in source code (Fluri et al. 2007). They find that newly-added code is rarely commented. Interestingly, in contrast to their results, we find that the commenting behavior of developers in Pharo is different. Developers comment newly-added code, as well as commenting old classes. In another study, Fluri et al. claim that the investigation of commenting behavior of a software system is independent of the object-oriented language under the assumption that common object-oriented languages follow similar language constructs to add comments (Fluri et al. 2009). We investigate the assumption with another object-oriented programming language and discover that Pharo follows a different comment convention for class comments. Pharo separates the class comment from the source code and supports different kinds of information like warnings, pre-conditions, and examples in class comments.

Comments contain useful information to support various tasks in software development cycle. Previous literature has explored this idea and analyzed various systems to find the information contained in comments. We mapped taxonomies of other related work to our work to establish which systems have been analyzed, which kinds of comments are frequently analyzed, and which categories from these works are available in our taxonomy in Table 5. Several categories from their taxonomy mapped to multiple information types in our taxonomy. We highlighted such categories with the symbol (M) in Mapping to our taxonomy in Table 5. In the next paragraphs, we discuss all these related works.

Ying et al. categorize a specific type of comment, namely Eclipse task comments, to see what information they contain. They categorize them on the basis of the various uses of the task comments, such as for communication, or to bookmark current and future tasks (Ying et al. 2005). Similarly Hata (Hata et al. 2019) categorized the links found in comments. Padioleau et al. use multiple dimensions to analyze comments and propose comment categories based on the meaning of a comment. They use W questions such as “What is in a comment?”, “Who can benefit?”, “Where is the comment located?”, and “When was the comment written?” Our aim is to support developers to find important and different kinds of information from the class comment so we choose one specific dimension, namely “What is in a comment?”, and classify Pharo class comments accordingly (Padioleau et al. 2009). Haouari et al. categorized the comments based on their position relative to code, comment type, style, and their quality (Haouari et al. 2011) Similar to their work, we also categorized comments based on their content. They proposed three subcategories of comment type, namely Explanation comments, Working comments, and Other. However, due to the abstract nature of these categories, especially Explanation comments, most of our categories can fit into it. We categorized the comments based on what specific types of information developers provide.

Steidl et al. assess the quality of comments in Java and C/C++ programs based on different comment categories. They proposed seven high-level categories based on the position and syntax of the comments, e.g., inline comments, block comments etc. (Steidl et al. 2013). We focus particularly on class comments, which map to their Header comments. Additionally in Pharo, four other categories (task comments, copyright comments, member comments, and section comments) from their work are available inside Pharo class comments, but are not annotated with any specific tags, and do not have a fixed position as in Java and C/C++. Farooq et al. compared comments of popular programming languages based on the types of symbols used to denote them, parsing rule, recursivity, and usage of the comments for various purposes such as documentation, and debugging (Farooq et al. 2015). In our case, the position of Pharo class comments is fixed and does not contain commented code as Pharo class comments are presented in a separate region, therefore, the categorization based on position does not apply to this case.

Pascarella et al. propose a taxonomy of code comments for Java projects (Pascarella and Bacchelli 2017). Five of our categories, namely Intent, Examples, Warnings, License, and References to external documentations, are close to their taxonomy categories Rationale, Usage, Notice, License, Pointer respectively. However, our categorization is specific to class comments. We found a number of cases in which the categories from their work did not fit Pharo comments, such as Ownership, Commented code, Directive, Formatter, Discarded, and Exception, due to unavailability of such information in the Pharo class comments. We found other, different types of information that developers write in Pharo class comments, such as warnings, observations, and contracts, that are not reported in their work. Zhang et al. constructed a Python comment taxonomy based on the work of Pascarella et al. (Zhang et al. 2018). Shinayam et al. identified the information embedded in local comments, as shown in Table 5 (Shinyama et al. 2018). Mapping to their work showed that Pharo class comments contain low-level information also in addition to high-level information. Based on the mapping analysis, several categories from related work did not map to our taxonomy. As the scope of comments we analyzed is different from other works e.g., Pascarella et al. and Zhang et al., it is still possible that other kinds of Pharo comments (method comments or inline comments) contain other missing information types. Additionally, all of the previous classifications have been performed on external projects of a language rather than internal core libraries such as String, or Collection. We categorized the comments from Pharo internal (core) and external projects to identify if developers have different commenting practices in internal and external projects. In future work, we plan to investigate the class comments of other popular languages and compare them to Pharo commenting practices.

Nurvitadhi studies the impact of class comments and method comments on program comprehension in Java, and creates a template for class comments in Java (Nurvitadhi et al. 2003). He suggests to include the purpose of the class, what the class does, and the collaboration between classes. The Pharo class comment template covers similar aspects with CRC style for the class comment. However, whether developers follow these aspects or not in their comments is unstudied. We therefore evaluate the adherence of the template to developer commenting practices. Jiang et al. study the source code comments in PostgreSQL. Their focus is on the function comments i.e. comments before the declaration of the function named header comments and comments within function body and trailing the functions named non-header comments. They observe that there is an initial fluctuation in the ratio of header and non-header comments due to the introduction of a new commenting style, but they do not investigate further about the commenting style (Jiang and Hassan 2006). Marin investigates the psychological factors that drive developers to comment (Marin 2005). The study concludes that developers use different comment styles in their code depending on the programming language they have used earlier. We also partially confirm this result as we find Java style block comments present in Pharo class comments. To best of our knowledge, we are first to conduct a study to evaluate the commenting style of developers, and measure the extent of their adherence to the standard guidelines.