Machine learning-based analysis of adverse events in mesh implant surgery reports

To ensure data quality, reduce noise, and enhance coherence, the raw data underwent several preprocessing steps using the Python programming language. The following steps were undertaken:

Following data preprocessing, we have a dataset comprising 2422 reports, containing a total of 373,814 words. Once the stop-words were eliminated, the dataset was reduced to 272,486 words.

To analyze adverse events based on patient reports, Machine Learning approaches were employed, focusing on specific columns outlined in Table 1. To identify adverse events, we employed a combination of Named Entity Recognition (NER) and Keyword matching approaches (Kanya et al. 2016; Fong et al. 2018). NER is a text analysis technique used to identify and classify named entities in text, such as person names, organizations, locations, medical conditions, symptoms, drugs, etc. By analyzing linguistic patterns, contextual information, and syntactic features, NER algorithms can extract relevant entities from the text, aiding in understanding the context and extracting meaningful information (Jagannatha et al. 2019). In the context of adverse event identification, NER can be used to identify specific symptoms or medical conditions mentioned in patient reports, highlighted in italics. However, for a better understanding of the data, we also employed a Keyword matching approach. This involves creating a predefined list of keywords such as unigrams, bigrams or phrases related to adverse events (Nasar et al. 2019). Table 2 presents a list of keywords used for extracting adverse events. The keyword list comprises terms specifically related to mesh-implanted devices and their associated complications, which were derived from manual examination of reports and published in previous studies (Voreacos 2011; Watanabe 2012; Zhao et al. 2021). By matching these keywords against the patient reports, occurrences of the keywords can be identified. This approach helps capture explicit mentions of adverse events that may not be detected by the NER model (Wang et al. 2018).

The NER approach utilizes linguistic patterns and contextual information to capture a wide range of adverse events, while the keyword matching approach ensures that explicit mentions of specific adverse events are not overlooked. This combined approach enhances the accuracy and comprehensiveness of adverse event identification, enabling effective analysis and a better understanding of patient experiences. As a result, a comprehensive set of adverse events and their corresponding frequencies in the patient reports were identified by combining the outputs of NER and keyword-matching techniques. The identified adverse events and their frequencies are in Table 3. The higher frequencies of adverse events highlight key concerns such as pain, bowel problems, device problems, and sexual problems. Additionally, this study recognizes the presence of potential adverse effects with lower frequencies, which may not pose immediate hazards to patients but can still have a significant impact on their lives. These include allergic reactions, sleeping issues, constipation, and diarrhea.

Topic modeling aim to simplify large volumes of unstructured text by identifying and extracting smaller, more manageable topics. By doing so, we gain insights into the underlying themes present in the text data and obtain an overview of the discussed content (Bhattacharyaa et al. 2018). With topic modeling, we can also assess the density of topics within the text, allowing us to pinpoint what is significant and where it holds relevance. In this research, we have employed network topic modeling as outlined by Gerlach et al. (2018). In our approach, a 'topic' is formally defined as a cluster of words, where each word within a topic is assigned, a probability denoted by \(p(word|topic\)). This probability reflects the likelihood of the word's occurrence within that specific topic. The overall document is then represented as a mixture of these topics, characterized by the distribution \(p\left(topic|document\right)\), as proposed by Blei et al. (2003).

Our method utilizes a hierarchical stochastic block model (hSBM), as developed by Peixoto (2014, 2015), to perform community detection within a bipartite network of documents and words. This bipartite network structure allows us to avoid the assumption of a Dirichlet prior for the distribution of topics, which is a common limitation in traditional topic modeling approaches like Latent Dirichlet Allocation (LDA) (Selvi et al. 2019). Moreover, the hSBM does not require pre-specifying the number of topics, an advantage in dealing with data of complex or unknown topic structure.

Given the intricate nature of our dataset, we have incorporated post-processing techniques, particularly a coherence threshold measure, to refine and optimize the number of topics extracted. This coherence threshold, as suggested by Peixoto (2014), helps in pruning and merging topics based on their semantic quality and interpretability, ensuring that the resulting topics are both relevant and meaningful. After training the hSBM model and learning the topics, we modified the topic representation and applied coherence measures to the learned topics. We then selected a threshold value and pruned topics that fell below this coherence threshold. Through this process, we effectively identified five topics in the final representation. Furthermore, the efficiency of the model is evaluated by coherence score against with LDA. The pseudo code for the hSBM model is outlined in Table 4.

We linked discovered topics from hSBM model to their relevant adverse events based on their co-occurrence in the dataset. This association allows us to understand which adverse events are discussed within each topic (Gerlach et al. 2018). To visually represent these relationships, a Sankey diagram was created. In this diagram, each topic is depicted as a node, and the connection between the nodes represents the transition densities. The width of the flows in the diagram can be proportionate to the strength of transitioning from one topic to another (Śliwczyński et al. 2017). Utilizing the hSBM model and the Sankey diagram, we can effectively capture the associations between topics and the specific adverse events discussed in the corpus. This approach offers a comprehensive visualization that highlights the interplay between topics, their representative words, and the specific adverse events that emerge in the dataset.

We initially performed comprehensive data exploratory analyses on the processed reports through a combination of manual inspection and automated methods, including keyword extraction and Named Entity Recognition (NER) machine learning techniques, to gain a deeper understanding of the data. Figure 2 displays the trend in report frequency by year, with a noticeable increase post-2010, particularly in the years 2011–2012 and again in 2017–2018. The growing number of reports in these intervals indicates a need for closer examination of hernia mesh surgery practices during these times. In-depth analysis of these reports revealed that they predominantly detail the symptoms and adverse effects patients experienced post-surgery. Some reports specifically name the type of hernia surgery undertaken, while others mention hernia surgeries in broader terms. However, the described symptoms often provide enough detail to infer the specific types of surgeries. For illustrative purposes, a subset of these reports is included in Table 2 to enhance comprehension.

To ensure comprehensive coverage of adverse events related to hernia surgery, we compiled a list of the most commonly discussed adverse events reported by patients by using keyword extraction and NER. Table 3 presents the top 20 adverse events reported in the dataset, along with their corresponding frequencies. This analysis involved identifying specific keywords listed in Table 2 across the entire corpus and determining the number of reports that mentioned those keywords.

Table 3 reveals that the term “pain” is the most frequently mentioned adverse event in the dataset, occurring a total of 4,022 times and accounting for approximately 63% of the reports. Other commonly reported adverse events include bowel problems (approximately 30%), device problems (approximately 21%), sexual problems (approximately 21%), and urinary problems (approximately 15%). To gain a more detailed understanding of the adverse events over time, Fig. 3 illustrates the proportion of each adverse event from 2000 to 2021. The figure represents the normalization value, which is the number of reports related to a specific adverse event per year divided by the total number of reports for that event. Figure 3 focuses on the occurrence of the top 10 commonly reported adverse events, referred to as “Top 10”, and subtle adverse events, having less frequency in the dataset referred to as “Bottom 10”.

Several significant observations derived from the graphs, which shed light on the patterns of adverse events over time. In the Top 10 AEs category, there were notable peaks in the reporting of urinary problems, sexual problems, mental stress, device problems, and incontinence recurrence, occurring around 2011–2012 and again in 2017–2018. Additionally, chronic inflammation and bowel problems were also prevalent in this category. On the other hand, the Bottom 10 AEs category showed that prolapse recurrence, discomfort, sleeping issues, and urinary tract infections (UTIs) were the most frequently reported adverse events during the specified time span. Furthermore, an increase in reports of swelling, diarrhea, and allergic reactions was observed between 2017 and 2019 in this category.

Furthermore, the grouping of these adverse events indicates a relationship among them and suggests a potential association with specific hernia surgeries. This association will be further explored in the next section of our study on topic modeling.

To identify relevant groups of words aligned with the reported adverse events, we employed topic modeling using the hSBM (hierarchical stochastic block model) approach. The coherence score, which is a measure of how interpretable and meaningful the generated topics are, significantly aids in determining the effectiveness of a topic modeling algorithm (Fang et al. 2016). In our analysis, the coherence score of the hSBM model was found to be 0.69, compared to only 0.42 for the LDA model. This considerable difference in coherence scores suggests the superior effectiveness of the hSBM model for this dataset. Visual representations of our findings are provided in Fig. 4. On the left, word clouds depict the five topics discovered through the hSBM model, offering a visual interpretation of the topics' thematic content. On the right, topic density curves illustrate the respective distributions of these topics, providing insights into their prevalence within the dataset. The size of each word in the word clouds corresponds to the density \(p(word|topic)\) indicating the prominence of each word in a topic. In addition, we provide the document topic densities \(p(topic|document),\) represented by a Locally Weighted Scatterplot Smoothing (LOESS) curve, to show the variations in the density estimates (Jacoby 2000). As a result, Fig. 4 showcases distinct groups of adverse events represented by the majority of topics. Table 5 presents the representative terms for each topic based on descending density values.

Figure 4 depicts Topic 1, which revolves around discussions related to discomfort, hernia repair, device problems, mesh, and malfunction. These terms are frequently mentioned in conversations concerning the side effects reported after unsuccessful hernia surgeries (Robinson et al. 2005; Zhao et al. 2021). The topic implies that patients have reported infections and discomfort resulting from the use of mesh devices. The term "malfunction" suggests that issues can arise due to faulty or inadequately tested mesh devices. As a result, patients may experience infections, pain, discomfort, and other side effects, providing support for the association of Topic 1 with malfunctioned hernia mesh surgeries. Figure 5 showcases specific words that indicate this association and contribute to the labeling of the topics. The density graph presented in Fig. 4 representing the proportion of discussions focused on each topic at different time points throughout the timeline. The results indicate a slight increase in density during the early years, from 2002 to 2007, followed by a slight decrease. However, there is a notable surge in density around 2011–2012, followed by another increase in 2017–2018. This suggests that the issue of mesh malfunction was most frequently reported during 2011–2012 and later in 2017–2018. These findings are consistent with a study and the controversy surrounding implantable mesh devices raised by the FDA in 2011–2012, which prompted a reassessment of mesh implants (Watanabe 2012). Besides, the findings from Fig. 4 provide further support to this theory, as it shows that the prevalence of adverse events, specifically device problems, mesh erosion, and pain, aligns with the same timeline as Topic 1. Moreover, from Table 5, Topic 1 holds approximately 21% of the overall proportion among the five topics, underscoring its significance in the discussions.

Topic 2 comprises words such as “leak”, “urine”, “pains”, “cough”, “sneezing”, as well as “stress” and “incontinence”, indicating its strong association with stress urinary incontinence (SUI) (Togami et al. 2012; Mangir et al. 2020). The prominence of these words in the word cloud indicates a significant focus on SUI-related surgery within this topic. The density of Topic 2 in the Fig. 4 exhibits a similar pattern initially, but there is a noticeable increase in density during the years 2011–2012 and later in 2017–2018. This pattern aligns with the concerns raised by the FDA in 2011, specifically regarding the use of mesh in vaginal and SUI-related surgeries (Voreacos 2011; Togami et al. 2012). The proportion of this topic in Table 5 indicates that it is the most extensively discussed topic in the dataset, accounting for approximately 27% of total topic variation. The highest topic density in Fig. 4 and the distinct peak in the graph around 2011 further underscore the importance of this topic and its associated adverse events.

Topic 3 encompasses words such as “abdominal”, “inflammation”, “incisional”, “organs”, and “bladder”, which strongly relate to characteristics associated with incisional hernia repair (Harji et al. 2021), depicted in Fig. 5. The density graph in Fig. 4 visualizes the prevalence of this topic from 2000 to 2021, revealing a slight increase around 2007 and later in 2017, while remaining relatively low throughout the timeline. Additionally, the proportion of adverse events shown in Fig. 3, particularly bowel problems and chronic inflammation during the same time period, validates the appropriate labeling of this topic. The density values associated with this topic indicate that this specific surgery is not extensively reported in the dataset, potentially suggesting a consistent occurrence or a lower number of patient reports related to this event. The proportion of this topic is approximately 13%, which is the lowest among all the topics.

Topic 4 presents an intriguing focus on reports related to inguinal hernia repair. The presence of words such as “bulge”, “pressure”, “groin”, “unease”, “redness”, and specifically the term “inguinal” strongly indicates that this topic primarily centers on discussions concerning inguinal hernias and their associated symptoms (Lockhart et al. 2018). The topic density coverage illustrates a gradual increase in reported cases each year. While there was significant discussion on this topic in 2011–2012, the density experienced a sharp surge from 2015 to 2018, reaching its highest value. This indicates a higher number of reported inguinal hernia cases during that particular period. These findings align with the occurrence of adverse events depicted in Fig. 3. Furthermore, with an approximate proportion of 24%, Topic 4 emerges as the second most discussed topic in the corpus, highlighting the prominence of inguinal surgery as the second most reported procedure in the dataset. Besides, the increasing density values from 0.023 to 0.071 over time in Fig. 4 underscore the importance of carefully considering these surgeries in the future.

Topic 5 captures discussions that potentially revolve around umbilical hernias, as indicated by the presence of words such as “tissue”, “infected”, “belly”, “button”, “mesh”, and specifically the term “umbilical”. The density values associated with this topic range from 0.023 to 0.046 over the timeline, with a notable increase observed around 2012. The topic density suggests a relatively steady number of reported cases, except for the spike observed in 2012. This topic represents approximately 14% of all five topics in the corpus, maintaining a consistent proportion throughout the analysis.

In summary, topics 2 and 4 received the highest proportion of discussion among the five topics identified. The word clouds associated with each topic provided valuable insights into the symptoms and concerns discussed by patients. Figure 5 visually represents the key terms that significantly contributed to the labeling of the topics based on their word density values. Therefore, the topic modeling approach employed in this study proved to be effective in identifying the most discussed topics. The probability density of the topics, combined with existing research, further supported the validity of the topic modeling approach. Additionally, it allowed for the identification of trends in the occurrence of specific surgeries and provided valuable insights into patient opinions and experiences.

The Sankey diagram depicted in Fig. 6 the intricate relationships between the Top 10 and Bottom 10 reported adverse events (AEs) with related topics derived from hernia surgery patient reports. The diagram's band widths are proportionate to the frequency of connections between AEs and topics, offering a quantifiable measure of their associations (Martina 2019). Notably, "pain" emerges as a prevalent AE across the spectrum but shows a pronounced link to three specific topics: Topic 1, indicative of complications with malfunctioning hernia repair devices; Topic 2, related to stress urinary incontinence (SUI) surgeries; and Topic 4, associated with inguinal hernia procedures. “Infection” is another AE that spans all topics, with a substantial connection to Topic 3, concerning incisional hernia surgeries, and Topic 5, related to umbilical hernia repairs.

The diagram’s flow reveals a strong correlation between device malfunctions and AEs like pain and erosion in Topic 1, hinting at mesh erosion as a possible issue in device failure (Huerta et al. 2016). Conversely, Topic 2 encompasses a spectrum of AEs such as urinary complications and mental stress, expanding upon the issues noted in SUI surgeries as detailed by Mangir et al. (2020) and bringing additional concerns such as sleeping disturbances and mental stress to light.

The band of the sanky diagram in Fig. 6 also brings to light the prevalent AEs of nausea and allergic reactions for incisional hernia repairs in Topic 3, as detailed by Rastegarpour et al. (2016). Moreover, the diagram spotlights sexual difficulties, pain, bowel issues, and diarrhea as significantly associated with inguinal hernia surgeries in Topic 4. In the realm of umbilical hernia repairs, denoted as Topic 5, infection and constipation stand out as common patient-reported AEs.

All these findings revealing the multidimensional nature of patient experiences, the diagram allows us to trace the journey of individual AEs across different surgical contexts, offering insights into possible areas for medical intervention and patient care improvement. This visual representation serves as an analytical tool to pinpoint critical issues within the dataset, guiding healthcare professionals towards understanding the nuanced interplay of symptoms and surgical outcomes.