Hybrid AI-Based dynamic risk assessment framework with explainable AI practices for composite product cybersecurity certification

The cybersecurity certification scheme establishes a systematic framework for evaluating software and hardware such as ICT products, services, and processes within the European regulatory landscape. It is a formal procedure that provides independent verification and validation that ICT products meet specified security requirements and can adequately protect against identified threats and vulnerabilities [2]. The certification framework becomes particularly critical for composite ICT product, which consist of multiple interconnected components from different vendors that must work together securely while maintaining individual security properties and collective system integrity. The European Cybersecurity Certification Scheme is the Common Criteria based European candidate cybersecurity certification scheme (EUCC) which follows CC based Security Evaluation, and ISO/IEC 15408 and ISO/IEC 18045 [15]. The EUCC framework provides three assurance levels - Basic, Substantial, and High - with increasing rigor of evaluation methods and depth of security analysis required to demonstrate compliance with security objectives [16]. The EUCC enables any applicable ICT product to undergo a formal Conformity Assessment (CA)–a security evaluation that certifies compliance against specific security objectives and requirements [17]. A Target of Evaluation (TOE) comprises the product’s software, firmware, or hardware, whereas for integrated systems, this extends to a Composite ToE (C-ToE) encompasses multiple individual TOEs that function requiring assessment of both individual component security and inter-component security relationships [18]. Security Profiles (SPs) define implementation independent security requirements for categories of ICT products that meet specific needs. The certification process is structured, starting with scope definition using the Target of Evaluation (ToE). There are two types of requirements need to be evaluated through the certification process including Security Functional Requirements (SFRs), which specify the mandatory security capabilities the ToE must implement (e.g., authentication), and Security Assurance Requirements (SARs), which define the necessary evaluation rigor to ensure the SFRs are correctly implemented, with confidence measured by the Evaluation Assurance Level (EAL).

However, the current certification processes are time-consuming and costly, characterized by a static approach that doesn’t consider continuous changes within the system context [2, 5]. Specifically, it relies on point-in-time assessments that fail to capture the dynamic security posture of systems that continuously evolve through updates and evolving threats and vulnerability exploitation. This makes maintaining continuous conformity extremely challenging particularity for complex and composite systems. Moreover, the audit evidences generated by the applications, and system also need proper explanation for the entire conformity assessment process, particularly when automated security tools generate evidences . This necessity drives the need for AI based dynamic risk assessment approaches and Explainable AI (XAI) by enabling real-time risk evaluation, automating vulnerability assessments, and providing the transparent, auditable evidence required for ongoing conformity assessment of these complex ICT products. The Figure 1 shows how the certification scheme flows from EU Cybersecurity ACT to certification levels and challenges.

Bild vergrößern

The integration of AI and XAI within the conformity assessment processes has recently getting attention particularly for evidence collection, assessment and justification with specific security requirements and objectives. Recent research demonstrates how AI enhances decision-making amidst complex risk scenarios and evolving regulatory landscapes, with AI-driven practices including proactive risk management, precise risk assessment, and real-time monitoring that reshape compliance operations [19]. The integration of explainable AI techniques enables auditors to validate automated compliance decisions and justify the adequacy of evidence against regulatory requirements, particularly in frameworks like GDPR, HIPAA, and PCI-DSS [20]. Papastergiou et al. addressed certification challenges by proposing a Composite Inspection and Certification (CIC) System specifically designed for cybersecurity assessment of interconnected ICT products, services, and processes within Digital Business Ecosystems [1]. The work emphasizes that product-level security assessment needs to consider not only individual component vulnerabilities but also cascading effects of cyber-attacks across interconnected systems. Investigation of regulatory implications incorporating XAI into cybersecurity frameworks highlights challenges of balancing transparency with cybersecurity requirements and evaluates practical methodologies for XAI implementation in audit contexts [21]. Empirical validation comparing LIME and SHAP applicability demonstrates that both techniques effectively explain model decisions, with SHAP providing more consistent global explanations while LIME offers superior local interpretability for individual predictions [22]. This research provides practical evidence that XAI techniques can maintain detection accuracy while providing the transparency required for audit purposes, addressing the critical need for explainable automated decision-making in security compliance frameworks. Another dimension of XAI technique, i.e., Permutation Feature Importance (PFI), is also used to demonstrate feature importance, similar to SHAP. PFI is widely considered for specific AI models including Random Forest (RF) due to its model-agnostic capability to measure global feature significance by evaluating the decrease in model accuracy when a feature’s values are randomly shuffled [23]. However, PFI can be biased when features are highly correlated, which necessitates careful application or the use of conditional variants [24, 25].

The integration of hybrid AI architectures and large language models has emerged as a transformative approach in cybersecurity applications, offering enhanced capabilities for threat detection, vulnerability assessment, and security decision-making. Advanced hybrid frameworks integrate multiple learning paradigms including stacking ensemble methods, Bayesian model averaging, and conditional ensemble approaches to achieve enhanced accuracy in intrusion detection and threat classification [26]. These systems leverage meta-classifier approaches to combine outputs from diverse base classifiers, enabling robust performance across varied attack scenarios while maintaining explainable decision rationale. Large language model applications in cybersecurity demonstrate superior capabilities in understanding both natural language and code semantics, enabling comprehensive analysis of security-related documentation, code repositories, and threat intelligence [27]. Hybrid architectures combining LLMs with traditional machine learning methods have shown enhanced performance in vulnerability assessment applications, leveraging natural language understanding capabilities while maintaining efficiency and interpretability of conventional ML algorithms for structured data analysis [28]. These integrated approaches provide more holistic threat evaluation capabilities than single-paradigm solutions.

In summary, the existing works reveal the significant of AI for cybersecurity with limited focus on the certification. Moreover, the black-box nature of AI models creates fundamental challenges towards transparent and auditable decision-making process. The adoption of XAI techniques mainly focuses on cybersecurity context rather than formal cybersecurity certification schemes. Specifically,there is a lack of systematic comparison regarding the evolution of Large Language Models–from GPT-3’s few-shot learning to GPT-4’s complex reasoning–to determine the optimal balance of instruction-following and cost-efficiency required for certification tasks. Additionally, while Permutation Feature Importance (PFI) is recognized for demonstrating feature importance in specific models like Random Forest [23], its specific application alongside SHAP and LIME to mitigate correlation biases and validate audit evidence within a certification framework remains strictly limited. This research gap necessitates the development of a comprehensive framework that combines the predictive power of hybrid AI models with explainable techniques specifically tailored for cybersecurity conformity assessment, enabling certification bodies to leverage advanced AI capabilities while maintaining the transparency, traceability, and systematic validation required for formal certification processes in increasingly complex digital ecosystems.

The existing XAI techniques can be broadly categorized based on their approach to generating explanations and the type of insights they provide [29]. Three primary explainable AI methods support cybersecurity certification: Marginal Feature Contribution (sensitivity analysis of parameter changes), SHAP (mathematically rigorous global/local explanations), and LIME (instance-specific interpretable models). As shown in Fig 2, these techniques differ in scope and presentation but collectively enable comprehensive AI decision explanations for certification purposes.

Bild vergrößern

To align the evaluation of XAI with the specific assurance requirements of the EUCC and Common Criteria schemes [4, 15], this framework adapts standard interpretability metrics into certification-specific criteria. The evaluation focuses on two dimensions critical for generating valid audit evidence:

The integration of XAI can effectively support the cybersecurity certification processes and several existing standards and frameworks such as the EU Artificial Intelligence Act, ISO/IEC 42001 and the NIST AI Risk Management Framework – call for transparent, accountable AI-driven security systems that can demonstrate trustworthy operation throughout their lifecycle [31‐33]. As XAI provides five critical capabilities for conformity assessment:

The framework adopts a hybrid XAI approach derived from the taxonomy of explainable methods for cybersecurity established by Yan et al. [30] and Elkhawaga et al. [29]. While numerous interpretability methods exist, the selection of SHAP, LIME, and Marginal Analysis is empirically justified by the specific requirements of the Common Criteria (CC) and EUCC schemes. Certification audits require not just model transparency, but distinct types of evidence: global consistency to prove unbiased risk scoring, local fidelity to justify individual asset classifications, and sensitivity analysis to validate the adequacy of security countermeasures. Consequently, the framework integrates these three techniques as follows:

The hybrid model combines ensemble learning with GPT-3.5 to improve vulnerability exploitation score prediction accuracy while maintaining computational efficiency. This integration provides three key benefits: ensemble methods excel at identifying predictive features from large cybersecurity datasets through statistical analysis [30]; GPT-3.5 provides superior contextual understanding to interpret complex feature relationships, recognizing patterns traditional algorithms miss regarding vulnerability attributes, exploit availability, and temporal factors [31]; and feeding only ensemble-selected features to GPT-3.5 achieves better accuracy while reducing computational overhead [32]. The model employs a two-stage approach:

Bild vergrößern

The process follows the Common Criteria-based EUCC scheme and consists of four sequential phases where initial phases define the scope of the C-ToEs , calculate the risk level and formulate the risk-based protection profile. Finally, the process operationalises the XAI to support the auditor with the conformity assessment. This process is unique in integrating dynamic risk assessment with explainable AI validation within the established Common Criteria framework, enabling objective quantification of control effectiveness beyond traditional checklist-based evaluations.

The C-ToE vulnerability dataset requires systematic analysis to identify and prioritize vulnerabilities based on their exploitation potential for risk assessment purposes. The implementation process begins with the systematic gathering of vulnerability and threat data from diverse industry sources. As illustrated in Fig 4, the framework aggregates data from static vulnerability databases (containing severity metrics), dynamic threat intelligence feeds (providing real-time exploitation status), and vendor-specific security advisories. This multi-source aggregation is critical for establishing a ground truth that reflects both technical severity and actual industry risk. Following gathering, the raw data undergoes rigorous preprocessing–including cleaning, filtering, and categorical encoding–to generate the structured training set required for the hybrid model. This task aims to extract key vulnerabilities that demonstrate high exploitation likelihood from the comprehensive vulnerability dataset, enabling effective mapping between vulnerability characteristics and exploitation probability. The task adopts the proposed hybrid model in a systematic manner to predict vulnerability exploitation scores, which are subsequently used to calculate relevant risk levels for conformity assessment. This task consists of four distinct steps that progress from data preprocessing through model evaluation.

Bild vergrößern

Data Preprocessing:This step involves cleaning, formatting, and optimizing vulnerability data specifically for hybrid model training, focusing on vulnerabilities affecting C-ToE assets identified in Phase 1.Key Feature Selection using Ensemble Learning: This step extracts key features using three machine learning algorithms that examine data from different perspectives to identify features predicting vulnerability exploitation probability. Random Forest builds multiple decision trees and calculates feature importance by measuring each feature’s contribution to reducing prediction error, Gradient Boosting builds sequential models learning from previous mistakes to identify features most useful for correcting errors and capturing complex patterns, and ElasticNet applies mathematical penalties to irrelevant features while preserving important ones through regularization. Each method has distinct strengths: Random Forest handles mixed data types with stable rankings, Gradient Boosting captures complex non-linear patterns, and ElasticNet provides rigorous linear selection, but combining all three provides robust feature selection that leverages each method’s benefits while mitigating individual weaknesses. Individual rankings are combined using weighted averaging with optimal weights determined through cross-validation experiments to optimize prediction performance while maintaining computational efficiency for GPT-3.5 processing.

Coefficient of Determination(R\(^2\)):R\(^2\) shows how much of the actual variation in vulnerability scores is explained by the model as mathematically presented in equation 1. As presented in equation 1, R\(^2\) value close to 1 means the model’s predictions align closely with real-world patterns, while a value close to 0 indicates poor predictive capability [43].Where:Root Mean Square Error(RMSE): RMSE represents the average size of the error between predicted and true scores, with a higher penalty for larger mistakes as mathematically presented in equation 2. Lower RMSE values indicate that the model’s predictions are generally close to the true values. RMSE is expressed in the same units as the target variable, making it directly interpretable [44].Where:Mean Absolute Error (MAE): MAE calculates the average absolute difference between predicted and actual scores, giving a simple, easy-to-interpret measure of how far off the predictions are on average as mathematically presented in equation 3. Unlike RMSE, MAE treats all errors equally without penalizing larger errors more heavily [45].Where:The above-mentioned metrics provide the capabilities to confirm how accurately the model is trained. That are not linked with each other, however collectively offer a multidimensional evaluation of model performance. R\(^2\) indicates how well the hybrid approach captures vulnerability exploitation patterns by measuring the proportion of variance explained. RMSE and MAE provide complementary perspectives on prediction accuracy, with RMSE being more sensitive to outliers and MAE offering a more robust measure of typical error magnitude. Cross-validation ensures consistent performance across different data splits, validating the model’s robustness for vulnerability score prediction applications. The integration of these metrics provides a comprehensive picture of hybrid model performance and serves as evidence for the added value of combining ensemble learning with GPT-3.5 contextual analysis, ultimately enhancing predictive accuracy and reliability in real-world cybersecurity scenarios.

This task ranks the vulnerabilities based on the predicated exploitability score by using the proposed hybrid model. The rank is scaled into five priority levels based on their exploitation probability ranges:

This final task calculates the risk level for the identified asset so that suitable control can be selected according to the defined security declaration and risk level. The individual risk level calculation depends on the related vulnerability score for each ToE. The assessment generates a comprehensive risk profile that lists all vulnerability-asset combinations with their individual risk levels. This detailed approach enables security teams to prioritize specific vulnerabilities affecting specific assets, supporting more targeted and effective risk mitigation strategies Hence, each vulnerability-asset combination receives a risk classification based on the vulnerability’s exploitation probability and potential impact:Once the risks are prioritized, it is necessary to declare the controls for mitigating the risks. This control selection is a decision-making process to choose the right control strategy and suitable recommended controls. Hence, the C-ToEs Owner, based on the defined security objectives, the respective SFRs and the risk assessment results, shall undertake optimal decisions to apply security controls that mitigate vulnerabilities and/or threats and satisfy the defined SFRs on the C-ToEs. The control selection reviews the identified vulnerabilities as risk factor for the risk event. Therefore these risk factors allow to considers the most common vulnerabilities that have impacted on the most assets which require immediate attention. The control selection uses mapping procedure to recommends security controls provided by the NIST SP 800-53 catalogue per specific SFR [46]. The catalogue provides list of security and privacy controls to safeguard the assets of the organization

This step extracts the most influential features from the key features selected by the ensemble learning component in Phase 2, identifying which specific factors have the strongest impact on vulnerability score predictions and should be the focus for security control validation. SHAP analysis calculates Shapley values for each feature across all predictions to determine average contributions to final vulnerability scores, providing consistent global importance rankings by examining feature contributions across the entire dataset and identifying features with the most systematic impact on model predictions. LIME analysis generates local explanations for specific vulnerability predictions by creating simplified interpretable models around individual instances, perturbing input features of high-scoring vulnerabilities and observing how changes affect predictions to identify features most responsible for specific high-risk predictions. The combination of SHAP and LIME produces a refined set of influential features that have the strongest impact on vulnerability score predictions, providing a focused list of critical vulnerability characteristics that should be addressed by security controls.

The process systematically varies each influential feature across its value range while holding other features constant. This allows the system to observe the impact on vulnerability score predictions and identify the values that produce the lowest risk scores (the "Optimal Counterfactual State"). By comparing the current feature distribution against this optimal state, the framework measures the potential improvement available through better control implementation. To ensure the assessment is rigorous, this step employs a defined decision framework based on statistical distribution and residual risk management:

The translation engine functions through a simple mapping pipeline:For example, a "Adequate" pair triggers the following template structure:

Template ID: TPL-CRIT-ADQ

Structure: "The risk classification for [Asset Name] is driven by [Mapped Feature Name] (Importance: [SHAP Value]), which exceeds the critical threshold of [\(\mu +\sigma \)]. While the attack potential is elevated, the implemented control [Control ID] demonstrates a gap closure of [\(\varDelta G\)%], satisfying the adequacy threshold for AVA_VAN compliance."

By strictly adhering to these string-based definitions, the framework ensures that the generated evidence satisfies the Repeatability and Reproducibility requirements of the ISO/IEC 15408 evaluation methodology [4].

P-NET is a pioneering Competence Center dedicated to advancing 5G/6G and emerging digital technologies, operating as an advanced 5G infrastructure facility providing carrier-grade and commercial-grade private 5G Standalone (SA) networks [48]. The organization serves as a dynamic testing and experimentation environment for pre-commercial verification, technological innovation and tailored vertical deployments under the Network as a Service (NaaS) model with customizable and programmable network capabilities. P-NET bridges the gap between academic research and industrial implementation of next-generation telecommunications technologies, providing comprehensive research and development services for advanced wireless communications, edge computing applications, IoT integration, and XR applications while collaborating with major telecommunications operators, equipment manufacturers, and academic institutions to accelerate 5G/6G solution development and deployment.

The Testing and Integration Service provides advanced software-intensive facilities for users to integrate and test new 5G/6G technologies in controlled, pre-commercial validation environments replicating real-world operational conditions, with key functionalities including 5G Network Function Testing for VNFs and Network Services deployments through OpenSlice OSS orchestration, Multi-Tenant Testing Environment provision through network slicing capabilities enabling simultaneous experiments by multiple research teams, Edge Computing Validation for MEC applications with low-latency processing utilizing 1082 CPUs and 4.5TB RAM, Radio Access Network Testing across four indoor locations using AW2S and ERICSSON radio units supporting handover and mobility scenarios, and Interoperability and Compliance Testing for equipment validation against 3GPP and ETSI specifications. These functionalities minimize pre-commercial validation costs, reduce technical integration uncertainties, and accelerate time-to-market for 5G/6G innovations.

The experimental validation implements a two-stage computational approach to validate the hybrid ensemble-LLM framework for vulnerability exploitation score prediction.We conducted an experiment to efficiently train and evaluate our proposed hybrid AI model–an ensemble learning + LLM (GPT-3.5) framework designed for dynamic vulnerability risk assessment within composite ICT product conformity evaluation–using the comprehensive CVEJoin dataset. CVEJoin aggregates over 200,000 CVE-mapped vulnerability records, enriched with diverse features from sources such as the National Vulnerability Database, threat intelligence feeds, and social indicators. To ensure a valid predictive model and avoid data leakage, the dataset was split into input features and a target variable. The input features consist of static technical metrics (e.g., CVSS scores, CWE classifications) and dynamic contextual signals (e.g., exploit counts, CTI mentions). The EPSS score was strictly isolated as the prediction target and was never included in the input feature set to prevent data leakage. Predicting the EPSS score rather than using the raw value is essential for certification because raw scores lack context. By predicting the score based on C-ToE specific features (e.g., specific vendor configurations, attack vectors, and CTI mentions), the model generates a context-aware risk score that reflects the specific application domain, rather than a generic global probability. Unlike raw EPSS, which provides a static probability derived from global data, our predictive approach allows the model to weigh specific technical attributes–such as the presence of a ’Network’ attack vector combined with ’Low’ attack complexity–differently depending on the learned risk profile of the composite product. This granularity enables the XAI components (SHAP/LIME) to attribute risk to actionable technical flaws rather than an opaque probability score, directly supporting the conformity assessment requirement for explainable, evidence-based risk estimation.

This section presents the comprehensive implementation of the proposed hybrid AI-driven conformity assessment framework through systematic application to the P-NET Testing and Integration Service use case. The implementation demonstrates the practical application of the four-phase methodology developed in Chapter 4, validating the framework’s effectiveness in supporting dynamic risk assessment for composite ICT product certification.

The proposed hybrid model demonstrates significant advancement in vulnerability exploitability prediction through strategic integration of traditional machine learning algorithms with large language model capabilities. The framework achieves superior predictive performance with R\(^2\) score of 0.891 compared to ensemble-only approaches at 0.852 and individual algorithms ranging from 0.745 to 0.847, representing 4.6% improvement over ensemble methods and 19.6% enhancement over the weakest individual algorithm. This performance improvement translates directly to more accurate risk assessment capabilities for composite ICT product certification, enabling certification bodies to make evidence-based decisions regarding vulnerability impact and control adequacy. The ensemble learning component provides robust feature selection through integration of Random Forest, Gradient Boosting, and ElasticNet algorithms. The systematic reduction from 52 engineered features to 24 key features through ensemble consensus demonstrates the framework’s ability to distil complex vulnerability datasets into manageable feature sets suitable for GPT-3.5 processing without sacrificing predictive accuracy. GPT-3.5 integration provides contextual analysis capabilities that traditional machine learning approaches cannot achieve through statistical pattern recognition alone, with training convergence from initial loss of 0.0923 to final validation loss of 0.0521 over twenty epochs demonstrating effective model adaptation with minimal overfitting. The economic feasibility represents a significant practical advantage, with structured prompt templates requiring approximately 300 tokens per prompt. For the training dataset of 4,770 records, total training cost is estimated at $2.91 for complete GPT-3.5 fine-tuning. Furthermore, the fine-tuning process was computationally efficient, completing in approximately 45 minutes on the specified NVIDIA A100 hardware. Cross-validation performance demonstrates consistent model behaviour with mean R\(^2\) of 0.887 and standard deviation of 0.009, indicating superior stability compared to individual algorithms and ensemble-only methods.

The integration of explainable AI techniques effectively supports cybersecurity conformity assessment by providing clear, understandable explanations of how security decisions are made. Rather than relying on traditional checklist approaches, the framework enables auditors to see exactly which factors influence risks and justify the chosen security controls. In particular, SHAP analysis identified the key features, i.e., Base_score (0.168), has_public_exploit (0.142), whereas LIME analysis shows that public exploit availability is the key feature for almost 87% critical vulnerabilities, with an average influence of 0.221. This consistency across different analysis methods builds auditor confidence in the framework’s reliability. The adoption of XAI met the established explainability criteria. Specifically, fidelity was demonstrated through consistent feature rankings that aligned with cybersecurity principles - vulnerabilities with higher CVSS scores and available exploits logically received higher risk ratings. Scalability was achieved both computationally, processing 1,193 vulnerability predictions efficiently, and cognitively, maintaining clear explanations whether analysing individual vulnerabilities or summarizing risks across all five P-NET assets. The marginal analysis methodology enabled systematic validation of security control effectiveness beyond simple pass-fail decisions. For P-NET, the evaluation revealed that 21 out of 24 implemented NIST controls adequately satisfy their security requirements, while identifying specific areas needing improvement in privilege management, configuration standards, and patch deployment. This detailed assessment helps organizations understand not just whether their controls work, but how well they work and where improvements are needed.Moreover, beyond the existing practice of XAI presented in this paper, future work can consider the framework as a defense mechanism against adversarial attacks and human errors. By exposing the precise feature contributions driving a risk score, techniques like SHAP could allow auditors to detect adversarial perturbations where an attacker might subtly alter input features to artificially lower a risk classification; the resulting explanation would reveal the inconsistency between the hidden high-risk attributes and the low score. Furthermore, future research may investigate how the framework mitigates human errors leading to attacks by transforming complex, opaque risk data into interpretable evidence, thereby reducing the cognitive load on auditors and minimizing the likelihood of overlooking vital misconfiguration during the certification process.

In comparison with existing works, research in regulatory requirements engineering demonstrates the growing need for automated compliance approaches, as evidenced by Kosenkov et al. [49] who identified a lack of structured cybersecurity certification approaches in the current state of the art. While recent studies explore automated analysis of GDPR requirements [50], compliance verification in Android apps using LLMs [51], and generative AI for traceability [52], our research uniquely addresses cybersecurity conformity assessment by applying AI-driven vulnerability analysis to validate security control adequacy. Unlike prior XAI-based dynamic risk assessments using Deep Q-Network (DQN) and LIME [5], which lacked comprehensive auditor support, this framework offers significant practical implications by addressing the fundamental challenge of scalability; the systematic processing of 5,963 vulnerability records demonstrates the capability to handle enterprise-scale certification scenarios while reducing labor requirements. Crucially, to ensure the XAI explanations are not merely plausible but mathematically faithful to the model’s behavior, the framework integrates Counterfactual Sensitivity Tests via Marginal Analysis. Unlike static heatmaps, this approach verifies the causal link between features and risk scores by simulating the counterfactual state (i.e., "What if the control was optimized?"), thereby providing auditors with actionable "recourse" to alter decision boundaries from "High Risk" to "Acceptable." Consequently, stakeholders including auditors, AI researchers, and C-ToE owners benefit from a robust, reproducible method for verifying compliance against evolving standards. The practical implementation of the work offers significant implications for cybersecurity certification. The framework addresses the fundamental challenge of scalability in traditional certification processes, where manual assessment of complex composite ICT products requires substantial time and specialized expertise. The systematic processing of 5,963 vulnerability records across five critical assets demonstrates the framework’s capability to handle enterprise-scale certification scenarios while maintaining detailed traceability at the individual asset level. The economic advantages extend beyond the low-cost GPT-3.5 fine-tuning to encompass reduced labor requirements and accelerated certification timelines. The framework’s ability to provide automated control adequacy validation and systematic XAI explanations reduces the time required for evidence collection and analysis, enabling certification bodies to process more certification requests with existing personnel. Stakeholders including auditors, AI researchers, certification bodies, policymakers, manufacturers, CToE owners, and users from any cybersecurity application domain can benefit from using this framework.

We exhibit potential threats and limitations that may affect the validity of the proposed approach and findings. The operationalisation of XAI practice using counterfactual based marginal feature analysis, SHAP , and LIME is one of the key contributions of this work for justification of security control and explaining audit evidence. However, SHAP often considers that the features are independent, while outcome from LIME may varies due to the random sampling. Moreover, counterfactual analysis often lacks diversity due to consideration of single possible option for the correlation among features. This certainly impact to create consistent and deterministic audit evidence. In future, we aim to correlate the outcome from the multiple different explainers to minimise the limitation of each technique. We have focused only vulnerability exploitation as dynamic security parameters for the risk assessment. However, there are other parameters such as assets dependencies to determine the criticality of the compromise dependent assets, and threat intelligence data, which also evolved. To address this, we plan to consider additional parameters for the dynamic risk assessment. The evaluation of our approach considers a single use case scenario with vulnerability related CVEjoin dataset and one of the co-authors from the pilot scenario involved for the evaluation. This may not generalise our findings due to the consideration of single use case and dataset context and lack of availability of auditor feedback. To address this, we plan to consider pilot cases from different application context, inclusion of different security dataset such as threat intelligence data and involvement of other relevant stakeholder for improving the applicability of our approach and generalisation of the findings.

The framework’s reliance on a third-party LLM API (OpenAI) introduces potential challenges regarding long-term reproducibility and future model availability. Furthermore, vulnerability ecosystems are inherently dynamic, with EPSS scores and exploitability metrics evolving over time; consequently, the current model represents a specific temporal snapshot, and operational deployment would necessitate periodic retraining to mitigate this drift. Finally, although the framework successfully generates audit-ready evidence, we have not yet conducted empirical user studies with professional auditors to quantitatively measure the resulting reduction in cognitive load.

To address potential biases in XAI explanations, we employed a multi-faceted validation approach. However, SHAP often considers that the features are independent, while the outcome from LIME may vary due to the random sampling. To mitigate these biases, we integrated Permutation Feature Importance (PFI) during the model training phase as a global baseline. This triangulation–comparing PFI’s global rankings with SHAP’s attribution and LIME’s local insights–helps identify and reduce algorithmic bias, ensuring that the generated ’Audit Evidence’ is robust and not merely an artifact of a single explanation method’s assumptions.