An FCM-based hybrid method for DDoS attack detection in resource-constrained devices

Firewalls, antivirus software, and intrusion detection systems have improved over the years in becoming increasingly accurate in detecting a wide range of cyber attacks. The traditional Information Technology (IT) infrastructure is constantly monitored and updated by system and network administrators. However, the Operational Technology (OT) infrastructure, which the IoT devices are part of, is more likely to lag behind in standard security practices of upgrading and patching their software. Therefore, the OT infrastructure is particularly vulnerable to sophisticated, modern-day attacks.

Many recent instances of high-profile cyber attacks demonstrate the security challenges for devices in highly sensitive environments or with modest processing capabilities. In particular, the Stuxnet attack [6], also dubbed as the world’s first cyberweapon, sabotaged Iran’s nuclear program by targeting the computer systems that controlled the centrifuges and gas values. An attack on Target^® Corporation’s HVAC systems in 2014 [7] illustrated the risks of remotely managed Heating, Ventilation and Air Conditioning (HVAC) systems. Hackers stole the login credentials of the company that managed the HVAC systems to penetrate Target’s payroll systems. At that time, Qualys^TM, the vulnerability management software company, noted that there were about 55,000 HVAC systems without adequate security. They were able to hack and reprogram consumer-grade WiFi cameras to create a permanent backdoor. The backdoor was then used by malicious actors to get command and control over other devices in the network and steal data. Carpentier et al. [8] demonstrate an attack against a highly secure network using a Bluetooth^TM smart-bulb. They execute a simple JavaScript within a web-based Bluetooth Application Programming Interface (API) using a web browser as an attack vector. Their attack runs in stealth mode and outsmarts all the security controls within the network to gain access to the smart-bulb.

Reconnaissance is an important first step for attackers in the discovery and selection of target systems. The discovered vulnerabilities of the target can be exploited in many ways. A denial-of-service attack is one such exploit wherein the attacker sends a large number of scan requests to overwhelm the bandwidth and processing resources of the target. Protection of systems against such attacks is challenging, as the service ports must be kept open to respond to client requests. For instance, since a web server must listen on TCP ports 80 and 443, requests to these ports cannot be blocked. Apart from the next-generation security apparatus, enterprises can utilize DDoS protection service providers that filter and clean traffic. However, a large number of IoT devices are not protected by traditional security infrastructure. Worse yet, many devices are left at their default factory settings and thus highly susceptible. Therefore, equipping such devices with a low-overhead host-based intrusion detection system is advisable.

The MITRE ATT&CK^® framework [9], which is similar to the cyber kill chain described in Section 1, is a knowledge base of adversarial tactics, techniques, and procedures. This framework provides a broad and comprehensive list of techniques used by adversaries and is based on real-world examples. Based on this framework, cyber-security professionals develop a structural approach to threat modeling and defense methodologies. This framework identifies reconnaissance as an important tactic and lists the ten techniques that adversaries utilize for reconnaissance. The first of the reconnaissance techniques is “active scanning.” A wide range of proprietary and open-source scanning tools is available to either gather information passively about a target host or to weaponize scanning as an active tool for a denial-of-service (DoS) attack. Some examples of easily accessible open-source tools that may be used for mounting attacks are: Kali Linux^®, Metasploit, Wireshark^TM, TCPdump, nmap, Burpsuite^TM, and Nessus^®. Modern computer systems are capable of scanning a wide swath of the Internet in minutes and gather significant information about the systems running across the globe. In addition, there are online search engines such as Shodan^® and Censys^® that scan the Internet and provide additional Internet intelligence for a fee. As a consequence, Internet-wide port scanning activity is ever-increasing.

A. Cui and S. J. Stolfo [10] present a quantitative analysis of insecure embedded network devices by conducting an Internet-wide scan (excluding some networks) in 2010. Their study reveals that about 540,000 embedded devices were publicly accessible and vulnerable to attack. Insight into Internet-wide scanning and vulnerability identification is provided by O’Hare et al. [11]. They propose a scanning tool called Scout that combines data from Censys and the National Vulnerability Database (NVD) of the National Institute of Standards and Technology (NIST) to passively identify potential vulnerabilities. Port scanning is performed by both institution-sanctioned penetration testers as well as nefarious adversaries, but their motives differ. The former aim to discover and patch the vulnerabilities, while the latter seek to discover and exploit these vulnerabilities.

Full size image

Information on open ports can be utilized to launch a denial-of-service (DoS) attack. In a DoS attack, the target machine is inundated with a large number of spurious requests so as to render it unable to respond to legitimate user requests. Servers accessible over the Internet are naturally vulnerable to external attacks since the service ports have to be publicly visible. An IoT device, such as a web camera running a web server for access, can be easily overwhelmed by such large requests, as it has limited processing, memory, and storage capacity. A far more common type of DoS attack is when an adversary mounts a massive scan operation from multiple bots, resulting in a distributed denial-of-service (DDoS) attack. A DDoS is difficult to detect and block as the attack has seemingly multi-point origins. Common service ports such as HTTP (80), DNS (53), and LDAP (389) are easy targets of DDoS attacks.

Sharafaldin et al. [12] provide a taxonomy of DDoS attacks on the basis of the type of attack (reflection or exploitation), the protocols targeted, and the types of packets that are utilized. The taxonomy is succinctly captured in Fig. 1 reproduced here from their paper. It is apparent from the figure that DDoS attacks are possible at the network, transport, as well as application layers of the TCP/IP reference model.

Exploitation-based DDoS attacks, such as SYN flood or UDP flood, are usually volumetric in nature. In a volumetric attack, the adversary sends SYN or UDP packets to the target through numerous intermediate bots so as to overwhelm the target with requests. A SYN flood [13], sometimes known as a “half-open” attack, is a network-tier attack that bombards a server with TCP connection requests without responding to the corresponding server-sent acknowledgments. The large numbers of these half-open TCP connections consume the server’s resources, thereby crowding out legitimate traffic. After multiple attempts, the victimized server sends an RST packet and terminates the session. However, by then, it may be too late to avoid service disruption. A UDP flood attack [14] is similar to a SYN flood attack, but performed with UDP packets. A large number of UDP packets are sent to random UDP ports of a target server in order to consume its resources disproportionately. The server, if not listening on that port, keeps responding with a “destination unreachable” Internet Control Message Protocol (ICMP) message.

In reflection-based attacks, the attacker hides behind multiple reflector servers to attack the target machine. This type of attack is effective using protocols such as DNS, LDAP, and NTP, as the servers constantly advertise their presence and are expected to be available and listening on the associated ports. Thus, an attacker can send a large number of DNS requests to public DNS servers with the source IP address set to the victim machine. The DNS servers respond to all these DNS queries, but the responses go to the victim machine and not the attacking host, thus overwhelming the victim machine.

Much research effort is directed towards using machine learning (ML) methodologies in Intrusion Detection Systems (IDS). Some publicly available datasets provide a wide variety of attack vectors, the most current being the CICIDS2017 [15] and the CICDDoS2019 [12] datasets, published by the Canadian Institute for Cybersecurity. These two datasets differ in some of the attacks that they capture. In addition to providing the datasets, the authors also demonstrate successful detection by first identifying the best features and then analyzing the predictions of selected ML algorithms. The Random Forest algorithm performs the best in their experiments. Wang et al. [16] propose a Multi-Layer Perceptron (MLP) based attack detection method that incorporates sequential feature selection and a feedback mechanism to dynamically reconstruct the detector based on detection errors. They experiment with the five best features and train and test three different datasets. Saheed et al. [17] conduct a performance analysis of an IDS solution based on supervised learning algorithms using the UNSW-NB15 dataset. They reduce features using the dimensionality-reducing Principal Component Analysis (PCA) algorithm and then use other classification algorithms to train and test; XGBoost performs the best. Casajús-Setién et al. [18] propose an anomaly-based Network IDS (NIDS) using the transformer model, a type of neural network which is a sequence transduction model that allows reasoning from a set of observed training sequences to a new set of test sequences using the WUSTL-IIoT-2021 dataset. Becerra et al. [19] present performance analysis of six ML algorithms using the CICDDoS2019 dataset. The Pearson correlation coefficient is used to reduce the number of features to 22 and to demonstrate that the Random Forest algorithm has the best accuracy.

Jemal et al. [20] present an effective Convolutional Neural Network model for detecting DoS and DDoS attacks, tested on the BoT-IoT dataset. Zhou et al. [21] propose a DDoS attack flow classification system based on a feature selection method that computes the best threshold value for each feature. Gniewkowski et al. [22] propose a novel detection method based on the features related to the temporal dependencies in the client’s behavior. Their so-called “IQRPACF” method not only identifies large values that stand out from the rest (Inter Quartile Range - IQR), but also utilizes a Partial AutoCorrelation Function (PACF) to find frequent and repetitive changes in the data. Mihoub et al. [23] propose a new method for the detection and mitigation of DDoS attack, based on the looking-back approach for machine learning. This method is able to detect the attack type precisely; the inputs for this model are the top ten input features of the selected ML or DL algorithms, along with the attack types seen until that time instant. The Random Forest algorithm, paired with looking-back steps, is the best performer with 99.81% accuracy. Alzubi et al. [24] present an Artificial Neural Network (ANN) based IDS combined with Salp Swarm Algorithm (SSA) for optimization. In fog and edge computing, the Effective Seeker Optimization combined with Machine Learning-Enabled Intrusion Detection System (ESOML-IDS) feature selection (FS) method in [25] chooses an optimal subset of features to identify intrusions. Alweshah et al. [26] propose a wrapper feature selection model using the emperor penguin colony optimization algorithm and K-nearest neighbor classifier for intrusion detection in IoT devices.

Prasad et al. [27] propose a custom feature selection method based on Pearson Correlation Coefficient (PCC) combined with Decision Tree (DT)classification for intrusion detection using the UNSW-NB’15 dataset. A Transformer architecture-based model is proposed by Fan et al. [28] to detect DDoS attacks using the CICDDoS2019 dataset. They use the pigeon hole optimization algorithm for feature selection and the Time Enhanced Transformer for Security (TETS) model for training and predictions. They test DNS, LDAP, NTP, and Portmap DDoS attacks against Convolutional Neural Network, Recurrent Neural Network, Long Short Term Memory+Transformer, Convolutional Neural Network+Transformer, along with their own model. A detailed survey of intrusion detection solutions for IoT devices is presented by Kharaisat et al. in [29]. They classify IDS for IoT based on placement, detection, and validation strategies. The detection methods utilize machine learning algorithms such as supervised learning, unsupervised learning, reinforcement learning ensemble, and deep learning models. They also discuss statistical methods like the hidden Markov model and detection techniques based on fuzzy logic.

Table 1 captures the salient points and the limitations of ML and DL algorithms used for intrusion detection. They generally suffer from one or more of the following limitations. (1) High computational complexity for training and/or testing with large datasets for effective predictions, (2) non-determinism in algorithms such as Random Forest and Extra Tree Classifier, due to the randomness built into their logic, (3) focus on just a few specific attack types, (4) limited baseline models for comparison, and (5) lack of causal understanding within these models leading to mistaken inferences (false positives and false negatives) [32]. More recent studies, such as [24‐26] and [28], also exhibit one or more of these limitations. In terms of the interpretability of these trained models, eXplainable Artificial Intelligence (XAI) methods like SHapley Additive exPlanations (SHAP) [30] and Local Interpretable Model-Agnostic Explanations (LIME) [31] do help interpret ML predictions. However, their processes are complex and add to the already high cost of computation. Furthermore, they are executed on the model after its training phase, and thus add an extra step. In addition to these limitations, the heterogeneity of resource-constrained devices presents an additional level of challenge in the implementation of trained models. These considerations are strong motivators for our work. Our proposed hybrid algorithm has low computational complexity, making it suitable for resource-constrained devices, is scalable, deterministic, and is effective for detecting a large number of DDoS attacks. Furthermore, our model is transparent about the calculations and impact of the various input features on the final classification. Our work, presented in this paper, thus deviates from the normal approach of using ML algorithms as black-box detectors.

In our previous work [33], we ran experiments to determine the causal relationship between traffic characteristics of incoming network packets that would lead to their classification into benign or attack packets. We used statistical analysis with a focus on key features such as packet count, packet size, and CPU utilization to detect reconnaissance attacks. It relies on experimental hand-selection of a few aforementioned features, as well as the manual selection of detection thresholds to determine where the traffic is malicious or benign. In [34], we introduced a greatly improved method that considers a vastly large set of influencing features and employs a novel hybrid technique for the detection of SYN and UDP flood attacks. It is both effective and computationally efficient compared to pure ML algorithms. This manuscript is an extended version of our paper presented at CSNet 2024 [34]. Here, we extend our hybrid approach to detect all types of DDoS attacks, not just the SYN flood and UDP flood attacks. Furthermore, we automate the computation of the threshold value that serves as the decision boundary between attack and benign packets. We use ML algorithms to simply compute the weights of the input features against the output feature. We then use FCM to train and test the models and demonstrate that this method is relatively fast and reliable for a few specific feature selection algorithms for most types of DDoS attacks. Our model is meant to be executed directly on edge devices.

Fuzzy cognitive maps (FCMs) were originally proposed by Kosko [35] as an evolution of R. Axelrod’s work on Cognitive Maps [36]. An FCM is a fuzzy digraph that represents hazy causal relationships (uncertainty) between causal objects in the graph called “concepts” or “features.” The features are represented by the vertices in the digraph, and the directed edges connecting features are called “weights.” An FCM is constructed using these features and weights. Figure 2 shows a complex map of concepts interconnected by edges with weights. Concept \(C_4\) is the output feature, concepts \(C_1\) and \(C_3\) have positive impact on \(C_4\) with weights \(W_{14}\) and \(W_{34}\) respectively. Concept \(C_2\) and \(C_5\) have negative influence on \(C_4\) with weights \(W_{42}\) and \(W_{45}\). The impacts are reflected in the weights between any two concepts. Some input concepts also have an influence on other input features, thus creating a complex web of inter-dependencies. In a traditional FCM, initial values are assigned to both features (vertices) and weights (edges) of the digraph by experts in the corresponding field. Features are the influencing factors that impact the system’s outcome. The extent of the impact is determined by the values of features and weights and by the iterative algorithm that updates these values on each pass. Thus, an FCM may be used as an analysis tool for inferring causal relationships between influencing factors (features) and outcomes of interest.

Full size image

FCMs may be analyzed using one of two approaches: causal or dynamic [37]. The causal approach represents our certainty about one concept’s influence over another. So, the value on each edge is either 0 or 1. However, the dynamic approach represents the magnitude of influence of one concept over another. Thus, the value on an edge can be any real value, usually normalized between \(-1\) and \(+1\). FCM concepts can be identified as input and output features. Single-layered FCMs measure only the influence of multiple input features on a single output feature. Multi-layered FCMs measure the influence of each feature on all other features.

For instance, in Fig. 2, if the edge weight \(W_{13}\) is positive, then concept \(C_1\) has a positive (reinforcing) impact on concept \(C_3\). Thus, if \(C_1\) increases, so does \(C_3\). On the other hand, if \(W_{13}\) is negative, then \(C_1\) has a negative impact on \(C_3\), thereby attenuating \(C_3\)’s value. \(C_3\) in turn, can either reinforce, attenuate or have no effect on the output feature \(C_2\) depending on whether the edge weight \(W_{32}\) is positive, negative or zero. To make matters more interesting, note that not only does \(C_1\) indirectly influence \(C_2\) via \(C_3\) in the manner described above, but it also directly influences \(C_2\) via the edge weight \(W_{12}\). In this manner, in a multi-layered FCM, complex interdependence, i.e., correlations among concepts (features) can be modeled.

FCMs, when first implemented, were used to compute the mathematical value of weights based on the linguistic value of weights assigned by experts [38]. Many experts can identify the features that they consider to be important and assign weights to the outgoing directed edges from those features. Each expert’s input creates a distinct FCM. FCMs from various experts can be algorithmically combined to compute a single resultant FCM. FCMs are used mainly for time series forecasting to compute the future state of a system or to find values of features for a stabilized state of the system. FCMs are also used for classification problems. An FCM is somewhat similar to a neural network since both have input and output features with link weights. However, a neural network has a hidden layer between its input and output layers, whereas an FCM is fully transparent.

For the example FCM shown in Fig. 2, the features are labeled as \(C_j (j=1,2,..., N)\), and the weight values \(w_{ji}\) are as shown on the directed edges from \(C_j\) to \(C_i\). In our dynamic FCM, \(C_j =[0, 1]\), and \(w_{ji} =[-1, 1]\). In Fig. 2, the cumulative impact of input features \(C_1\), \(C_3\), \(C_4\) on output features \(C_2\), \(C_5\) (in red), is computed iteratively by the following rule [35, 39].where \(A_i^{t+1}\) is feature \(C_i\)’s value at time \(t+1\), \(A_i^{t}\) is \(C_i\)’s value at time t, \(w_{ji}\) is the weight (strength) of the fuzzy influence of \(C_j\) over \(C_i\), and f is the squashing function that transforms the computed value of \(C_i\) into our required interval [0, 1].

FCM feature and weight value computations, activation functions, and learning methods may vary so as to provide better predictions for future feature values. For instance, Stylios et al. [40] propose modifications to Eq. 1 that include adding \(A_j^t\) to the summation for time series predictions. Bueno et al. [41] provide a comparison of the four types of activation functions used in FCM—the sigmoid function, the hyperbolic tangent function, the step function, and the threshold linear function. The sigmoid function offers significantly greater advantages over the other functions as it allows decision support by defining middle intensities and not the extreme (0 or 1) cases. Papageorgiou et al. [42, 43] explore active Hebbian and nonlinear Hebbian-type learning algorithms to train FCMs. Detailed surveys on FCM are provided by Jetter et al. [44], Papageorgiou et al. [45], and Felix et al. [46] in which they discuss FCM concepts, advancement in FCM methodologies, and FCM applications. Tools and software for utilizing FCMs are also discussed.

FCMs are mainly used for time series forecasting, to compute the future state of a system or to find values of features for a stabilized state of the system. In addition, FCMs are also used for classification problems in various domains. Nápoles et al. [47] present a detailed review of the use of FCMs for classification problems. Salmeron et al. [48] use particle swarm optimization for FCM learning to classify a patient’s rheumatoid arthritis profile. In [49], Froelich proposes a new algorithm for generating discrimination thresholds. Nápoles et al. [50] combine rough set theory and FCM for pattern classification by grouping input neurons as three-way decision rules—positive, negative, and boundary. In [51], Mls et. al propose a new algorithm for optimization of the connection matrix (weights matrix) that would address the incompleteness and uncertainty of experts’ opinions. The dynamics of the state vectors of several candidate FCMs are simultaneously presented to the human expert on the Graphical User Interface (GUI) of the related application. Doing so aids the expert in visualizing and evaluating the quality of the whole population at once. Without the need for an analytically expressed fitness function, the expert uses the interface to prioritize the best FCM candidates in every population. Wozniak et al. [52] propose a new method Real-Coded Genetic Algorithm (RCGA), that is based on Genetic Algorithms (GAs), to generate a virtual population. This methodology does not require human intervention and provides high-quality models that are generated from historical data.

In the cyber-security domain, the contextual information in FCM has been used for decision support in Intrusion Detection Systems (IDS). Nápoles et al. [53] propose a supervised learning algorithm using Rough Set Theory and FCM. Concepts are grouped as positive features (weight is \(+1\)), negative features (weight is \(-1\)), and boundary features that include all the inconsistent features (weight is 0.5), based on a threshold value. Aparicio-Navarro et al. [54‐57] use FCMs along with the Dempster-Shafer theory (for data fusion) and basic probability assignment values to include contextual information at different stages of the intrusion detection process. They claim that their approach provides effective solutions by reducing false predictions.

In these papers, the FCM construction process is not codified, and the grouping of features and assignment of weights requires user intervention, and perhaps human judgment. Our work presented here is different in the following way. We have a clearly defined process to construct and utilize FCMs. We first use a feature selection algorithm to extract the weights for all input features, then utilize this information to construct the weighted FCM graph; this step provides a clear representation of the importance of each feature as explained in Section 4.2. The feature importance scores are then provided to the FCM Eq. 1 to train and test the model. An automated threshold computation process provides the final output classification. This results in a clear, simple, light-weight, and transparent approach to intrusion detection in resource-constrained devices.

We use the CICDDoS2019 dataset [12] for our study. The dataset contains data on several types of DDoS and reflective DoS attacks. It includes raw packet captures (pcap) and comma-separated value (csv) files for each type of attack, along with some benign traffic. The dataset is generated over a test-bed that consists of an attack network and a victim network. The victim network consists of all commonly used equipment. A benign profile agent generates realistic background traffic, and the attacks are executed using third-party tools. The raw pcap files are passed through an open-source tool called the “CICFlowMeter” [58] that generates Biflows from pcap files and extracts about 88 features from the packet captures.

In our study, we focus our computations and analysis on both the volumetric and reflective DDoS attack data available, namely SYN flood, UDP flood, DNS, LDAP, MSSQL, NTP, NetBIOS, SNMP, SSDP, Portmap, TFTP, and UDPLag. We do not consider WebDDoS attack packets as the sample is very small. CICDDoS2019 captures packet data on two different days: the first on January 12^th, designated as the training day, and the second on March 11^th, designated as the testing day. In this study, we refer to the January 12^th csv files as <attack>-Jan and the March 11^th csv file as <attack>-Mar. The authors of this dataset sort the flows by attack type. However, some csv files have more than one attack packet mixed in. In order to train and test each attack, we separate those and group the records for each type of attack. We note that the benign packets are low in number and cause a significant imbalance in the dataset. In order to address this imbalance, we combine benign packets from all the January records and March records separately and include them in all the attack record files. Our resulting dataset contains one csv file per day for each attack (training, testing) with the same number of benign packets per day.

We first cleanse the datasets by removing those features that (1) are duplicates, or (2) have all zero values, or (3) do not contribute to decision making or are correlated. We therefore reduce the datasets to 27 features (down from 88). For instance, we remove the “Forward Packet Length Max” and “Forward Packet Length Min” features as they have the same type of influence on the output feature as the feature “Total Length of Fwd Packets” and are thus marked as redundant. We drop features like “PSH Flag count,” “Fwd Avg Packets Bulk” that have all zero values and features like “Source port,” “Flow ID,” “Timestamp” that have no impact on the output, i.e., final classification. We do not use any data balancing techniques in our study; we demonstrate later in Section 5 that our hybrid model performs well with the preprocessed data despite being imbalanced.

The target column “Label” contains the two categories of the classification of packets—attack (SYN, UDP, LDAP, DNS, etc.) and benign (normal) on the corresponding datasets. These two categories are assigned numerical values, 0 for attack and 1 for benign, for computational purposes. The features have a wide range of values between them. For example, the “Flag” features have binary values—either 0 of 1, but features like “Flow Duration” or “Flow Inter Arrival Time” have values that could be as low as zero or as high as millions, or even \(\infty \). To make fair comparisons of features, we normalize the data so that all feature values fall in the interval [0, 1], using the normalization formula in Eq. 2.where

\(FN_i^j\) is the j^th feature’s normalized value at the \(FA_i^j\) is the j^th feature’s actual value at the i^th flow,

\(F_{max}^j\) is the j^th feature’s maximum value,

\(F_{min}^j\) is the j^th feature’s minimum value.

Some attack packets are available only on either the training day or the testing day. For such attacks, we split the dataset into training (80%) and testing datasets (20%). For attacks that have records in both January and March files, we use the January files in full for training and reduce the March files to about 20% of the training records so as to conform to the standard practice of 80–20 ratio for training and testing. Table 2 shows the summary of the datasets used for training and testing, and the packet counts for each category in the dataset after cleansing operations.

After the data is cleansed and normalized, we use the following feature selection algorithms in ML to measure the influence of the 26 input features on the output feature. Thus, feature selection algorithms are not used for actual feature selection, but to compute the weights between the input features and the output feature for our FCM models. Here, we evaluate the impact of input features on the output feature using a single-layer FCM model. The ML algorithm descriptions are referenced from SciKit Learn [59], an open-source ML library for Python, and from the Machine Learning Mastery website [60]. In our initial quest to find a simple solution, we first chose a few well-known statistical feature selection methods, such as SelectKBest-classification (SKB) and SelectKBest-ChiSquared (SKB-C2), and found the results to be promising. We then expanded the search to ML-based feature selection methods, selecting a few tree-based and boosting ML algorithms such as Random Forest (RF) and eXtreme Gradient Boosting (XGB), that are known to perform well in supervised learning classification problems. We also ran the Gradient Boosting Classifier (GBC) and Mutual Information (MI) algorithms and have presented those results in our earlier work in [34]. However, since those algorithms were two orders of magnitude slower than the best-performing algorithms with slightly inferior accuracy, we have dropped them from our analysis here. In addition, we tested the Recursive Feature Elimination (RFE) and the RFE Decision Tree Classifier (RFE-DTC) algorithms. These algorithms assign the rank 1 to the top ten features, and rank the remaining features as 2, 3, and so on. However, they do not output the selection scores (weights). For this reason, we were unable to utilize them in our hybrid approach.

Full size image

The score for each input feature represents the measure of that feature’s influence on the output feature. The raw feature scores reported by these algorithms for the LDAP attack dataset are shown in Table 3. SKB-C, SKB-C2 and CatB report large-magnitude scores that we convert to the range [0, 1]. For each algorithm, the scores for features are unique with very few exceptions, indicating that our data preprocessing steps effectively removes redundant features. The features and weights for the DNS Attack FCM obtained from the feature selection algorithm of AdaB is plotted in Fig. 3. However, none of these algorithms indicate whether the score correlations are positive or negative. Only in this specific step (more fully explained in method 2, Section 4.3.2) do we use our domain knowledge to add appropriate \(+\) or − signs to the weights. We thus ensure that the scores are appropriately transformed into FCM edge values (i.e., weights \(w_{ji}\) in Eq. 1), falling within our desired range of \([-1, 1]\). The initial feature values \(A_j^t\) are extracted from the dataset, its the value of each feature for each record in the dataset. For example, the feature “Total length of Fwd Packets” may have a value of 60 bytes for a certain packet, and the “Flow Bytes/s” may have a value of 292 bytes. All the feature values in the dataset are first normalized using Eq. 2 so they are comparable on the same scale. These values are then used along with the weights in computing the value of “Label,” the target column using the FCM formula in Eq. 1. Here, remind the reader that the feature selection process is used mainly to obtain the weights between input features and the output feature to fit our FCM model. In Section 4.3, we use our pre-processed dataset with 26 input features as the starting point for our experiments with FCM models.

Next, we create an FCM model for each feature selection algorithm with its ten selected inputs and one output (label), using the tool Mental Modeler [64]. Figure 4 shows one example FCM model associated with the SKB-C feature selection algorithm for an LDAP attack. The transformed scores from the feature selection algorithms are utilized as weights in the FCM model. The blue arrows indicate positive influence, and the orange arrows indicate negative influence. The thickness of the line indicates the intensity of the influence of the input feature on the output feature. A feature with a positive influence implies that an increase in its value increases the chance of that packet being a benign (BN) packet. A feature with a negative influence implies that an increase in its value increases the chance of that packet being a SYN attack packet. The final classification of a packet is based on the cumulative effect of all input features on the output feature.

Consider, for instance, in Fig. 4, the two input features “Average Packet Size” (negative influence) and “Down Up Ratio” (positive influence) in the FCM model for SKB-C algorithm for LDAP attack dataset. A large packet size increases the possibility that the packet is classified as an LDAP attack packet on account of its negative influence on the output feature. On the other hand, a small “Down Up Ratio” increases the possibility that the packet is classified as an attack packet due to its negative influence. This action is consistent with real-world situations in which large-sized packets are likely to be LDAP attack packets, as they are the responses from an LDAP server in an LDAP amplification or reflective DDoS attack. Similarly, a small “Down Up Ratio” is reflective of the congestion in the network due to a DDoS attack.

Full size image

We define a predictor’s performance in classifying a packet as “benign” or “attack” by measuring its accuracy, precision, recall, F1 score, and confusion matrix.

We also define the following terms for our performance metrics (see Eqs. 3–6), consistent with the definitions in Pedregosa et al. [59]. Accuracy measures how close the measured values are to the actual value. Precision measures true positive predictions among the total positive predictions made by the model, the correct prediction of a certain class out of all the predictions of that class. It tells us how many of our predictions are of positive or negative classes were correct out of the total predictions in that class. Recall is the true positive rate or sensitivity, a ratio of true positives to the total number of elements that belong to the positive class, the correct predictions of a certain class out of the all the actual values of that class. It tells us how many of the positive or negative classes we were able to predict correctly. The F measure, also known as F1 score, is the harmonic mean of precision and recall values.The “confusion matrix” represents the four possible outcomes in a \(2\times 2\) matrix, in a binary classification: the true positives, the true negatives, the false positives, and the false negatives. Figure 5 shows the confusion matrix with the associated labels and formulas. The rows represent the true labels for positive and negative classes, and the columns represent the predicted labels for the positive and negative classes. In our case, the positive class is the “attack” label and the negative class is the “benign” label. The figure also shows the formulas for precision, recall, and accuracy as they pertain to the confusion matrix. The computation of the confusion matrix is explained with an example in Section 5.

Even after rectifying somewhat the imbalance in the CICDDoS2019 dataset as described in Section 4.1 by adding benign records from all csv files to all the attack csv files, we note the following. With such imbalanced data, it is best to evaluate the performance of the FCM predictions by examining not just the accuracy, but also the precision, recall, F1-score, and confusion matrix in totality. Normally in ML methods, a single dataset is split into “train” and “test” sets, and the model’s performance and accuracy are measured. However, we train the model using one dataset (for example, syn-Jan) and test using another dataset (for example, syn-Mar) to demonstrate that the learning process from one dataset is reliable and fast for detection on another dataset. We experiment with and compare the results of two different methods for the detection of all types of attacks included in the dataset: (1) the traditional pure ML-only approach described in Section 4.3.1 and (2) our hybrid ML-FCM approach described in Section 4.3.2.

Full size image

Full size image