A novel feature-based framework enabling multi-type DDoS attacks detection

One detection solution is the traditional anomaly detection. These methods extract a certain feature from the traffic and compare the value with that of normal traffic, and a DDoS attack is detected if the deviation exceeds a threshold. The authors in [47] proposed a generalized entropy method, which measures the probability distribution differences between the attack and legitimate flows, to detect the Low Rate DDoS attack. In [50], an adaptive correlation analysis method was proposed, which analyzes the correlation between the aggregated flows on the victim and the suspicious flows, to decide whether the Pulsing attack is ongoing. In [28], the authors proposed a normalized entropy method, which measures the abnormal changes of randomness concerning IP destination flows, to detect the SYN flooding attack. The authors in [51] proposed an expectation of packet size method, which measures the difference on the variance of packet size between the attack and normal traffic, to detect the Low Rate DDoS attack. An entropy variations method was proposed in [49], which measures the entropy variation between the normal and attack traffic, to detect and traceback DDoS attacks. A joint entropy-based detection and mitigation scheme was proposed in the Software-defined networking (SDN) [25]. An attack is detected when both bandwidth and joint entropy of the attack traffic exceed the thresholds. Wang et al. proposed a three-layers sketch-based structure to detect and mitigate DDoS attacks [46]. The detection theory is based on the divergence of probability distance for the incoming requests, because the attackers are usually persistently launching malicious requests. However, due to only a few features being considered, these methods are only effective on specific attacks and can hardly detect different types of attacks. Moreover, the detection performance is also limited when an attacker mixes various types of attacks. Recently, machine learning (ML) methods offer a new route for many classification and regression tasks. Apart from the usage in the prevention of cyber attacks on energy internet systems [31], predicting traffic flow for electric vehicles charging service [30], and network representation learning [17, 33, 48], they have also been applied to classify normal and attack traffic based on a set of features and thus decide whether a DDoS attack is ongoing [23, 43]. Zhu et al. proposed a privacy-preserving cross-domain detection for SDN [53]. They apply the KNN algorithm to detect a certain type of attack. The authors in [42] proposed a genetic algorithm combined with the KNN classifier to detect known and unknown DDoS attacks. Asmir et al. proposed a framework with feature engineering and machine learning-based detection [10]. Seven features are selected and tested on five methods, but most of the features (i.e., from_router, from_switch, to_router, and to_switch) could only be obtained in the simulation environment. Alsirhani et al. proposed a DDoS attack detection system that dynamically selects classification algorithms to detect different DDoS patterns [13]. Ali et al. [12] proposed a method based on multilevel autoencoder-based feature learning. Jia et al. proposed a Convolutional Neural Network (CNN) model to detect DDoS attacks [24]. However, their performance heavily relies on the extracted features that present the key patterns of different types of attacks. The selected features, such as ratio of ICMP, number of SYN, number of ACK, number of HTTP, variance of packet size, inbound to outbound traffic ratio, entropy of IP source flow, and entropy of packet type, are ineffective to detect multiple types of attacks. Therefore, the existing methods cannot determine the most informative features for various types of DDoS attacks detection.

To accurately detect SYN flooding, DNS amplification, Low Rate, Pulsing, Spoofing, and mixed DDoS attacks, in this paper, we comprehensively analyzed the characteristics of various types of DDoS attacks and proposed five new features from heterogeneous packets including entropy rate of IP source flow, entropy rate of flow, entropy of packet size, entropy rate of packet size, and number of ICMP destination unreachable packet. The average detection accuracy of the proposed features on specific types of attacks and mixed attacks are 0.99. Compared to the existing methods, the detection accuracy improvements are between 21% and 53%.

Our main contributions are summarized as follows:The rest of the paper is organized as follows. Section 2 presents the details of the proposed framework. The feature performance analysis is demonstrated in Section 3. Section 4 evaluates the performance of the proposed framework using real data and compares it with the state-of-the-art methods. Finally, Section 5 concludes the paper.

Since all features need to be extracted from samples, prior to the proposed framework, it is necessary for us to measure over sliding windows of time and split the raw dataset into file fragments to collect samples. For a given raw pcap file L with time length T, the raw file is split as \(L = \{l_i | i \in (1,N)\}\), where \(l_i\) denotes the ith file fragment in time interval \([T_i, T_{i+\triangle t}]\) according to window time \(\triangle t\). Then we can have the number of file fragments N as follows:

In this subsection, we analyze the characteristics of different DDoS attacks, introduce the proposed five new features including entropy rate of IP source flow, entropy rate of flow, entropy of packet size, entropy rate of packet size, and number of ICMP destination unreachable packet, demonstrate their improvements over the existing features, and explain their effectiveness against all five DDoS attacks.

The extracted features are preprocessed using the Min-Max normalization and Pearson correlation coefficient, respectively. Note that the packet-based and flow-based features are extracted separately. Then by following [23, 43], and [37], they are further combined to construct the feature matrix in the classification step.

After all features are extracted and selected, the classification process is conducted to classify normal and attack samples. To test the effectiveness of the features, six classification methods are applied, which are Decision Tree [41], Deep Learning [29], K Nearest Neighbor [19], Logistic Regression [27], Random Forest [16], and Support Vector Machine [44]. For a fair comparison, we use the default parameters for all algorithms in sklearn [9].

For each classification method, the classification process is performed on a victim who suffers specific types of attacks and mixed type attacks. In specific types of attacks classification, a victim suffers one type of attack at a time and we conduct binary classification between the normal and attack traffic. In the mixed type attacks classification, a victim is attacked by the mixed attacks. We use binary classification to detect whether the victim has been attacked and multi-class classification to identify the types of attacks.

Because various packet types are utilized by botnets, such as HTTP, ICMP, SYN, and DNS, which target multiple layers of the victim [45], the extracted 22 common features range from the data link layer to the application layer to monitor abnormal behaviors in those layers. Meanwhile, these features are privacy-preserving since they are not related to the content, which protects the application data. These features can be categorized into packet-based and flow-based features and extracted separately.

We use a 64-bit Windows 10 system with an Intel(R) Cores(TM) i7-8650U CPU @1.90 GHz and 16 GB RAM to deploy our framework. The datasets we used for experiments are the real public datasets shown in Section 3.2.1. The algorithms to parse the packet header, extract features from the datasets and conduct ML methods for classification are implemented in Python 3.7. In the following experiments, all raw files are split into file fragments using a two-seconds time window, which is a common choice for intrusion detection [38].

Since imbalanced data is the inherent characteristic in real DDoS attacks detection, in this paper we use the F1-score, which is a widely used metric for imbalanced data [14], to evaluate the classification performance. Although the overall accuracy is another common evaluation metric, it is not suitable for imbalanced distribution [36]. Therefore, in this paper we only used F1-score to measure the detection performance for all of the methods. The F1-score can be calculated as:where C is the total number of predicted classes, Precision\(_c\) is the Precision value of the cth class, Recall\(_c\) is the Recall value of the cth class, \(c = 1,2,\ldots ,C\). TP is the true positive which means that the positive point is detected as positive, FP is the false positive which means that the negative point is detected as positive, and FN is the false negative which means that the positive point is detected as negative. From (11) and (12), we can clearly see that the F1-score is an unweighted average over the classification result of each class, meaning that it is ideal for imbalanced data. The value of F1-score ranges from 0 to 1 and a higher F1-score value indicates a better result.

Due to the abrupt increasing of DDoS attack data, the selection of the splitting ratio for the training and testing set may cause overfitting or underfitting. Attackers can change the strength and peak time according to the attack ability and defense ability of the target. Therefore, using the fixed training and testing set may produce a high variance, while splitting the main attack features in the training set may cause overfitting. In contrast, the biased data that occur in the testing set may cause underfitting. A commonly used approach to assess the quality and stability of the data and classifiers is the K-fold cross-validation (CV). In K-fold CV, the samples are divided into K equal subsets and trained for k times. For each iteration, one subset is used for testing, \(K-1\) groups are applied for training, and the mean value is used for overall evaluation. In this paper, all classification methods are applied with 10-fold CV and the F1 score is the averaged value of the 10-fold CV.

To evaluate the performance of our proposed features, we design the experiments for two strategies that attackers may take to launch the five attacks: (i) launching a specific type of attack and (ii) launching mixed types of attacks. As mentioned in Section 2.4, the classifications are conducted using Decision Tree (DT), Deep Learning (DL), K Nearest Neighbor (KNN), Logistic Regression (LR), Random Forest (RF), and Support Vector Machine (SVM) classifiers.

In this section, we select Random Forest as the classifier and compare our framework using the proposed five features with the following eight state-of-the-art algorithms. The window time \(\bigtriangleup t\) for all methods are set to 2 for a fair comparison. The detailed descriptions and settings of these methods are listed as follows.Entropy-based methods, i.e., GE, Entropy, and SAFETY, identify a file fragment as an attack when its entropy value is smaller than that of the normal benchmark or a threshold based on the assumption that the number of flows is stable. However, this classification criterion should be opposite on some datasets when the number of flows increases [52], e.g., in the LLS-Spoofing detection pair task. Thus, we manually reverse their classification labels if the mean entropy of attack samples is greater than that of normal samples. Note that the entropy-based methods do not provide the entropy thresholds in their parameter settings, and thus we have to search for the optimal thresholds by ourselves. For a fair comparison, we search for the best threshold for each entropy-based method on each dataset and report the highest F1-score as the best performance for each method. The search range is in 100 entropy values between minimum and maximum entropy values. For our proposed framework, the five new features, which are entropy of packet size, entropy rate of packet size, entropy rate of IP source flow, entropy rate of flow, and number of ICMP destination unreachable packet, are selected for all datasets. We conduct comparison experiments in two scenarios: a victim suffers a specific attack and five mixed attacks.