Machine-synaesthetic approach to detecting network DDoS attacks. Part 1
Hello. Recently, we have a large number of new launches and today I would like to announce the course “Network engineer” , which starts in early March. Traditionally, we begin to share with you useful material on the topic, and today we are publishing the first part of the article “Machine-synesthetic approach to detecting network DDoS attacks”. Go!
Annotation . According to the authors, anomaly detection systems or ADS (anomaly detection systems) seem to be the most promising direction in the detection of attacks, since these systems can detect, among other things, unknown attacks (zero-day attacks). To identify anomalies, the authors propose to use machine synesthesia. In this case, machine synesthesia is understood as an interface that allows the use of image classification algorithms in the task of detecting network anomalies, allowing the use of non-specialized image detection methods that have recently been widely and actively developed. The proposed approach is that network traffic data is projected into an image. From the results of the experiments it is clear that the proposed method for detecting anomalies shows high results in detecting attacks. On a large sample, the value of the integrated efficiency indicator reaches 97%.
One of the methods for ensuring network availability is the use of network anomaly detection mechanisms. Before determining the anomaly, it is necessary to find out what is considered a normal condition. We consider the state of the system as “normal” (or “functionally viable”) when it performs all the functions assigned to it.
Consequently, an anomaly is a state in which the behavior of the system does not correspond to the clearly established characteristics of normal behavior [1]. The introduction of mechanisms for the rapid detection of such anomalies will significantly increase the chances of an effective response to incidents of a network availability violation.
Known network anomalies are so diverse that they cannot be categorized using the same classification. However, there is a clear distinction between active and passive, external and internal, intentional and unintended anomalies, etc. Since these differences do not reflect all the characteristics of the phenomenon under study, the author [2] proposed a classification of anomalies based on the object of influence, that is, the information system, consisting of hardware, software and network infrastructure.
In accordance with the chosen approach, network anomalies can be divided into two main groups: node failures and security breaches. Site failures include hardware failures, design and configuration errors, software errors, and equipment performance problems. Security breaches of the network include the following anomalies: network scanning, denial of service, malware, distribution of network worms, exploitation of vulnerabilities, traffic analyzers (sniffer) and network modifiers (packet injection, header substitution, etc.).
The greatest financial damage to telecom operators is caused by denial of service (DoS) incidents. DoS attacks, in turn, can be divided into two types: unintentionally caused “attacks” (design errors and network settings, a small amount of allocated computing resources, a sharp increase in the number of calls to a network resource), and deliberate attacks, such as UDP flood, TCP -SYN flood, Smurf ICMP broadcast flood and ICMP flood. Deliberate attacks pose the greatest threat, as they are harder to mitigate more effectively, and potentially they can lead to large losses.
Analysis of the results of studies published in [3,4,5,6,7,8], as well as reports of the main developers of information security systems, showed that there is no single effective algorithm for detecting and preventing denial of service attacks. Vendors usually offer an expensive solution that implements a hybrid algorithm based on signature search methods and blacklisting the IP addresses of the attacker's node as a form of risk reduction. An example is the ATLAS system from Arbor, Ltd. Thus, the problem of developing tools for detecting distributed DoS attacks with a high degree of efficiency remains relevant.
According to the authors, anomaly detection systems or ADS appear to be the most promising direction in the detection of attacks, since these systems can detect, among other things, unknown attacks (zero-day attacks). Almost all the anomaly detection models described in the literature can be divided into five groups:
To identify anomalies, the authors propose to use machine synesthesia. In this case, machine synesthesia is understood as an interface that allows the use of image classification algorithms in the task of detecting network anomalies, allowing the use of non-specialized image detection methods that have recently been widely and actively developed [23]. The proposed approach is that network traffic data is “projected” into the image. The accumulation of image changes gives us a video stream, analyzing which, we can conclude about the abnormal state of the observed data transmission network.
The basis of any anomaly detection system is the module that analyzes network packets and determines their potential harmfulness. In fact, ADS tries to classify network traffic into two subsets: normal traffic and network attacks (it does not matter which detection technology is used: based on signatures or statistical). Consequently, the very concept of ADS fits very well with the goals of image classification algorithms — matching the original image with a class of images from a set in accordance with certain characteristics. In addition, image classification as a mathematical tool for analyzing network traffic data and detecting network attacks has several advantages over the anomaly detection methods discussed earlier. These benefits are presented below.
3.1. Representation of multidimensional TCP / IP traffic data as an image
The authors propose to solve the problem of presenting network traffic metadata in such a way that it will allow using the pattern recognition algorithm to detect anomalies in the video stream.
Consider a network terminal device that collects traffic in a virtual channel. Each assembled package has a set of metadata, represented as a vector p:
where n is the dimension of the vector, P is the set of all vectors, id is the session identifier, date is the timestamp of registration by the terminal, x1, ..., xn is the direction, addresses and ports of the sender and receiver, packet size, protocol type, timestamp (as in TCP segment header), various flags and service fields.
To project the traffic into an image, the orthogonal projection method [24] is used: each vector p is represented by a point in multidimensional space, where n is the dimension of space, then all points (packets) belonging to one session are projected into two-dimensional space:
where a, b are empirically obtained basis vectors for projection into two-dimensional space, X is the original vector constructed from p by removing the id and date elements, X0 is the result of the projection, × is the vector product, () is the scalar product.
The next step in the visualization of a network session is the connection of all its points forming a convex shape. The last step is to fill the resulting shape with color. Then everything is repeated for the next network session. The resulting image is obtained when the imaging process was performed for all network sessions intercepted by the terminal. The accumulation of changes or the differentiation of this image gives us a video stream. In fig. Figure 1 shows examples of images that reflect the correct ("normal state") behavior of the network.
The end of the first part.
How do you like the material? Put pluses, write comments and register for an open lesson , which our instructor, a Cisco Academy instructor in the direction of CCNA Security - Alexey Kulinichev , will conduct already on February 18th.
Annotation . According to the authors, anomaly detection systems or ADS (anomaly detection systems) seem to be the most promising direction in the detection of attacks, since these systems can detect, among other things, unknown attacks (zero-day attacks). To identify anomalies, the authors propose to use machine synesthesia. In this case, machine synesthesia is understood as an interface that allows the use of image classification algorithms in the task of detecting network anomalies, allowing the use of non-specialized image detection methods that have recently been widely and actively developed. The proposed approach is that network traffic data is projected into an image. From the results of the experiments it is clear that the proposed method for detecting anomalies shows high results in detecting attacks. On a large sample, the value of the integrated efficiency indicator reaches 97%.
1. Introduction
One of the methods for ensuring network availability is the use of network anomaly detection mechanisms. Before determining the anomaly, it is necessary to find out what is considered a normal condition. We consider the state of the system as “normal” (or “functionally viable”) when it performs all the functions assigned to it.
Consequently, an anomaly is a state in which the behavior of the system does not correspond to the clearly established characteristics of normal behavior [1]. The introduction of mechanisms for the rapid detection of such anomalies will significantly increase the chances of an effective response to incidents of a network availability violation.
Known network anomalies are so diverse that they cannot be categorized using the same classification. However, there is a clear distinction between active and passive, external and internal, intentional and unintended anomalies, etc. Since these differences do not reflect all the characteristics of the phenomenon under study, the author [2] proposed a classification of anomalies based on the object of influence, that is, the information system, consisting of hardware, software and network infrastructure.
In accordance with the chosen approach, network anomalies can be divided into two main groups: node failures and security breaches. Site failures include hardware failures, design and configuration errors, software errors, and equipment performance problems. Security breaches of the network include the following anomalies: network scanning, denial of service, malware, distribution of network worms, exploitation of vulnerabilities, traffic analyzers (sniffer) and network modifiers (packet injection, header substitution, etc.).
The greatest financial damage to telecom operators is caused by denial of service (DoS) incidents. DoS attacks, in turn, can be divided into two types: unintentionally caused “attacks” (design errors and network settings, a small amount of allocated computing resources, a sharp increase in the number of calls to a network resource), and deliberate attacks, such as UDP flood, TCP -SYN flood, Smurf ICMP broadcast flood and ICMP flood. Deliberate attacks pose the greatest threat, as they are harder to mitigate more effectively, and potentially they can lead to large losses.
Analysis of the results of studies published in [3,4,5,6,7,8], as well as reports of the main developers of information security systems, showed that there is no single effective algorithm for detecting and preventing denial of service attacks. Vendors usually offer an expensive solution that implements a hybrid algorithm based on signature search methods and blacklisting the IP addresses of the attacker's node as a form of risk reduction. An example is the ATLAS system from Arbor, Ltd. Thus, the problem of developing tools for detecting distributed DoS attacks with a high degree of efficiency remains relevant.
2. Existing approaches
According to the authors, anomaly detection systems or ADS appear to be the most promising direction in the detection of attacks, since these systems can detect, among other things, unknown attacks (zero-day attacks). Almost all the anomaly detection models described in the literature can be divided into five groups:
- a) based on behavioral storage patterns [9,10]. Software implementation of this approach must be compiled into the kernel of the operating system, which is almost impossible (for example, in trusted computing systems). In addition, the constant presence of the monitoring component leads to an overall slowdown of the entire system by about 4-50%;
- b) based on frequency [11,12]. Common disadvantages of frequency methods are their poor adaptability, since the reference values of frequencies are determined once with the help of training sets or according to expert data. Moreover, these methods are usually “stateless”, that is, the order in which signs appear is not taken into account;
- c) based on the neural network of the classifier [13,14,15,16,17]. The disadvantage of many neural networks is their poor suitability for processing unordered data sets. Introducing an artificial order for a set of element values will only distort the picture, since the neural network will recalculate weights in accordance with the proximity of numerical values;
- d) based on the synthesis of state machines (state machine) [6,9,18,19,20]. The main disadvantage of this approach is the complex process of building a finite state machine by analyzing the attack scenario. In addition, there are restrictions on the types of attack algorithms that can be described using regular grammars;
- e) other, special: based on Bayesian networks [21], genetic algorithms [22], etc. Most of the works offer only the basic idea, an algorithm, often unsuitable for practical use.
3. Suggested approach
To identify anomalies, the authors propose to use machine synesthesia. In this case, machine synesthesia is understood as an interface that allows the use of image classification algorithms in the task of detecting network anomalies, allowing the use of non-specialized image detection methods that have recently been widely and actively developed [23]. The proposed approach is that network traffic data is “projected” into the image. The accumulation of image changes gives us a video stream, analyzing which, we can conclude about the abnormal state of the observed data transmission network.
The basis of any anomaly detection system is the module that analyzes network packets and determines their potential harmfulness. In fact, ADS tries to classify network traffic into two subsets: normal traffic and network attacks (it does not matter which detection technology is used: based on signatures or statistical). Consequently, the very concept of ADS fits very well with the goals of image classification algorithms — matching the original image with a class of images from a set in accordance with certain characteristics. In addition, image classification as a mathematical tool for analyzing network traffic data and detecting network attacks has several advantages over the anomaly detection methods discussed earlier. These benefits are presented below.
- The mathematical apparatus for classifying images is well developed and tested in practice in many other areas of science and technology.
- A large number of image classification algorithms and wide possibilities for their improvement make this mathematical apparatus very flexible and provide extensive potential for improving the detection of intrusions into the network.
- Most image classification algorithms that demonstrate high practical efficiency are relatively easy to understand and implement in software.
- Classification of images is very effective even with a very large amount of input data. This fact makes us consider these methods particularly suitable for analyzing large network traffic dumps.
- Classification of images can be applied even in the absence of a priori information about the importance of certain characteristics of network packets in the context of detecting certain types of network attacks.
- Interpretation of the results is fairly simple and intuitive.
3.1. Representation of multidimensional TCP / IP traffic data as an image
The authors propose to solve the problem of presenting network traffic metadata in such a way that it will allow using the pattern recognition algorithm to detect anomalies in the video stream.
Consider a network terminal device that collects traffic in a virtual channel. Each assembled package has a set of metadata, represented as a vector p:
where n is the dimension of the vector, P is the set of all vectors, id is the session identifier, date is the timestamp of registration by the terminal, x1, ..., xn is the direction, addresses and ports of the sender and receiver, packet size, protocol type, timestamp (as in TCP segment header), various flags and service fields.
To project the traffic into an image, the orthogonal projection method [24] is used: each vector p is represented by a point in multidimensional space, where n is the dimension of space, then all points (packets) belonging to one session are projected into two-dimensional space:
where a, b are empirically obtained basis vectors for projection into two-dimensional space, X is the original vector constructed from p by removing the id and date elements, X0 is the result of the projection, × is the vector product, () is the scalar product.
The next step in the visualization of a network session is the connection of all its points forming a convex shape. The last step is to fill the resulting shape with color. Then everything is repeated for the next network session. The resulting image is obtained when the imaging process was performed for all network sessions intercepted by the terminal. The accumulation of changes or the differentiation of this image gives us a video stream. In fig. Figure 1 shows examples of images that reflect the correct ("normal state") behavior of the network.
The end of the first part.
How do you like the material? Put pluses, write comments and register for an open lesson , which our instructor, a Cisco Academy instructor in the direction of CCNA Security - Alexey Kulinichev , will conduct already on February 18th.
Source: https://habr.com/ru/post/440210/