Why Use IDS?
True Positives: These are alerts that something is not right when it is actually not right. Example: The IDS finds a packet as containing malicious code and it was actually true that the packet had malicious code, as confirmed by investigation.
True Negatives: These are alerts that something is right when it is actually right. Example: The IDS finds a packet as containing no issues and it actually had no issues.
False Positives: These are alerts indicating that something is not right with a packet when actually it is right. Example: The IDS finds a packet as having malicious code but it is actually a genuine code.
False Negatives: These are alerts that something is right when actually it is wrong. Example: The IDS finds that a packet does not have any malicious code but it actually does contain a malicious code, as found through investigation.
Types of IDS
Host-based IDS: Protects the end system or the network resources.
Network-based IDS: Monitors network traffic for attacks. A Network IDS is deployed on the network near a firewall, on the DMZ or even inside the trusted internal network.
Host-Based IDS (HIDS)
System level protection. Protects from attacks directed to the system
Any unauthorized activity on the system (configuration changes, file changes, registry changes, etc.) are detected and an alert is generated for further action
HIDS functionality works only if the systems generate logs and match against the pre-defined policies. If for some reason, systems do not generate logs, HIDS may not function properly
If hackers bring down the HIDS server, then HIDS is of no use. This is true for any vulnerability protection software
Network-Based Intrusion Detection System (IDS) / Intrusion Prevention System (IPS)
Intrusion Detection System (IDS)
Intrusion Prevention System (IPS)
Passively monitors network behavior and “detects” attacks
Actively analyzes network behavior and “prevents” attacks in real-time
Supports both Network and Host level detection
Supports both network and host level detection
Passive monitoring, does not sit in the data path
Active monitoring, deployed in-line mode
Key measure is detection accuracy
Key measure is lesser number of false positives
NIDS: ISS, Cisco, Enterasys, Symantec
HIDS: ISS, Symantec, Enterasys
NIPS: McAfee Intrushield, NetScreen, Tippingpoint.
HIPS: Cisco, McAfee (Entercept). Snort – an open source Network IDS/IPS developed by Sourcefire
Protects from attacks at the host level
No Bandwidth Impact
Impacts host resources – CPU, memory
Operating System dependent
One agent can protect one host only
Protects network and network resources
Protects against DoS attacks
Sensor hardware is process intensive
Prone to false positives.
How Does Detection Work?
An e-mail with an attachment containing a known malware with an interesting subject (for example, an e-mail with the subject “I love you”).
A “remote login” by an admin user, which is a clear violation of an organization’s policy.
Simple method to create
High false positives rate
Applicable across all protocols
High false negative rate
Multiple signatures are required for a single vulnerability
Too many Telnet sessions on a single day
HTTP traffic on a non-standard port
Heavy SNMP traffic
A web application logged in remotely by a specific set of users
An application which has a specific acceptable password design
Traffic during the peak hours and non-peak hours as defined by the organization
Connectivity pattern from an external partner network
Connecting from a set of mobile devices to the database server
Types of Anomaly
Unusual TCP segmentation and TCP flags combination
Incorrect IP fragmentation and reassembly flags
Erroneous source and destination port numbers
Illegal protocol commands and its usage
Running protocol on non-standard port
Presence of shellcode in unexpected application protocol fields
Misuse of protocol and protocol services
Statistical Anomaly Detection – Statistical DDoS
Detects Unknown Attacks
Prone to false positives
Prevents DoS attacks, Buffer Overflows
Longer detection time
Analyzing Intrusion may be difficult with Anomaly
Difficulty in creating baseline
Stateful Protocol Analysis Detection
Reasonable checks on the standard protocol before an alert
Cannot detect variations to the generally acceptable protocol behavior policy
Cannot detect any conflict between the standards and how they are implemented
IDS/IPS System Architecture and Framework
Alert/Event Viewer: Displays all the intrusions detected by the sensors, which have violated the defined set of policies. The alert viewers should be able to provide drill-down capabilities to view all the details of individual alerts such as host, destination, service, type of attack, and action taken.
Incident Generator: This enables the creation of real-time correlative analysis of attacks on the network. This should provide the type of incident that has occurred and when it has occurred.
Report Generators: To generate various security reports for the management and further analysis. It should have the capability of generating reports automatically as well e-mailing them to individuals.
System Configuration Tools: Provides all the system configuration features. Setting polices, profiles, responses to attacks, sensor mode of operation, user created profiles, baseline scheduling, defining user roles and responsibilities, sending alerts to central network management console, and other sensor level configurations. It should also have the capabilities to send alerts to the central network management console and alerting administrators through triggering cell phone calls and SMS services.
Signature Update Server
Logging capabilities: Should support logging related to intrusion detection, incidents, and other system-related information and should be able to categorize the severity, the impact, and the priority of the intrusions and provide information regarding the prevention actions it has taken. The system architecture should have capabilities to store logs both locally and at a central repository and should have the capability to synchronize time with Network Time Protocol.
Detection Capabilities: Should have broad and extensive detection capabilities; up-to-date threat signatures; the flexibility of customization and fine tuning of the baseline profiles and user-defined profiles to improve detection capabilities; the capabilities to set threshold limits to minimize false positives; and be able to block a connection after a set of failed connection time (retries).
Code viewing and editing capabilities: Technologies should support viewing the virus code or threat code to understand the nature of the threat. This helps in writing a customized signature locally.
Prevention Capabilities: It should have the flexibility to configure prevention capability for each type of attacks. It should support recommendation for prevention for certain unknown attacks and DoS attacks. This helps the administrator to fine tune the policy and reconfigure the sensors.
Attack types Detected by IDS
Attacks Detection by IDS/IPS
Shellcode in password
Too many strange IP fragments
Too much UDP than TCP
Many HTTP requests than responses
Anomaly (Unknown attacks)
TCP SYN Flood attack
TCP or UDP Flood
Ping of Death
Denial Of Service (DOS)
Distributed Denial Of Service (DDoS)
IP fragmentation overlap, options, etc.
TCP segmentation overlap, options usage
All checksum/length consistency
Transport layer reconnaissance and attacks
DNS request – Illegal field value and combinations
HTTP, SNMP, SMTP – Illegal use of commands
Unusually short or long field lengths
Unknown protocol port numbers – Gnutella on port 80, HTTP on port 89
SQL Injection Attack
Telnet/FTP escape sequence attacks
Application protocol anomaly
Port scan/Network Scan
Responses by IDPS to the Intrusions
Block or Deny the packet: When the next packet arrives from the same source, IDPS can simply block that particular user’s data packets entering the network by automatically configuring the sensor to “block.” The intended bad packet never reaches the destination and it is blocked at the entrance itself.
Reset connection : Reset the session when the next packet arrives from the same source. Close the session of the intrusion source. The goal is to terminate the attack before it succeeds. When the attack is detected, RESET connection instructions should be sent to the host in the trusted network. Unfortunately, if the RESET packets are not received in time by the host in the trusted network, then the attacker may succeed. RESET is applicable only for TCP packets and cannot be used for UDP or ICMP packets.
Dropping the packet: Completely drop the packet with intrusions. As soon as the intrusion is detected, identify the source and automatically configure the sensor to drop the packet from that source. The bad packets never reach the intended destination.
Reconfigure firewall: Depending on the type of deployment and where the sensor is deployed, as soon as the intrusion is detected, IDPS can instruct the firewall next to it to change the “Access rules/policies” to deny the packet from the intrusion source, thus preventing any attackers from succeeding.
Intrusion Prevention: During in-line mode, the sensors are in prevention mode either by blocking the traffic or dropping the packets in case of intrusion, thus preventing malicious packets reaching their intended destination. The sensors can be configured for countermeasures such as reset connection, reconfigure firewall, or block the traffic as soon as they detect any intrusions by mediating the traffic flow.
However, the risk of in-line mode is the granularity of identifying the malicious packets. The sensors should be designed to take preventive measures only against those packets that are malicious. Reconfiguring a firewall with false positives can prevent genuine traffic from entering or leaving the trusted network.
Processing at wire speed: Sensors deployed in in-line mode should process packets at wire-speed, otherwise the traffic passing through the sensors can become a bottleneck and hinder the network performance.
Traffic Normalization (Packet Scrubbing): Though the baseline profiles have been created with what is perceived to be normal traffic, sometimes ambiguities such as IP fragmentation can cause false positives. The sensors can reassemble IP fragments, TCP segments, at the sensor level, normalize the traffic, re-evaluate the profiles, and improve upon false positives.
It is important to deploy in-line mode sensors in a high-availability state. In in-line mode, there are high chances of the sensors becoming single points of failure which will result in a complete breakdown of the network. If a network is running in in-line mode, it is recommended to have two sensors in a high-availability mode as shown in the Figure 11-10.
IDS/IPS in Context
We first defined intrusion in lay terms. Then we mentioned that IDS helps to detect intrusions and differentiated it from a firewall. We also learned that IDS peels off the packet and inspects it to understand whether the packet has any malicious code or can lead to any malicious activities. We also mentioned that IDS complements a firewall by doing what a firewall cannot do.
We looked at why we need to use IDS. We mentioned that IDS not only provides alerts on intrusions but also enables us to take appropriate actions including corrective actions, based on root causes, to eliminate such intrusions in the future. We looked into a few of the important terminologies like false positives, true positives, false negatives, and true negatives in the context of the results of IDS.
We then explored both the important types of IDS: host-based IDS and network-based IDS/IPS. We went through the details of host-based IDS including how it monitors the access to the system, its application, and sends alerts for any unusual activities. We then explained that it constantly monitors event logs, system logs, application logs, user policy enforcement, rootkit detection, file integrity, and other intrusions to the system. We then explained how the changes in logs can be interpreted by IDS and alerts are provided by IDS against any intrusions. We looked into the context of network-based IDS/IPS in that it inspects the network packets and checks against the stored malicious signatures to determine whether a packet has been sent with a malicious intention or not. We then differentiated between IDS and IPS. We further explored the pros and cons of both host-based IDS and network-based IDS/IPS.
We explained how the signature-based detection and anomaly-based detection are used by IDS to identify the intrusions and provide the alerts. We then explored Protocol Anomaly and Statistical Anomaly Detection. We also looked into the advantages and disadvantages of the Anomaly-based Detection.
We then explored Stateful Protocol Analysis Detection and listed the pros and cons of this form of detection.
We explored on the architecture of IDS/IPS. In this context, we looked into the functions of important components of the IDS/IPS like Appliance (Sensors), Database, Management Console, and Signature Update Server, including the need to keep the signatures updated so that the detection is appropriately ensured. We also looked into the important capabilities of the IDS/IPS of needing to have logging capabilities, detection capabilities, prevention capabilities, code viewing, and editing capabilities.
We then discussed various attacks the IDS/IPS can detect and prevent. We further discussed the various responses of IDS/IPS, including blocking, denying, or dropping a packet; resetting the connection; or reconfiguring the firewall.
We discussed various modes in which the IDS/IPS can be deployed like SPAN mode, TAP mode, and in-line mode. We also looked at how IPS needs to be supported by wire speed processing in in-line mode.
We ended the chapter with a final note on how firewall, IDS/IPS, and anti-virus play complementary roles to each other.