Fundamental Requirements of SIEM

"If SIEM product cannot meet a few specific performance requirements, it's underlying features, no matter how well implemented, become useless in day-to-day operations."

Realizing the Purpose of SIEM

Finding and Resolving Security Incidents

Discussions of Security Information and Event Management (SIEM) often focus on features, but it is important to remember that all features must support an overlaying purpose or function. The two most important functions of SIEM are Incident Detection and Incident Response: one requiring the simplification of large amounts of data; the other requiring depth of detail and the addition of context. Many SIEMs fail to meet these basic expectations because both functions require better system performance than what is commonly available. By setting performance baselines for event collection, concurrent analysis, and report response time, however, it is possible to establish the minimum requirements for a truly functional SIEM.

Ultimately, it all comes down to data. With the increasing proliferation of data networks, and the reliance on those networks for critical business applications, the amount of raw information that needs to be managed is already beyond the capabilities of most data processing systems—and the situation is getting worse. To meet the fundamental requirements of SIEM (incident detection and incident response), the first necessity is to develop a data processing architecture capable of meeting these scalability and performance requirements. At a minimum, a SIEM must be capable of all of the following, without exception:

The collection of new events at rates consistent with observed network behavior. This includes network flow collection as well as the collection of logs and events from network devices and hosts. A baseline example for a mid-size network shows the need to support thousands of events per second and tens of millions of events per day.
Storing data for a period of at least one year in an accessible manner, making that data available for immediate investigation. With millions of events being collected every day over one year, this requires the ability to manage billions of events and flows, concurrently.

The ability to access data and retrieve actionable intelligence, on all collected data, in "a timely manner."

A SIEM must be able to provide actionable intelligence on tens of millions of events per minute, to enable mitigation of detected incidents.

Once these basic requirements are met, additional features add value to the system: correlation of individual events to detect larger incidents; notification and workflow management; vulnerability and risk assessment; etc. However, all of these features add value only if the appropriate amount of observed network and device activity is being analyzed, and the results are achieved within an appropriate timeframe.

Why Performance is Fundamental

A SIEM needs to detect risks and threats to our infrastructures (Incident Detection) and then allow forensic investigations, in a timely manner, for the remediation of those incidents (Incident Response). Examining both of these key functions, it becomes clear that each is dependent upon the collection, storage, and analysis of data, with more value being realized as the amount and scope of data is increased.

Incident Detection

Incident Detection is the first goal of SIEM, and it can be seen, once detection is broken down into it's fundamental parts, that the amount of data monitored is critical to the process. It is therefore a strict requirement that all available data be supported, at observed rates occurring in the network, for a SIEM to provide ay value at all. Incident Detection involves:

Evaluating Risk—to help determine the severity of individual attacks, and to help prioritize forensic investigations. For risks to be accurately assessed, all aspects of an infrastructure need to be monitored, and supplemental information (such as vulnerability data, user/identity information, system information, and behavioral patterns) must be applied to that data. Because risk evaluations apply to both the detection of new security breaches and to forensic investigations, the faster the assessment can be made, the better. In addition, because experienced hackers are likely to implement slow attacks, events must be observed over longer periods of time to truly evaluate risk.
Evaluating Events—the correlation of observed events and activity is often the focal point of SIEM evaluations, because it enables "too much data" to be reduced to "an actionable amount of data." However, the reduction of data is an imposition, forced upon SIEMs by their architectural weaknesses: the only acceptable "actionable amount" of data is "all data" — anything else is a compromise. In order to detect patterns that may be indicative of a larger, more complex event (an "incident"), as much data must be correlated as is possible, from as wide a scope as possible. Furthermore, the end result of correlation must be the simplification of the user interface only—the full granularity of all collected events and activity need to be preserved for forensic purposes.
Incident Notification—to inform security professionals of possible risks and of observed incidents, so that further action can be taken. Unfortunately, this function of incident detection often trumps the requirement for data preservation, which becomes necessary once further action begins, i.e., during the process of incident response. Incident notification must occur in real time.

Incident Response

The end-result of incident detection is the need to take further action. What is often overlooked is that there are often hard- and soft- dollar costs, as well as potential exposure and liability concerns, that continue to accumulate throughout the process of incident response. Example: certain vulnerabilities, such as DNS cache poisoning, take time to exploit. In the case of DNS cache poisoning, internal tests have shown an average of four minutes is required for the exploit to succeed. While the result of successful DNS cache poisoning can be extremely detrimental to ongoing business operations, the exploit can be detected almost immediately, leaving a window of just under four minutes for an incident response team to act preventatively on that notification.

Tasks that might be taken include looking at the returned IP address, issued by the invalid DNS response, and proactively restricting traffic to or from that host.

When is Incident Response Fast Enough?

Consider the following example. A SQL-slammer attack is detected by an intrusion prevention device, and a notification is sent to an administrator. He or she must use a SIEM to:

Examine the SQL slammer event
Examine the target of that attack
Determine what other systems might have been targeted by the source of the initial attack
Determine what other events might have originated from the source of the attack
Verify if the target is vulnerable to the attack
Verify if any other identified targets (from step 3) are vulnerable to the attack
Assuming (for the sake of example) that one or more identified targets are vulnerable, determine what other events are associated with the potentially compromised server(s)
Even more investigations might be warranted based on the severity of the attack, which requires knowledge and consideration of assets, threats, and vulnerabilities in addition to the event and flow analysis performed above. Because a SQL slammer attack will result in root privileges, this investigation is not academic: it is a race condition. An attacker will quickly move to compromise other systems, create or alter accounts, open backdoors, or install root-kits — and only with rapid-response forensic capability will a security operations team be able to prevent the irrevocable compromise of the network.

So, when is incident response fast enough? When it is faster than the attack.

This translates to the ability to perform multiple queries per minute on all collected information. This is required to fully detect attack vectors and to determine the propagation of an attack through the network (the above example would require a minimum of six event queries). Because flow information, event information, and log information is needed to perform these types of investigations, the system must support high collection rates and, as a result, much larger total volumes of events. Despite the high volume of data that must be managed, high-performance analysis and reporting is a strict requirement. Without the ability to meet these basic performance requirements, the system is incapable of performing these functions fast enough to allow mitigation of the threat.

security information management: capabilities vs. event and anlaysis requirements Looking at Incident Response outside of the previous example, it is clear that the immediate availability, of as much relevant information as possible, is important—as the relationship between all events and activities can provide valuable context during the investigative process. However, the speed at which that data can be accessed is just as important. If cost, exposure and liability can only be minimized if addressed in under four minutes, a SIEM that is incapable of returning search results, queries, and other analytical requests in a few seconds holds little value in terms of incident response. Though often overlooked when evaluating a SIEM, incident response is an important and necessary function, taking over where Incident Detection ends:

Minimizing Risk—by evaluating the infrastructure against observed activity and known attack patterns to ensure proper precautions are in place prior to an attack. This is reactive defense: adjustments to policies to prevent repeat occurrences of an attack.
Minimizing Loss—by stopping attacks that are underway, and determining attack profiles for the re-evaluation of policies and precautions, to assist in risk minimization while preventing duplicate attacks in the future. The most obvious requirements for the minimization of loss involve the responsiveness of data queries, analysis, and notifications/reporting. Additional requirements involve obtaining as much data (and context around that data) as possible: e.g., if a database is breached, is there any indication of who accessed the data, and how they gained access? What were the attack vectors? What data was being targeted? What exploits were used? Can they be defended? Without collecting as much data as possible from as many sources as possible (in this example, from database activity monitors, flow analysis, and firewall, IPS, and host log analysis), these questions cannot be answered.
Minimizing Exposure—by determining the extent of an attack (which data has or has not been compromised) and by providing a clear audit trail of activities. E.g., if a database is breached, what data was accessed? Was the data successfully retrieved? What were the contents of stolen data, and what are the implications thereof? Were they credit card numbers or social security numbers? Again, only very granular stores of data from multiple sources can provide the analysis needed to answer these questions (again, in this example, database activity monitors, flows, and various network device logs would be required data sources, at a minimum). If critical data is compromised, the need for rapid response is again paramount to minimize risk, loss and exposure.

Architectural Considerations

Technology Limitations

Available database technologies are built upon a schema of relational tables. Each table consists of a series of columns and rows, with each column being an index and each row being a unique data record. As records are added, the number of rows increases. Because each network flow, log and event represents an added row to a database table, these tables grow in size rapidly when collecting data at the rates required for network security. Tables can also grow through the addition of indices: adding new columns for specific, parsed data such as an IP address, username, or event identifier.

As tables grow, either through the addition of indices or data records, the processing required to perform a table scan increases. Table scans are required to search your collected data, making them the primary bottleneck in forensic operations. In fact, in a moderately indexed table (consisting only of: date, time, event id, description, source IP, and destination IP), a single scan of one million records can take several minutes. Scaling the database to ten million records can reduce response time to hours. This is because, for each search, every record of the table must be examined in sequence, pushing the limits of the host processor and memory.

To avoid these performance issues, SIEMs—which require tables consisting of many millions or even billions of unique records—employ a few tricks. This includes breaking data apart into separate tables, or even entirely separate databases, in order to limit the total number of records in any one table. This can improve performance somewhat when performing specific searches, but limits the usefulness of broad correlation. For example, putting events in one database table, and network flow information in another, may allow each to scale better, but the underlying performance limitations are then compounded when attempting to perform conditional searches on both events and flows, concurrently (in other words, when performing correlation). Another method is to reduce the total number of records stored at any given time, rolling older events and flows into an archive. Alternatively, the period of time over which queries are allowed may be limited to more manageable timeframes, such as 24 or 48 hours. This again improves performance under certain conditions, but compounds the problem when comparing new data to old: a common requirement in forensic investigations. Other methods include: the reduction of indices used to reduce overall table size; in-memory analysis to remove database limitations entirely; or the use of parallel processing (server clusters) to improve overall database performance.

While each solution helps to overcome specific limitations, each presents new problems—limitations of searchable data-points within the system, the reduction of historical analysis, added cost, and extensive maintenance—while failing to fully resolve the issue. Even with high-end server clusters, massive amounts of RAM, limited indexing, frequent archiving and strict restrictions on searches, commercial databases simply can't keep up with the heavy requirements of data collection, storage, retrieval, and analysis that is required by the fundamental practice of incident detection and incident response.

A Better Mouse Trap

To solve the need for greater scalability of the core database, and greater performance when retrieving data from that database, NitroSecurity developed NitroEDB: a data management engine that was designed from the start to support high-speed insertions, and fast access to stored data in very large datastores. With hundreds of man-years invested into research and development in a focused effort to accomplish these specific performance goals, NitroEDB is uniquely capable of extremely high performance—more than enough to overcome the inherent limitations that hinder other solutions. Applied to SIEM, NitroEDB enables NitroView Enterprise Security Manager (ESM) to: collect more information from more sources (more total events per second); store it for long periods of time (total events); and provide extremely fast access to all stored data, even over time (query response time). This translates to:

The ability to collect many tens-of-thousands of events per second, collected from network flows, events, and logs.
The ability to store billions of events, with all of that data being accessible at the same time, without penalty. The amount of total events managed does not impact the rate at which new events are collected, nor does the rate at which events are collected impact the total number of events that are manageable.

The ability to immediately access data collected over very long periods of time, performing multiple queries per minute on the billions of stored events, without impacting event collection and storage.

As a result, NitroView ESM is better equipped to:

Detect incidents, because more event data can be analyzed concurrently, over longer periods of time. Because correlation, detection, and notification features are only as good as the information upon which they can act, this greater breadth of data managed adds significant value to NitroView ESM.
Respond to incidents, providing real-time, operational forensic capabilities to reduce the time required for mitigation and remediation of detected incidents.

Because more data is collected and managed concurrently, there is a greater ability to trace an event to some other relevant context (such as identity, physical location in the network, patch levels, authority, etc). Also, because the investigations occur very rapidly, it is possible to respond to an exploit or incident very quickly after the initial notification, reducing total risk, exposure and liability.

Conclusion

Without both the features necessary to perform rudimentary incident detection and response, and the basic performance characteristics with which to support the requirements of data collection, correlation and analysis, a SIEM cannot meet basic customer requirements. A solution that achieves in one area, but not in the other, is effectively only "one half of a SIEM," and will provide little if any value to daily security operations. As a baseline, a mid-size company requires a SIEM capable of:

Accepting new data feeds, including events, logs and network flows, at a rate of thousands of events per second
Managing and analyzing billions of events concurrently, including sufficient storage and processing of those events
Performing rapid reporting and analysis in minutes, when querying millions of events, to derive actionable intelligence from collected events

Without first meeting these base requirements, no SIEM is capable of its full potential. NitroSecurity's NitroView Enterprise Security Manager (ESM) provides a core architecture capable of exceeding these requirements, allowing for quick and easy access to actionable intelligence on all collected security event data.

Appendix A: Sample Data for SIEM Sizing

Vendor	Device	Daily Events	EPS	Qty Total EPS
Netscreen	Firewall	50,112	0.58	1	0.58
Microsoft	Windows WMI	250,560	0.29	10	2.90
Cisco Pix	Firewall	300,672	1.74	2	3.48
Cisco ASA	Firewall / VPN	400,032	4.63	1	4.63
Cisco IOS	Switch/Router (ACL)	207,360	0.12	20	2.40
HP	Procurve Switch	300,672	1.74	2	3.48
Linux	Server	103,680	0.06	20	1.20
squid	Proxy	50,112	0.29	2	0.58
NitroGuard	IPS	500,256	5.79	1	5.79
Cisco	Catalyst Switch	99,999,360	57.87	20	1,157.40