A key building block for any security analytics solution is the data management component. Such solutions are required to process and ingest large amounts of data from numerous sources, analyze the data in order to identify patterns and lastly make the data quickly accessible for investigation. The need to effectively analyze large datasets and perform more complex investigation to solve data-intensive problems is what enables the discovery of fine-grained anomalies and potential threats. The continuous monitoring model for rapid detection and response of security threats is one of the most intriguing use cases made available by adopting scalable data architecture such as Hadoop. At Fortscale we are solving such challenges since inception using our Hadoop-based architecture. Most recently, Fortscale partnered with Cloudera, the leader in enterprise analytic data management powered by Hadoop, in order to continue enhancing Hadoop based security analytics solutions.
At last month’s Gartner Security Summit, much of the discussion revolved around an historic shift from conventional incident response to continuous monitoring, or rapid detection and response. During her keynote speech, Gartner VP Avivah Litan repeated the idea that “most enterprise security is based on yesterday’s security concepts that use rules and signatures to prevent bad occurrences.” She added that “what’s needed is rapid detection and response, enabled in part through behavioral analytics.”
Subsequently, the paradigm of “incident response” is flawed, as it represents a reactive approach based on the wrong assumption that once the incident has been contained, the threat is contained.
According to FireEye’s M-Trends 2015 Report, attackers enjoy a median of 205 days (!) within the network before being discovered, showing how critical continuous monitoring is becoming for organizations. A more recent Gartner study points out that security analytics solutions enable enterprises to collect and analyze diverse data at high volume, perform long-term analysis and solve data-intensive problems. What’s key around all these requirements and is emphasized in the next figure, is how critical it is to choose the right big data architecture to enable the analytics solution to scale efficiently.
Here’s an example for why the conventional incident response process is limited. Several months ago, one of our customers discovered a sophisticated attack using Fortscale. The attacker penetrated the network, took over an unused set of credentials and then became completely inactive for an entire month in order to evade standard security controls. Once he returned and started working in full speed, Fortscale detected the suspicious behavior and helped the security team protect sensitive customer data by mitigating that instance of the threat. The following screenshot, based on the attack reenactment in our lab environment, shows the initial attempt by the attacker to establish access (left hand side of the chart, 2 blue dots), followed by 4-week “dwell time”, and lastly the “burst” of activity by the attacker towards his objectives:
However, that was just one instance of a broader attack campaign. The attacker came back again (and again) through multiple additional paths in order to pursue his objectives.
Ongoing attack campaigns and persistent adversaries necessitate transitioning from a reactive, linear process that is based on the premise that detection leads to containment to an ongoing, continuous process that can better meet the dynamic and persistent nature of threats:
The continuous response model dictates a different set of tools and capabilities that will enable a range of analytics solutions to model behavior across large volumes of data and through long periods of time in a data-intensive environment. For that purpose primarily, we architected Fortscale User Behavior Analytics since inception on top of a Hadoop infrastructure. In Gartner’s recent study on security analytics, the following distinction was suggested between SIEM (Security Information and Event Management) products and next-generation analytics products:
While SIEM enables near-real-time, short-term analysis, Hadoop-based solutions enable substantial historical retention (some of our deployments rely on a data set longer than a year), long-term analysis and deeper analysis to factor the contextual and behavioral patterns. This is a critical component in any security architecture, as the ability to factor a longer behavioral baseline with a greater breadth of data sources into the analysis is critical. Lastly, Hadoop allows the linear scalability, which is fundamental for the growing needs of large-scale enterprises. By leveraging on capabilities delivered by Cloudera Data Hub and Hadoop, we had the opportunity to see the powerful potential of advanced security analytics solutions.
For more information on the Fortscale-Cloudera partnership, click here.
