The Suricata Engine and the HTP Library are available to use under the GPLv2. The new engine supports “Multi-Threading, Automatic Protocol Detection (IP, TCP, UDP, ICMP, HTTP, TLS, FTP and SMB! ), Gzip Decompression, Fast IP Matching and coming soon hardware acceleration on CUDA and OpenCL GPU cards”. GPU integration allows the use of graphic cards to accelerate operations. Mike Cloppert in his post, “Detection, Bandwidth, and Moore’s Law” pointed out:

It appears the authors well understand the point in this post, and the corresponding state of the art in solving parallel computing problems. GPU’s are emerging as a good commodity solution to parallel processing. This is covered in depth by a number of recent publications discussing parallelism, and I am by no means an expert in this field, so I will simply leave follow-up on this point as an exercise for the reader.

The HTP Library is an HTTP normalizer and parser written by Ivan Ristic, creator of Mod Security and author of the soon to be released book “ModSecurity Handbook“. This integrates and provides very advanced processing of HTTP streams. The HTP library is required by the engine, but may also be used independently in a range of applications and tools. Additional details have been provided by Ivan in his post, “HTTP parser for intrusion detection and web application firewalls.” Ivan writes concerning the development, “For the first release of the parser the goal is to be able to parse HTTP streams reliably. In the subsequent versions I will work in the parser’s security properties (such as the ability to see through evasion attacks).”

New Ideas and Concepts

Quoting from the OISF announcement, some of the next generation capabilities include:

• Multi-Threading: so very necessary.
• Automatic Protocol Detection: the engine has keywords for IP, TCP, UDP, ICMP, HTTP, TLS, FTP and SMB. Users can write rules to detect a match within a stream regardless of the port the stream occurs on. This is important for malware detection and control. Detections for more layer 7 protocols are being developed.
• Gzip Decompression: the HTP Parser will decode Gzip compressed streams.
• Independent HTP Library: the HTP Parser will be usable by other applications such as proxies, filters, etc. The parser is available as a library under GPLv2 for easy integration ito other tools.
• Standard Input Methods: support for NFQueue, IPFRing, and the standard LibPcap to capture traffic. IPFW support will be available soon.
• Unified2 Output: support for standard output tools and methods.
• Flow Variables: it is possible to capture information out of a stream and save that in a variable which can then be matched against later.
• Fast IP Matching: the engine will automatically use a special fast matching preprocessor on rules that are IP matches only (such as the RBN and compromised IP lists at Emerging Threats).
• HTTP Log Module : HTTP requests can be automatically output into an apache-style log format file for monitoring and logging activity completely independent of rulesets and matching.

A few features to look forward to in a few weeks:

• Global Flow Variables: the ability to store more information from a stream or match (actual data, not just setting a bit) over a period of time allowing comparing values across many streams and time.
• Graphics Card Acceleration: using CUDA and OpenCL to make use of the processing power of even old graphics cards to accelerate the IDS. Offloading the very computationally intensive functions of the sensor will greatly enhance performance.
• IP Reputation: will allow sensors and organizations to share intelligence and eliminate many false positives.
• Windows Binaries: will be released once there is a reasonably stable body of code.

Folks Behind It

The team is listed on the OISF site. It is an all star cast including Matt Jonkman, Victor Julien, Will Metcalf, Nathan Jimerson, Margaret Skinner, Josh Smith, Brian Rectanus, Breno Silva Pinto, Anoop Saldanha, Gurvinder Singh Dahiya, Jason MacLulich, Jason Ish, Kirby Kuehl, Dennis Henderson, Martin Solum, Ivan Ristic, Pablo Rincon, and Gerardo Iglesias Galvan.

I also wanted to point out some of the heavy hitting organizations involved. The initial funding for OISF comes from the US Department of Homeland Security (DHS), the US Navy’s Space and Warfare Command (SPAWAR), and a number of private companies that participate in the OISF Consortium. The OISF is a part of the DHS Homeland Open Security Technology (HOST) program. OISF works with Open Source Software Institute and has received legal guidance from the Software Freedom Law Center.

OISF is a US nonprofit, a 501c(3) and will not commercialize, sell, patent, copyright, or profit from the engine. OISF Consortium members are donating coders, equipment, and financial support in exchange for the ability to commercialize the engine. The important take away is that OISF has long term support for future development of Suricata.

Final Thoughts

Suricata is a very exciting and promising IDS/IPS engine. It has a great group of people behind it and future development appears secured. It is a project that is in the early stages. Do not expect to download it and simply install on a production environment. For testing the software and providing feedback, the engine and the HTP Library are available for download. To keep apprised of the latest developments join the oisf mailing lists where you discuss and share feedback. The blog of Victor Julien, Suricata’s lead developer, is another great source for the latest news and information.

To finally answer the burning question: why the name Suricata? According to the OISF site, Suricata comes from the Latin genus name for the meerkat and “the Meerkat takes security and vigilance as a life or death responsibility. There is always at least one individual on guard, watching, ready to alert the entire organization. Very much like an IDS sensor. It is always watching, always ready to alert you to danger. Or something like that…”

The Security Content Automation Protocol (SCAP) is an attempt to help defenders by providing a collection of XML schemas/standards that allow technical security information to be exchanged between tools. For example, SCAP can help organizations looking for a way to respond appropriately to new vulnerabilities and threats by helping prioritize, allowing the most significant ones to be addressed sooner. It can also benefit those looking to provide interoperability across system security tools. There is even an effort to “encouraging the use of SCAP as a de-facto standard across the ICT industry for deploying trusted cloud computing services.”

Background

To help understand what exactly SCAP is, let us turn to the U.S. National Institute of Standards and Technology (NIST) Special Publications (SP) 800-117, “DRAFT Guide to Adopting and Using the Security Content Automation Protocol (SCAP):”

SCAP comprises a suite of specifications for organizing and expressing security-related information in standardized ways, as well as related reference data, such as identifiers for software flaws and security configuration issues. SCAP can be used for maintain the security of enterprise systems, such as automatically verifying the installation of patches, checking systems security configuration settings, and examining systems for signs of compromise.

NIST this month is looking for public comments on the first public draft of SP 800-126, “The Technical Specification for the Security Content Automation Protocol (SCAP).” Back in May, NIST released the draft for SP 800-117.

SCAP components consists of:

• Common Configuration Enumeration (CCE): provide unique identifiers to system configuration issues in order to facilitate fast and accurate correlation of configuration data across multiple information sources and tools.
• Common Platform Enumeration (CPE): a structured naming scheme for information technology systems, platforms, and packages.
• Common Vulnerability Enumeration (CVE): a dictionary of publicly known information security vulnerabilities and exposures.
• Common Vulnerability Scoring System (CVSS): a vulnerability scoring system designed to provide an open and standardized method of rating IT vulnerabilities. NIST has even provided a calculator for creating CVSS vulnerability severity scores.
• eXtensible Checklist Configuration Description Format (XCCDF): a specification language for writing security checklists, benchmarks, and related kinds of documents. NIST has released the NIST Interagency Report 7275 Revision 3 “Specification for Extensible Configuration Checklist Description Format (XCCDF) Version 1.1.4.”
• Open Vulnerability Assessment Language (OVAL): an information security community standard to promote open and publicaly available security content, and to standardize the transfer of this information across security tools and services.

The National Checklist Program (NCP), outlined in NIST SP 800-70, is the repository for SCAP-expressed checklists. The checklists provide detailed low level guidance on setting the security configuration of operating systems and applications.

In June, MITRE hosted the Security Automation Developer Days conference, which focused on the U.S. National Institute of Standards and Technology’s (NIST) Security Content Automation Protocol (SCAP). MITRE has made the minutes available, which includes discussion on NIST SP 800-126. Michael Smith has provided some great highlights from the conference in his post, “Security Automation Developers Conference Slides.” The problem with Michael is that it is difficult not to quote his whole blog, which is bad web etiquette. Please follow the link for some real insight concerning the slides. You can also view below Michael’s presentation, “Security Content Automation Protocol and Web Application Security:”

Back in September 2008, NIST sponsored the Fourth Annual Security Automation Conference. The Presentations are available. Ian Charters attended and posted his thoughts, “NIST and SCAP; Busting a cap on intruders Part 1.” The 5th Annual IT Security Automation Conference will be held October 26-30th, 2009 at the Baltimore Convention Center.

Make sure to check out below the OWASP video talk from SnowFROC 2009 by Ed Bellis (from Orbits) on vulnerability management titled “Doing more with less? Automate or die.”

Ed’s has also written an article for CSO Online, “How SCAP Brought Sanity to Vulnerability Management.”

Possible Problems

Some may argue that SCAP is overly complicated and people are better off relying solely on their vendor’s products and reports. That assumes that a single vendor product is sufficient to meet tomorrow’s security needs. Some organizations buy into the platform simplification model where basically they purchase a single vendor line of products in order to avoid interoperability problems. The problems is that one vendor frequently only does a few things well. The agility of the organization to adapt to changes in the security world becomes dependent solely on that single vendor. After investing so much into that one vendor, organizations find that they are completely locked in. Probably that is not the best position to be in when facing a very volatile IT environment.

Consider the below list where NIST outlines areas SCAP validation will cover (Source: NIST Interagency Report 7511 “Security Content Automation Protocol (SCAP) Version 1.0 Validation Program Test Requirements (DRAFT)):”

• FDCC Scanner: the capability to audit and assess a target system to determine its compliance with the FDCC requirements.
• Authenticated Configuration Scanner: the capability to audit and assess a target system to determine its compliance with a defined set of configuration requirements using target system logon privileges.
• Authenticated Vulnerability and Patch Scanner: the capability to scan a target system to locate and identify the presence of known vulnerabilities and evaluate the software patch status to determine compliance with a defined patch policy using target system logon privileges.
• Unauthenticated Vulnerability Scanner: the capability of determining the presence of known vulnerabilities by evaluating the target system over the network.
• Intrusion Detection and Prevention System (IDPS): the capability to monitor a system or network for unauthorized or malicious activities. An intrusion prevention system actively protects the target system or network against these activities.
• Vulnerability Remediation: the capability to install patches on a target system in compliance with a defined patching policy.
• Misconfiguration Remediation: the capability to alter the configuration of a target system to bring it into compliance with a defined set of configuration recommendations.
• Asset Scanner: the capability to actively discover, audit, and assess asset characteristics including: installed and licensed products; location within the world, a network or enterprise; ownership; and other related information on IT assets such as workstations, servers, and routers.
• Asset Database: the capability to store and report on asset characteristics including: installed and licensed products; location within the world, a network or enterprise; ownership; and other related information on IT assets such as workstations, servers, and routers.
• Vulnerability Database: a catalog of security-related software flaws labeled with CVEs where applicable. This data is made accessible to users through a search capability or data feed and contains descriptions of software flaws, references to additional information (e.g., links to patches or vulnerability advisories), and impact scores. The user-to-database interaction is provided independent of any scans, intrusion detection, or reporting activities. Thus, a product that only scans to find vulnerabilities and then stores the results in a database does not meet the requirements for an SCAP vulnerability database (such a product would map to a different SCAP capability). A product that presents the user general knowledge about vulnerabilities, independent of a particular environment, would meet the definition of an SCAP vulnerability database.
• Misconfiguration Database: a catalog of security-related configuration issues labeled with CCEs where applicable. This data is made accessible to users through a search capability or data feed and contains descriptions of configuration issues and references to additional information (e.g., configuration guidance, mandates, or other advisories). The user-to-database interaction is provided independent of any configuration scans or intrusion detection activities. Thus, a product that only scans to find misconfigurations and then stores the results in a database does not meet the requirements for an SCAP misconfiguration database (such a product would map to a different SCAP capability). A product that presents the user general knowledge about security-related configuration issues, independent of a particular environment, would meet the definition of an SCAP vulnerability database.
• Malware Tool: the capability to identify and report on the presence of viruses, worms, Trojan horses, spyware, or other malware on a target system.

It is difficult to imagine a single security product that is capable of doing all the above services well. There is a need to be able to share information between various systems performing these functions.

Game Changing Technology

Considering past statements by Aneesh Chopra, the first Chief Technology Officer of the United States, does not SCAP sound like an area that will be getting additional support by the U.S. government? ZDnet has posted a very interesting podcast of Chopra talking at the Computer History Museum. Chopra wrote a few months back:

If confirmed, I would emphasize a research program on “game-changing” ideas in cybersecurity, to find new ideas that might transform the nation’s information infrastructure to be more secure and simpler to understand and use. The goal is to make it “easy to do the right thing, hard to do the wrong things and easy to recover when the wrong thing happens anyway.”

Tim O’Reilly, one of the most insightful person around in respect to IT, wrote back in April “Why Aneesh Chopra is a Great Choice for Federal CTO.” Tim’s points out items that Chopra has accomplished in Virginia:

1. the first officially-approved open source textbook in the country, the Physics Flexbook;
2. integrating iTunes U with Virginia’s state education assessment framework;
3. the Learning Apps Development Challenge, a competition for the best iPhone and iPod Touch applications for middle-school math teaching;
4. a Ning-based social network to connect clinicians working in small health care offices in remote locations;
5. a state-funded “venture capital fund” to allow government agencies to try out risky but promising new approaches to delivering their services or improving their productivity;
6. a lightweight approval and testing process that allows the government to try out new technologies before making a full, expensive commitment.

Back in April 2007, Chopra was behind Virginia’s 95 agencies opening up their databases to the Google search engine, in order to make them widely accessible to the public. Chopra at that time stated the top priority of the state’s strategic plan for information technology, which was adopted last year, is increased access to government information. A great thing to do, provided security is insuring only the information you want is being accessed in the manner intended.

John Dvorak offers a different opinion of Chopra in his post “Special Report: Is US Chief Information Officer (CIO) Vivek Kundra a Phony?” Dvorak states, “It would be logical to assume that Kundra managed to get his buddy Chopra the CTO job despite the fact that Chopra’s technology background is essentially nil.” Whether O’Reilly or Dvorak is correct, Chopra needs to start reading the Guerilla CISO for great insight into security solutions. Michael outlines a plan on fixing government patch and vulnerability management through SCAP in the post, “Federated Vulnerability Management.” Here are a few of the ideas discussed in the post:

• Every IT asset reports into a patch management system of some sort. Group the assets allowing for identification of who is responsible when something has a problem.
• Do periodic network scanning.
• The orchestrator will correlate network scans with patch management status and gives a ticketing/alert/whatever where unmanaged devices are identified.
• The NVD feed is pushed down to the agencies/departments which are sent out as vulnerability alerts along with the checks to see if systems are vulnerable.
• Hardening guides are pushed from the agencies/departments in SCAP form and audit information is pulled of IT assets. Differences are automatically entered into a workflow and reporting system.

Imagine the additional possibilities when intrusion detection/prevention systems, patch remediation, asset scanner, and malware tools start sharing information.

SCAP and the Cloud

Aneesh Chopra should also read Christofer Hoff’s rational Survivability blog. In Hoff’s post, “Extending the Concept: A Security API for Cloud Stacks“, he considers building on the capabilities of SCAP to embed a “standardized and open API layer into each IaaS, PaaS and SaaS offering (see the API blocks in the diagram below) to provide not only a standardized way of scanning for network vulnerabilities, but also configuration management, asset management, patch remediation, compliance, etc.” Hoff goes on to write, “Further (HT to @davidoberry who reminded me about my posts on the topic) we could use TCG IF-MAP as a comms. protocol for telemetry.”

Hoff is another person who is difficult to quote without including his complete post. He makes the point that you gain “automated audit and security management capability for the customer/consumer and a a streamlined, cost effective, and responsive way of automating the validation of said controls in relation to compliance, SLA and legal requirements for service providers.” By doing so, Hoff points out, you are “not reinventing the wheel and we have lots of technology and standardized solutions we can already use to engineer into the stack.”

Update: Hoff pointed out (see comments area) some of the excellent work done by Iron Frog (Ben) in not only his post “Some thoughts for addressing the Assurance component of A6,” but also his series of post “Can we do the Security Stack API RESTfully? (parts 1, 2, 3, 4, and 5).”

Peter Mell, who recently changed positions at NIST from the SCAP validation program manager to the leader of the agency’s Cloud computing project, will likely agree with Hoff’s points. Expect NIST efforts in the Cloud to take SCAP into consideration.

Final Thoughts

As Michael Smith points out, in the Cloud one faces the same problems as a managed service provider, mainly how to allow the auditing of systems and the underlying infrastructure. An API could allow a managed services environment making security tasks much easier to customers. To quote Michael Smith, “we have in SCAP is Common Platform Enumeration (CPE) which allows you to specify the hardware and software (ie, how the infrastructure that you don’t know about is built) and eXtensible Configuration Checklist Description (XCCDF) which specifies the audit/compliance checks. Package them together and you have a way of describing what the infrastructure looks like and the technical auditing standard to go along with it.” Sounds like some game changing ideas that could transform the nation’s information infrastructure, helping it be more secure. I hope you are listening, Aneesh Chopra.

The original slides made heavy use of the Microsoft PowerPoint animation feature. Unfortunately, SlideShare does not currently support animation. You can download the presentation and the animations will work, but I ended up modifying the slides so they are more viewable online. SlideBoom will keep the animation, but it does it by creating a video of the presentation. I decided to stick with SlideShare and spare you the resulting nine minute video. While I should add audio and make a SlideCast, this post might never be completed if I wait until I have time to create a really nice web presentation.

Merriam-Webster defines a totem as any supposed entity that watches over or assists a group of people, such as a family, clan, or tribe. In this presentation I focused on how TOTEM assists in watching over and evaluating the threat an IP represents. The idea behind TOTEM is simple: compare threat information from sources such as watchlists (DShield, Emerging Threats, SenderBase, etc.) to activities with the organization (IDS/IPS, flow logs, etc.) and other locations (SANS ISC, DOE federated model, etc.). As new threat information and activity sources are added, a better evaluation can be rendered.

The purpose of this presentation has been to share the basic ideas behind TOTEM with the hope that others may provide helpful insight. So far I have not disappointed. I wanted to thank everyone for I have received some very intriguing ideas.

Joseph Campbell, an American mythology professor, writer, and lecturer, wrote “Computers are like Old Testament gods; lots of rules and no mercy.” In the security world, signatures would be the rules that computers follow. While signatures can be very useful, they also are very limiting. In a previous post, titled simply “IDS“, we discussed how Intrusion Detection System/Intrusion Prevention System (IDS/IPS) technology is moving away from being solely signature based to a blend of signature based, anomaly detection, and activity based methodologies.

This is not surprising when we examine other areas of security. Liam Tung, writer for ZDnet, has written an article titled, “Signature-based antivirus is dead: Get over it“. In the article, Simon Clausen, founder & CEO at PC Tools, reports that the security industry has been looking beyond blacklists. “I would very much disagree that AV is dead. Really, traditional signature-based AV is going to be dead in a few years, but what every antivirus company is evolving towards, like us, is behavioural AV technology, so AV will be alive.”

Bro is not signature based. Instead, it was developed to be activity based with some support for anomaly detection. From the beginning, Bro has always been focused on connections instead of just packets. The Bro development team continues to advance the software, as demonstrated with the extensive changes in version 1.4. Below are a few of the new features:

• Can import Netflow version 5 data.
• A BitTorrent analyzer is now available.
• The “Bro Lite” configuration is now deprecated and will not in
  general be supported.
• Substantial updates to Broccoli, a Bro client library.
• Extensive changes to allow Bro to process packets captured in the past intermingled with those captured in real-time.
• scan.bro has been heavily modified to better support distributed scan analysis.
• The new policy script targeted-scan.bro looks for repeated access from the same source to the same server, to detect things like SSH password-guessing attacks.
• GeoIP information now includes latitude and longitude.
• ssh.bro now supports the variable skip_processing_after_handshake which directs the event engine to omit any further processing of an SSH connection after its initial handshake.
• Google’s perftools have replaced mpatrol for leak-checking and heap-profiling.

Additional Information

For additional information on Bro, below is a list of a few good site.

• Bro Quick Start Guide – contains info on installing, configuring, and running Bro.
• Bro Wiki – intended for users and developers of Bro.
• The Bro Archives – mailing list archive.
• Emerging Bro – Bro signatures repository.
• The ICSI Networking Group Blog – the Blog for the Network Research at the International Computer Science Institute in Berkeley, CA. These are the folks that develop Bro.
• A Bro Blog – A blog by Seth Hall, a Bro master and contributor.
• C.S.Lee (geek00L) blog When {Puffy} Meets ^RedDevil^ .
• TaoSecurity – Richard Bejtlich has some good posts on Bro.
• Last Bro Workshop Material.

Supporting Software

Bro offers many configuration options, depending on how you will use the software. Below are a few libraries and software packages required by Bro:

Below are a few libraries and software packages that are not required, but you should consider installing. The packages, except GeoIP and Google Perftools, should have binaries available for your OS. Use these ports to install the packages and save yourself the trouble of having to keep the software updated. We will go through through the installation of GeoIP and Google Perftools from source code.

GeoIP Installation and Configuration

MaxMind GeoIP is a collection of APIs for looking up the location of an IP address. There is a collection of free GeoLite databases, which are not as accurate as the GeoIP databases, but will do for starting out and testing with Bro. To setup GeoIP for use with Bro, please follow the commands below.

root# cd /usr/local/src /usr/local/src root# wget \ http://www.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz /usr/local/src root# gunzip GeoLiteCity.dat.gz /usr/local/src root# mkdir -p /usr/local/share/GeoIP /usr/local/src root# mv GeoLiteCity.dat /usr/local/share/GeoIP/GeoIPCity.dat /usr/local/src root# wget \ http://www.maxmind.com/download/geoip/api/c/GeoIP.tar.gz /usr/local/src root# tar xzf GeoIP.tar.gz /usr/local/src root# cd GeoIP-1.4.5 /usr/local/src/GeoIP-1.4.5 root# ./configure /usr/local/src/GeoIP-1.4.5 root# make /usr/local/src/GeoIP-1.4.5 root# make check /usr/local/src/GeoIP-1.4.5 root# make install

Make sure /usr/local/lib is placed into your library path.

Google Perftools Installation and Configuration

Google’s perftools is a collection of a high-performance multi-threaded malloc() implementation and some performance analysis tools. Google’s perftools have replaced mpatrol for leak-checking and heap-profiling. We will compile Bro with –enable-perftools. By default, perftools will install under /usr/local directory. With perftools compiled into Bro, there are two command-line options made available:

To help with the installation of Google’s perftool, the ICSI Networking Group has written a post “Making Sure Your Bro Code Does Not Leak.” The post will provide additional information. The basic steps to install perftools are:

root# cd /usr/local/src /usr/local/src root# wget \ http://google-perftools.googlecode.com/files/google-perftools-0.99.2.tar.gz /usr/local/src root# tar xzf google-perftools-0.99.2.tar.gz /usr/local/src root# cd google-perftools-0.99.2 /usr/local/src/google-perftools-0.99.2 root# ./configure /usr/local/src/google-perftools-0.99.2 root# make /usr/local/src/google-perftools-0.99.2 root# make check /usr/local/src/google-perftools-0.99.2 root# make install /usr/local/src/google-perftools-0.99.2 root# export LDFLAGS=-L/usr/local/lib /usr/local/src/google-perftools-0.99.2 root# export CFLAGS=-I/usr/local/include /usr/local/src/google-perftools-0.99.2 root# export CPPFLAGS=-I/usr/local/include /usr/local/src/google-perftools-0.99.2 root# export LD_LIBRARY_PATH=/usr/local/lib

XML Analyzer

The XML analyzer is highly-experimental code written by Tobias Kiesling. Installation of Xerces-C++ and XQilla is required to use the XML analyzer. The code allows you to be able to easily adjust analysis capabilities to specific XML data formats. Xerces-C++ is a validating XML parser written in a portable subset of C++. XQilla is an XQuery and XPath 2 library and command line utility written in C++.

root# cd /usr/local/src /usr/local/src root# wget \ http://downloads.sourceforge.net/xqilla/XQilla-2.1.3.tar.gz /usr/local/src root# wget \ http://mirror.its.uidaho.edu/pub/apache/xerces/c/2/sources/xerces-c-src_2_8_0.tar.gz /usr/local/src root# md5sum xerces-c-src_2_8_0.tar.gz 5daf514b73f3e0de9e3fce704387c0d2 xerces-c-src_2_8_0.tar.gz /usr/local/src root# tar xzf xerces-c-src_2_8_0.tar.gz /usr/local/src root# tar xzf XQilla-2.1.3.tar.gz /usr/local/src root# ln -s XQilla-2.1.3 xqilla /usr/local/src root# cd xerces-c-src_2_8_0 /usr/local/src/xerces-c-src_2_8_0 root# patch -p1 < ../xqilla/src/xercesc_content_type.patch /usr/local/src/xerces-c-src_2_8_0 root# patch -p1 <../xqilla/src/xercesc_regex.patch /usr/local/src/xerces-c-src_2_8_0 root# export XERCESCROOT=`pwd` /usr/local/src/xerces-c-src_2_8_0 root# cd src/xercesc

FreeBSD 7 users will encounter a problem when trying the run the runConfigure command. The error “C compiler cannot create executables ” will be produced. The problem is on line 358 of runConfigure. The libc_r library cannot be found since it has been deprecated on FreeBSD since version 5.X and removed from version 7.0. Edit runConfigure to not include “-lc_r” in the list of threading libraries. Then issue the command:

/usr/local/src/xerces-c-src_2_8_0/src/xercesc root# runConfigure \ -pfreebsd -cgcc -xg++ -minmem -nsocket -tIconvFBSD -rpthread -s /usr/local/src/xerces-c-src_2_8_0/src/xercesc root# gmake /usr/local/src/xerces-c-src_2_8_0/src/xercesc root# gmake install

An easier option is to install xerces and xqilla through the FreeBSD port command:

root # cd /usr/ports/textproc/xqilla /usr/ports/textproc/xqilla root # make install clean

Other operating systems, such as Linux, do not require any special steps. You just need to run the commands:

/usr/local/src/xerces-c-src_2_8_0/src/xercesc root# runConfigure -plinux /usr/local/src/xerces-c-src_2_8_0/src/xercesc root# make /usr/local/src/xerces-c-src_2_8_0/src/xercesc root# make install

With Xerces-C++, configure and install XQilla.

/usr/local/src/xerces-c-src_2_8_0/src/xercesc root# cd /usr/local/src/xqilla/ /usr/local/src/xqilla root# ./configure --with-xerces=`pwd`/../xerces-c-src_2_8_0/ /usr/local/src/xqilla root# make /usr/local/src/xqilla root# make install

Bro Installation and Configuration

There a few options when installing Bro. Bro was not developed for the PHB. Advance security software provides the power to the user, with all the options to adapt it to your environment. To quote the Bro site, “Bro has been developed primarily as a research platform for intrusion detection and traffic analysis. It is not intended for someone seeking an ‘out of the box’ solution. Bro is designed for use by Unix experts who place a premium on the ability to extend an intrusion detection system with new functionality as needed, which can greatly aid with tracking evolving attacker techniques as well as inevitable changes to a site’s environment and security policy requirements.” With the Unix experts in mind, we will go through the steps involved to install both the stable and the development versions of Bro.

Current Stable Version

The current version should be the most stable. To install, follow these commands:

root# cd /usr/local/src /usr/local/src root# wget ftp://bro-ids.org/bro-1.4-release.tar.gz /usr/local/src root# tar xzf bro-1.4-release.tar.gz /usr/local/src root# cd bro-1.4

The configuration and installations appears below.

Subversion Trunk

Reading the posts on the Bro mailing list, reveals that modifications have already been made to the current release. Fixes are being made continuously. These changes, while fixing problems, might introduce new problems. You do have the option of getting the most up-to-date code possible through the subversion repository. The Bro development team has made available two subparts of the repository: the trunk and development branches. The trunk is the main development head from which releases are made on a regular basis. It should be fairly stable with changes passing a regression suite to ensure the code do not break existing functionality. It is still considered experimental and not suitable for critical deployment. Below is how to download code from the trunk.

root# cd /usr/local/src /usr/local/src root# mkdir bro-cvs /usr/local/src/bro-cvs root# cd bro-cvs /usr/local/src/bro-cvs root# svn checkout http://svn.icir.org/bro/trunk/bro /usr/local/src/bro-cvs root# mv bro bro-1.4.cvs /usr/local/src/bro-cvs root# cd bro-1.4.cvs /usr/local/src/bro-cvs/bro-1.4.cvs root# ./autogen.sh

Robin’s Development Branch

The developers merge their work into the the Bro subversion trunk. Robin Sommer has a separate branch which contains experimental code for:

• the Bro Cluster framework
• NetFlow support (by Bernhard Ager)
• a BitTorrent analyzer (by Nadi Sarrar and Bernhard Ager)
• an XML analyzer (by Tobias Kiesling)
• Python bindings for Broccoli
• restructured logic for taking drop decisions via Bro’s notice framework (by Brian Tierney and Robin Sommer)
• a test-suite for Bro’s communication & serialization subsystems
• various tweaks and bugfixes

If you want the latest work done by Robin and others mentioned above, you can get access to the code with the following commands.

root# cd /usr/local/src /usr/local/src root# mkdir bro-cvs /usr/local/src root# cd bro-cvs /usr/local/src/bro-cvs root# svn checkout http://svn.icir.org/bro/branches/robin/work /usr/local/src/bro-cvs root# mv work bro-1.4.robin /usr/local/src/bro-cvs root# cd bro-1.4.robin /usr/local/src/bro-cvs/bro-1.4.robin root# ./autogen.sh

Configure and Install

Because of the various bug fixes and the additional features which add interesting options, we are going to step through installation of Robin’s branch. Please use the version of Bro appropriate for your operation.

root# cd /usr/local/src/bro-cvs/bro-1.4.robin /usr/local/src/bro-cvs/bro-1.4.robin root# ./configure --enable-debug \ --enable-perftools --prefix=/usr/local/bro --with-xqilla /usr/local/src/bro-cvs/bro-1.4.robin root# make /usr/local/src/bro-cvs/bro-1.4.robin root# make check /usr/local/src/bro-cvs/bro-1.4.robin root# make install

If you run into any problems, go to back to the stable version of Bro and see if you can get it to compile. Then you may want to try the subversion trunk code.

Final Words

We have taken the first step and now have Bro installed. Installation is only the beginning. In an upcoming post, we will walk through configuring Bro and examining a simple policy. We will send some attacks against Bro, and see what kind of results are produced. Having completed the first step of installing Bro, we can move on and have some fun. As Humphrey Bogart put it in his famous last line from Casablanca, “Louis, I think this is the beginning of a beautiful friendship.”

The Three Wise Men Speak

As readers of this blog know, I have been looking at the Bro IDS. I hope to shortly release a post on setting up Bro 1.4, which was released this Friday. It will be interesting to watch these two IDS/IPS progress in the upcoming year. With two such development efforts, it was somewhat surprising to read the news release from the Open Information Security Foundation (OIS) that “OISF Receives Grant Funding for Open Source Next Generation IDS/IPS.” Good to see that innovation is occurring in an area of security previously given up for dead (see post, “IDS/IPS: The Mark Twain of the Security World“).

For cutting edge knowledge on Snort 3 development, Martin Roesch, Leon Ward, and Richard Bejtlich are three wise men who can provide greater insight. Fortunately, they have all written posts on Snort 3:

• Snort 3 Architecture Series Part 1: Overview by Martin Roesch
• Snort 3.0 Architecture Series Part 2: Changes and Betas by Martin Roesch
• Snort 3.0 Architecture Series Part 3: The command shell by Martin Roesch
• Snort 3 Beta on Ubuntu / Debian Installation by Leon Ward
• Using SnortSP and Snort 2.8.2 by Richard Bejtlich
• The power of Snort 3.0 by Richard Bejtlich

Overview

In this post, we will be installing Snort 2.8.3.1, the Snort Security Platform (SnortSP), and the Snort 3 analytical engine. Please see Martin Roesch’s, Leon Ward’s, and Richard Bejtlich’s posts for more in-depth discussion. I am going to discuss a few basic concepts of the Snort 3 architecture, go through installation, and discuss some configuration and operation of the software. I plan on following this post with a another concerning the setup and installation of Bro 1.4. I will follow that post with an analysis of results from the two systems. At some point, we will discuss integration of results into RTIR.

The Snort 3 architecture consists of the software framework, called the Snort Security Platform (SnortSP) and engines. Sometimes SnortSP will be referred to as “the framework” or just “framework.” SnortSP is designed to perform basically as an “operating system” for packet-based network security applications, providing common functionality that all programs need. For example, it operates on the data source performing such functions as acquiring the data (DAQ), decoding, flow normalization, IP defragmentation, and stream reassembly. SnortSP is composed of the action system, attribute management system (AMS), and the dispatcher. It allows you the ability to interact with the system through the command shell and snortd. From a developer’s point of view, SnortSP is what gathers data and handles any evasive techniques or other conditions that occur in suspicious and malicious traffic. SnortSP normalizes the data and then provides this cleaned up data to the engines for inspection. The engines are analysis modules that plug into SnortSP. Sometimes the engines will be referred to as “engine modules,” “analytics modules,” or simply the “analytics.” Multiple engines can run simultaneously. Here is a reference diagram.

Some of the major features of Snort 3 include:

• Shell-based user interface with embedded scripting language
• Native IPv6, MPLS and GRE support
• Native support for inline operation
• More subsystem plugin types such as data acquisition modules, decoders and traffic analyzers
• Multithreaded execution model – multiple analysis engines may operate simultaneously on the same traffic
• Performance increases

SnortSP comes with a Snort 2.8.2 detection engine implemented as a SnortSP engine module. Annother engine that is being developed is an network contextually aware engine that will allow Snort to understand what it is defending. Marty describes network context as, “essentially data about the environment that is being defended by Snort, the composition of the hosts in the network as well as the local network composition.” The idea is that no longer will you have to manually teach Snort rules. Marty reports “You teach Snort what the network looks like so it can defend itself accordingly. It tunes itself. My end goal is to have a self-tuning protection engine.” Richard provides insight into another upcoming engine, “SnortSP is expected to support a new engine called Policy Enforcement Point (PEP), a stateless (yes, not stateful) firewall that integrates with Sourcefire’s other products. Any engine running on the SnortSP will be able to call PEP to enforce access control decisions, assuming the sensor/IPS is in a place to take such actions.” It will be very interesting to see what engines are developed. This is the power of the design, giving Snort 3 the ability to grow to meet the needs of the security community.

Installation and Configuration

Generally, it is smart to start with what you know and determine expected results. We will begin with setting up and installing the latest stable version of Snort (Snort 2.8.3.1). Once it is configure and running, we will use some test data consisting of actual attacks that occurred in 2001. This allows us to develop a baseline of expected results. After that, we will move on to installing SnortSP, which comes with a implemented version of Snort 2.8.2 detection engine. First, you will need to have libpcre version 6.0 or greater installed.

PCRE (pcre-7.8)

The PCRE library is a set of functions that implement regular expression pattern matching using the same syntax and semantics as Perl 5. If you can install PCRE via a port specific to your operating system, that is the best way to install in order to avoid having to keep the software up-to-date. Below are the instructions for installing the software from source.

 root# cd /usr/local/src
 /usr/local/src root# wget \

http://downloads.sourceforge.net/pcre/pcre-7.8.tar.gz?modtime=1220617433&big_mirror=0

 /usr/local/src root# tar xzf pcre-7.8.tar.gz
 /usr/local/src root# cd pcre-7.8
 /usr/local/src/pcre-7.8 root# ./configure --prefix=/usr/local/pcre
 /usr/local/src/pcre-7.8 root# make
 /usr/local/src/pcre-7.8 root# make test
 /usr/local/src/pcre-7.8 root# make install

Libpcap and Large Files Support

Under some OSs, you need to compile libpcap and Snort to support large files (files large than 2G). Since the source code of libpcap will be needed for this configuration, we are going to install the resulting libpcap under /usr/local/snort. If the libpcap installed on your system does not produce an error, skip this step. You will want to follow these steps only if you get the following error when running Snort with a large file:

 root# ls -lh /data/ids/full2.pcap
-rw-r--r-- 1 root root 12G Oct 26 10:01 /data/ids/full2.pcap
 root# /usr/local/snort/bin/snort -o -A none -c /usr/local/snort/conf/snort.conf -l /logs/snort/logs \
-r /data/ids/full2.pcap
Error getting stat on pcap file: /data/ids/full2.pcap: Value too large for defined data type
RROR: Error getting pcaps
Fatal Error, Quitting..

First, we need to compile large file support into libpcap. As mentioned above, we will install the libraries under /usr/local/snort.

 root# cd /usr/local/src
 /usr/local/src root# wget http://www.tcpdump.org/release/libpcap-0.9.8.tar.gz
 /usr/local/src root# wget http://www.tcpdump.org/release/libpcap-0.9.8.tar.gz.sig
 /usr/local/src root# wget http://www.tcpdump.org/tcpdump-workers.asc
 /usr/local/src root# gpg --import tcpdump-workers.asc
gpg: key 89E917F3: "tcpdump.org (SIGNING KEY) " not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
 /usr/local/src root# gpg --verify libpcap-0.9.8.tar.gz.sig libpcap-0.9.8.tar.gz
gpg: Signature made Tue 25 Sep 2007 10:11:56 PM EDT using DSA key ID 89E917F3
gpg: Good signature from "tcpdump.org (SIGNING KEY) "
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 0227 54EB 4C30 9185 FD31  33A3 464D 3CEB 89E9 17F3
 /usr/local/src root# tar xzf libpcap-0.9.8.tar.gz
 /usr/local/src root# cd libpcap-0.9.8
 /usr/local/src/libpcap-0.9.8 root# ./configure --prefix=/usr/local/snort \
CFLAGS="-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE \
-D_FILE_OFFSET_BITS=64"
 /usr/local/src/libpcap-0.9.8 root# make
 /usr/local/src/libpcap-0.9.8 root# make shared
 /usr/local/src/libpcap-0.9.8 root# make install
 /usr/local/src/libpcap-0.9.8 root# make install-shared

In the next section, we will discuss how to install Snort with and without large file support.

Snort (snort-2.8.3.1)

The latest release of Snort, as of this writing, is Snort 2.8.3.1. Below we will get the software, verify, configure, and install the software under the /usr/local/snort area. Please adjust this to your environment. Reminder to Mac OS X and FreeBSD users, use the md5 command instead of md5sum.

 root# cd /usr/local/src
 /usr/local/src root# wget http://www.snort.org/dl/snort-2.8.3.1.tar.gz
 /usr/local/src root# wget http://www.snort.org/dl/snort-2.8.3.1.tar.gz.md5
 /usr/local/src root# cat snort-2.8.3.1.tar.gz.md5
f7c03b322dae31dafc45756630f2946c  snort-2.8.3.1.tar.gz
 /usr/local/src root# md5sum snort-2.8.3.1.tar.gz
f7c03b322dae31dafc45756630f2946c  snort-2.8.3.1.tar.gz
 /usr/local/src root# tar xzf snort-2.8.3.1.tar.gz
 /usr/local/src root# cd  snort-2.8.3.1

We are going to add in support to place alerts into a MySQL database. If MYSQL installed by the system, you can use the “–with-mysql” configuration option. In a previous post, “Introduction to MySQL,” we went through the installation of MySQL into the /usr/local/mysql directory. For such an installation, the –with-mysql-includes=/usr/local/mysql/include and –with-mysql-libraries=/usr/local/mysql/lib command options must be used. In order to use the dynamic plugin libraries, Snort needs to be able to find libmysqlclient.so. On some operating systems, you may have problems. Adding LDFLAGS=”-L/usr/local/mysql/lib/mysql” should work. If you do not need to compile in large file support, you can do the compilation simply with the command:

 /usr/local/src/snort-2.8.3.1 root# ./configure --prefix=/usr/local/snort \
--with-libpcre-includes=/usr/local/pcre/include \
--with-libpcre-libraries=/usr/local/pcre/lib \
--with-mysql-includes=/usr/local/mysql/include\
--with-mysql-libraries=/usr/local/mysql/lib

For adding in large file support, and having Snort use the libpcap installed under /usr/local/snort, do that with the following command:

 /usr/local/src/snort-2.8.3.1 root# CFLAGS="-D_LARGEFILE_SOURCE \
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64"  \
./configure --prefix=/usr/local/snort --with-libpcap-includes=/usr/localsnort/include \
--with-libpcap-libraries=/usr/local/snort/lib \
--with-libpcre-includes=/usr/local/pcre/include \
--with-libpcre-libraries=/usr/local/pcre/lib \
--with-mysql-includes=/usr/local/mysql/include\
--with-mysql-libraries=/usr/local/mysql/lib

After you configure Snort (with or without large file support), you continue to make and install it.

 /usr/local/src/snort-2.8.3.1 root# make
 /usr/local/src/snort-2.8.3.1 root# make check
 /usr/local/src/snort-2.8.3.1 root# make install
 /usr/local/src/snort-2.8.3.1 root# mkdir -p /usr/local/snort/etc
 /usr/local/src/snort-2.8.3.1 root# cp etc/* /usr/local/snort/etc
 /usr/local/src/snort-2.8.3.1 root# /usr/local/snort/bin/snort -V

   ,,_     -*> Snort! < *-
  o"  )~   Version 2.8.3.1 (Build 17)
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/team.html
           (C) Copyright 1998-2008 Sourcefire Inc., et al.
           Using PCRE version: 7.8 2008-09-05

Rules

Now we need some rules. For this example we will get the rules from the Snort and the Emerging Threats site. You will need to register for the rules at the Snort site. Do consider subscribing for the latest up-to-date rules. Registered users can only access rules 30 days after their release.

 root# cd /usr/local/src
 /usr/local/src root# wget http://www.emergingthreats.net/rules/emerging.rules.tar.gz
 /usr/local/src root# tar xzf emerging.rules.tar.gz

 /usr/local/src root# mkdir -p /usr/local/snort/rules
 /usr/local/src root# mv rules/* /usr/local/snort/rules
 /usr/local/src root# rmdir rules
 /usr/local/src root# wget \

http://www.snort.org/pub-bin/downloads.cgi/\

Download/vrt_os/snortrules-snapshot-CURRENT.tar.gz
 /usr/local/src root# md5sum snortrules-snapshot-CURRENT.tar.gz
184aed405da3f1043b82d81c98122237  snortrules-snapshot-CURRENT.tar.gz
 /usr/local/src root# mv  snortrules-snapshot-CURRENT.tar.gz /usr/local/snort/
 /usr/local/src root# cd /usr/local/snort/
 /usr/local/snort root# tar xzf snortrules-snapshot-CURRENT.tar.gz
 /usr/local/snort root# rm snortrules-snapshot-CURRENT.tar.gz
 /usr/local/snort root# vi /usr/local/snort/etc/snort.conf

Modify /usr/local/snort/etc/snort.conf to your environment. Make sure the RULE_PATH is set to /usr/local/snort/rules. If you wish to use the emerging threat rules, add:

include $RULE_PATH/emerging.conf

in the /usr/local/snort/etc/snort.conf file. Do not forget to adjust dynamicpreprocessor file and dynamicengine path. Mac OS X users will need to use the dynamic libraries. Uncomment the Mac OS X lines in the Snort configuration file.

Keeping Rules Current

If you want to setup a process to keep your rules as up-to-date as possible from the snort site, you will want to use the program Oinkmaster.

 root# cd /usr/local/src
 /usr/local/src root# wget http://prdownloads.sourceforge.net/oinkmaster/oinkmaster-2.0.tar.gz?download
 /usr/local/src root# md5sum oinkmaster-2.0.tar.gz
d2a1b56f51cf40e919c63206ca4ec8f8  oinkmaster-2.0.tar.gz
 /usr/local/src root# tar xzf oinkmaster-2.0.tar.gz
 /usr/local/src root# cd oinkmaster-2.0
 /usr/local/src/oinkmaster-2.0 root# cp oinkmaster.pl /usr/local/snort/bin
 /usr/local/src/oinkmaster-2.0 root# cp oinkmaster.conf /usr/local/snort/etc

Modify /usr/local/snort/etc/oinkmaster.conf changing “url=” to the location where you want rules to be archived. The URL will also require a Oink Code, which is available once you register with Snort and log into your account. Multiple URLs can be specified for multiple files. You can have Oinkmaster pull down rules from Snort and Emerging Threats. I use oinkmaster to pull down the latest rules, place them in new rules directory (ex: /usr/local/snort/new-rules), backup the previously rules, and send email about what is new. Some rules may have negative impact on Snort’s operation. It is unwise to automatically update the rules without review. Below, the backup/archive area is the directory /usr/local/snort/archive. Modify /usr/local/snort/bin/oinkmaster.pl to know where oinkmaster.conf is located. At that point you are ready to use oinkmaster to update your rules.

 /usr/local/src/oinkmaster-2.0 root# mkdir /usr/local/snort/archive
 /usr/local/src/oinkmaster-2.0 root# vi /usr/local/snort/bin/oinkmaster.pl
 /usr/local/src/oinkmaster-2.0 root# vi /usr/local/snort/etc/oinkmaster.conf
 /usr/local/src/oinkmaster-2.0 root# /usr/local/snort/bin/oinkmaster.pl \
-o /usr/local/snortr/new-rules -b /usr/local/snort/archive
 /usr/local/src/oinkmaster-2.0 root# crontab -l
10 6 * * * /usr/local/snort/bin/oinkmaster.pl -o /usr/local/snort/new-rules -b /usr/local/snort/archive \
| Mail -s "Snort Rule Updates" abbot@securitymonks.com 2>&1

For automatic updates, set up a cron job that runs the above command and email the results.

Testing Snort Using Attack Data

Leon Ward has made available a pcap file containing attacks that occurred back in 2001 against a honeypot. If you have other data that will produce interesting results, please feel free to use that.

 root# mkdir -p /data/ids/tcpdump
 root# cd /data/ids/tcpdump
 /data/ids/tcpdump root# wget rm-rf.co.uk/downloads/Honeynet-RFP-iis.tgz
 /data/ids/tcpdump root# tar xzf Honeynet-RFP-iis.tgz
 /data/ids/tcpdump root# /usr/local/snort/bin/snort -c /usr/local/snort/etc/snort.conf \
-A fast -l /data/ids/tcpdump -r ./Honeynet-RFP-iis.pcap
 /data/ids/tcpdump root# ls /data/ids/tcpdump/alert /data/ids/tcpdump/snort.log.*

Two results files should get created. The file /data/ids/tcpdump/alert will contain the alerts and /data/ids/tcpdump/snort.log.<date>, which contains the pcaps of the detected events.

SnortSP

At this point we are ready to install SnortSP. The first step is to make sure the system has the required software installed, which includes:

• Libdnet 1.10 or higher
• A recent Libpcap
• Lua 5.1.1 or better
• Depending o the OS, a UUID library

The second step involves getting the software, verifying, configuring, and installing the security platform. The engine will be built in the next section.

 root# cd /usr/local/src
 /usr/local/src root# wget http://www.snort.org/dl/prerelease/3.0.0-b2/\
snortsp-3.0.0b2.tar.gz
 /usr/local/src root# wget http://www.snort.org/dl/prerelease/3.0.0-b2/\
snortsp-3.0.0b2.tar.gz.md5
 /usr/local/src root# wget http://www.snort.org/dl/prerelease/3.0.0-b2/\
snortsp-3.0.0b2.tar.gz.sig
 /usr/local/src root# cat snortsp-3.0.0b2.tar.gz.md5
4b2259c08ebe66cf63d91359996c93d9  snortsp-3.0.0b2.tar.gz
 /usr/local/src root# md5sum snortsp-3.0.0b2.tar.gz
4b2259c08ebe66cf63d91359996c93d9  snortsp-3.0.0b2.tar.gz
 /usr/local/src root# gpg --verify snortsp-3.0.0b2.tar.gz.sig snortsp-3.0.0b2.tar.gz
gpg: Signature made Fri 18 Jul 2008 11:09:25 AM EDT using DSA key ID B10683B0
 /usr/local/src root# gpg --keyserver pgpkeys.mit.edu --recv-key B10683B0
gpg: requesting key B10683B0 from hkp server pgpkeys.mit.edu
gpg: key B10683B0: "Snort Release Team (Snort Release Team signing key) " not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
 /usr/local/src root# gpg --verify snortsp-3.0.0b2.tar.gz.sig snortsp-3.0.0b2.tar.gz
gpg: Signature made Fri 18 Jul 2008 11:09:25 AM EDT using DSA key ID B10683B0
gpg: Good signature from "Snort Release Team (Snort Release Team signing key) "
gpg: Note: This key has expired!

Primary key fingerprint: 0A0A BD0C 8FAA BDDC E7A2  A8C3 E6FA 8BA1 B106 83B0

The next step is to configure, install, and test SnortSP installation. We will be using the /usr/local/snort area as the installation directory. Please adjust to your environment.

 /usr/local/src root# tar xzf snortsp-3.0.0b2.tar.gz
 /usr/local/src root# cd  snortsp-3.0.0b2
 /usr/local/src/snortsp-3.0.0b2 root# ./configure --prefix=/usr/local/snort
 /usr/local/src/snortsp-3.0.0b2 root# make
 /usr/local/src/snortsp-3.0.0b2 root# make check
 /usr/local/src/snortsp-3.0.0b2 root# make install
 /usr/local/src/snortsp-3.0.0b2 root# mkdir -p /usr/local/snort/etc/SnortSP
 /usr/local/src/snortsp-3.0.0b2 root# cp etc/* /usr/local/snort/etc/SnortSP
 /usr/local/src/snortsp-3.0.0b2 root# /usr/local/snort/bin/snortsp -V
SnortSP Version 3.0.0b2

Snort 2.8.2 Detection Engine

The next step is to install the Snort 2.8.2 detection engine. We will immediately shut it down with the ssp.shutdown() command. Please see previous sections “Libpcap and Large Files Support” and “PCRE (pcre-7.8).” I am going to include the configuration options for libpcap, pcre, and MySQL. Please adjust the configuration if necessary in the same way snort-2.8.3.1 was modified.

 /usr/local/src/snortsp-3.0.0b2 root# cd src/analysis/snort
 /usr/local/src/snortsp-3.0.0b2/src/analysis/snort root# ./configure --prefix=/usr/local/snort \
 --with-platform-libraries=/usr/local/snort/lib/snortsp \
 --with-platform-includes=/usr/local/snort/include \
 --with-libpcap-includes=/usr/localsnort/include \
 --with-libpcap-libraries=/usr/local/snort/lib \
 --with-libpcre-includes=/usr/local/pcre/include \
 --with-libpcre-libraries=/usr/local/pcre/lib \
 --with-mysql-includes=/usr/local/mysql/include\
 --with-mysql-libraries=/usr/local/mysql/lib
 /usr/local/src/snortsp-3.0.0b2/src/analysis/snort root# make
 /usr/local/src/snortsp-3.0.0b2/src/analysis/snort root# make check
 /usr/local/src/snortsp-3.0.0b2/src/analysis/snort root# make install
 /usr/local/src/snortsp-3.0.0b2 root# /usr/local/snort/bin/snortsp \
-L /usr/local/snort/etc/SnortSP/snort.lua
   ,,_     -*> SnortSP! < *-
  o"  )~   Version 3.0.0b2 (Build 9) [BETA]
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/team.html
           (C) Copyright 2008 Sourcefire Inc.
> Control thread running - 3086609296 (9501)
> ssp.shutdown()
 /usr/local/src/snortsp-3.0.0b2 root#

The command “snortsp -L /usr/local/snort/etc/SnortSP/snort.lua” is telling snortsp to use the file “snort.lua“. Examine the file for definitions of functions. Below are a few example functions calls that may be of interest:

 root# /usr/local/snort/bin/snortsp -L /usr/local/snort/etc/SnortSP/snort.lua
snort> ssp.help()
[*] SnortSP Commands:
    help()
    set_log_level( [debug|info|notice|warn|error|critical] )
    shutdown()
  Available subsystems within SnortSP have their own help() methods:
    dsrc        - Data Source
    eng         - Dispatcher/Engine
    analyzer    - Analytics Modules
    output      - Output Modules
  For example: dsrc.help() will call the Data Source help function
snort> dsrc.help()
[*]  Data Source Config Structure
 = {name        = ,
                  type        = ,
                  intf        = ,
                  flags       = [1|2|8],
                  command     = ,
                  snaplen     = ,
                  maxflows    = ,
                  maxidle     = ,
                  flow_memcap = ,
                  filename    = ,
                  max_count   = ,
                  display     = ,
                  gtp_support = [enable]
                  defrag      = {
                                 policy = <>,
                                 max_trackers = <>,
                                 timeout = <>,
                                 memcap = <>
                                }
}

    name: User string used to refernce the instantiated data
          source object.  If no name is provided a UUID is
          generated.
    type: Name of the DAQ module to use.  The list of available
          DAQ modules can be obtained using the
          dsrc.list_daq_modules() command.
    intf: Name of the interface to use for traffic acquisition.
    flags: Bitwise OR of available mode flags for a given DAQ.
           Some modes may not be available on all DAQs.
           Generally you will want to use either mode 1, 2
           or 8.  Default is 0.  To enable inline mode use a
           value of 10.
           Available modes are:
                1 = FILE READ MODE
                2 = PROMISC MODE
                8 = INLINE MODE
    command: BPF filter or other filter command for DAQs that
             support them.  Default is no filtering.
    snaplen: Max bytes to capture per packet.  Default is 0.
    maxflows: Max number of flows to keep in the flow manager
              at one time.  If this number is exceeded the least
              recently used flow is purged.  Default is 8192.
    maxidle: Number of seconds a flow can be idle before being
             timed-out and removed from the flow table.  Default is
             60.
    flow_memcap: Max number of bytes to allow to be used by the
                 flow table.  Default is 16MB.
    filename: In file playback mode, specifies the name of the
              file to run through the SnortSP instance.
    max_count: Number of packets to process before stopping this
               thread.  Default is 0, which is unlimited.
    display: Name of the display mode for printing packets as they
             are processed.  Available modes are:
                none    = Don't print packets at runtime.
                basic   = Display basic packet info on one line.
                classic = Display packets using the classic tcpdump
                          format.
                plus    = Classic mode + MAC header printout.
                max     = Full printout of every packet header
                          field, one field per line.
             Default is none.
    gtp_support: Activate support for the GTP protocol.  Default is
                 disabled.
    defrag: Specifies the IP/IPv6 defragmentation paramters.
                policy: Reassmebly policy for overlapping fragments.
                    Avaliable modes are:
                        
                    Default is bsd.
                max_trackers: The maximum number of unique packets
                              in the process of being reassembled at a
                              at any given time.  Default is 8192.
                timeout: The maximum time in seconds for a fragmented
                         packet to receive all fragments.  Default is 60.
                memcap: Maximum number of bytes to allow to be used for
                        fragment reassembly.  Default is 32MB.
[*] Data Source Commands
    new(  ) - Instantiate a new data source object
    delete(  ) - Delete a data source object
    list() - List all of the instantiated data source objects
    list_daq_modules() - List all of the available DAQ modules
    show( source_name ) - Show the config of the named
                          object
    help() - Print this message

  For example, the following commands will create a data source
  using the afpacket DAQ in inline mode bridging the eth2 and
  eth3 interfaces allowing 262144 flows to occupy 10 MBs of
  memory while timing out flows after 5 minutes of idle time:
      dsrc1 = {name="src",
               type="afpacket",
               intf="eth2:eth3",
               flags=10,
               snaplen=0,
               maxflows=262144,
               maxidle=300,
               flow_memcap=10000000}
      dsrc.new(dsrc1)

snort> dsrc.list()
[*] 0 data sources configured
snort> sniff("eth0")
Creating new data source
Flow manager created with 16384 flow capacity
Engine "e1" created
Linking engine "e1" to data source "src1"
Calling engine_start()

init_pcap: Initializing network interface eth0
[*] Data Source Config:
        Name: src1
        Type: pcap
        Interface: eth0
        Filename:
        Snaplen: 1514
        Flags: 0x00000002
        Display: None (0)
        Filter command:
        DAQ: 0x8078b00
        User Context: 0x9378170
        Max flows: 16384
        Max idle: 60
        Memcap: 10000000
[*] Flow Manager Config:
        Max flows: 16384
        Max idle: 60
        Memcap: 10000000
[*] DAQ config:
   Interface: eth0
   Snaplen: 1514
   Datalink: 1
   Count: 0
   Packet Count: 0
   Promisc flag: 1
   File flag: 0
   pcap ptr: 0x9396488
   analysis context ptr: 0xb71ce008
[*] Spawning engine thread!
snort> e1 thread running - 3072121744 (24685)

snort> dsrc.list()
[*] 1 data sources configured
Name: src1    DAQ: pcap    interface: eth0    Running
snort> dsrc.show("src1")
[*] Data Source Config:
        Name: src1
        Type: pcap
        Interface: eth0
        Filename:
        Snaplen: 1514
        Flags: 0x00000002
        Display: None (0)
        Filter command:
        DAQ: 0x8078b00
        User Context: 0x9378170
        Max flows: 16384
        Max idle: 60
        Memcap: 10000000
[*] Flow Manager Config:
        Max flows: 16384
        Max idle: 60
        Memcap: 10000000
[*] DAQ config:
   Interface: eth0
   Snaplen: 1514
   Datalink: 1
   Count: 0
   Packet Count: 117
   Promisc flag: 1
   File flag: 0
   pcap ptr: 0x9396488
   analysis context ptr: 0xb71ce008
snort> fsniff ("eth0", "port 21")
snort> dsrc.list()
[*] 2 data sources configured
Name: src2    DAQ: pcap    interface: eth0    Running
Name: src1    DAQ: pcap    interface: eth0    Running
snort> ssp.shutdown()
 root#

Lua

Lua is a lightweight scripting language embedded in SnortSP. The Snort Detection Engine is configured via a conf file, but SnortSP is configured by a Lua file. The Snort conf file location is specified in the Lua file. In this section, we are going to be running the shell script sspiffy.sh. The program comes with the SnortSP package and it will convert Snort 2.8.x files into a Lua scripts and a new configuration file for SnortSP.

Previously, we ran snort 2.8.3.1 against the Honeynet-RFP-iis.pcap data with the command:

 root# cd /data/ids/tcpdump
 /data/ids/tcpdump root# /usr/local/snort/bin/snort -c /usr/local/snort/etc/snort.conf \
-A fast -l /data/ids/tcpdump -r ./Honeynet-RFP-iis.pcap

We are going to create and use /data/ids/logs where result log files will be kept. First, backup the snort configuration file and result files previously created with snort 2.3.8.1.

 root# cd /usr/local/snort/etc
 /usr/local/snort/etc root# cp snort.conf snort.conf.orig
 /usr/local/snort/etc root# mkdir -p /data/ids/logs
 /usr/local/snort/etc root# mv /data/ids/tcpdump/alert /data/ids/logs/alert.orig
 /usr/local/snort/etc root# mv /data/ids/tcpdump/snort.log.* /data/ids/logs/

We defined dynamicpreprocessor and dynamicengine in snort configuration file (/usr/local/snort/etc/snort.conf) file, so we do not need to specify their location via command line with –dynamic-preprocessor-lib-dir and and –dynamic-engine-lib. We will use the /usr/local/snort/etc as our configuration directory. The script sspiffy.sh will create a new snort.conf file. To generate a Lua file to run SnortSP with snort, we would use sspiffy.sh as follows:

 root# cd /usr/local/snort/etc
 /usr/local/snort/etc root#  /usr/local/src/snortsp-3.0.0b2/sspiffy.sh \
/usr/local/snort -c /usr/local/snort/etc/snort.conf.orig \
-r /data/ids/tcpdump/Honeynet-RFP-iis.pcap -l /data/ids/logs -A fast 

Checking command line arguments ...

snort.conf is a copy of /usr/local/snort/etc/snort.conf.orig.2 with the following changes:
#SSP - set in lua: dynamicpreprocessor directory /usr/local/snort/lib/snort_dynamicpreprocessor/
#SSP - set in lua: dynamicengine /usr/local/snort/lib/snort_dynamicengine/libsf_engine.so
#SSP - deleted: preprocessor frag3_global: max_frags 65536
#SSP - changed: preprocessor frag3_engine: policy first detect_anomalies

snort.lua is the script that configures and executes SSP.

To run SSP as configured:
/usr/local/snort/bin/snortsp -L snort.lua -P
 -S  

 /usr/local/snort/etc root#  ls snort.lua snort.conf
 /usr/local/snort/etc root#  vi snort.lua snort.conf

The two files that were created in your current directory are snort.lua and snort.conf. Take a look at the files and modify them where appropriate. Make sure the full path to snort.conf is specified in snort.lua (conf=”/usr/local/snort/etc/snort.conf”). To run snortsp with the generated files, use the command:

 /usr/local/snort/etc root#  /usr/local/snort/bin/snortsp -L /usr/local/snort/etc/snort.lua

Now go to the /data/ids/logs area and compare the alert file just generated with the alert.orig file generated by snort 2.8.3.1. The results do match. The differences between the files comes down to SnortSP alert file not including log entries for http_inspect.

Final Thoughts

The new Snort architecture is a very exciting design change that will help future versions of Snort respond quickly to an every changing security battleground. Besides the potential speed performance, along with the ability to take advantage of the multicore systems being developed, what I find particularly interesting is that the new architecture provides a framework where developers can build their applications. There is real potential here for intriguing development and flexibility. Different environments have different requirements and needs. Soon organizations will have the ability to choose from different engines resulting in custom configurations that best address each organization’s risks.

Everyone is looking for a turn key solution. People need to accept the fact that security is hard. By the time you find a solution, the problem has changed. With an ever changing attack vector, there are no turn key solutions that can do anything more than provide help. This is an important concept, because once accepting this fact, it changes a person’s view on viable solutions. That is what I like about the Snort 3 architecture. It is designed to be capable of being adjusted rapidly, which is a really cool concept. All security tools should be designed in such a manner. Our adversaries adjust quickly. We need to be able to also. On the eve of Snort turning ten, it is good to see that the Sourcefire team has been building such a promising IDS that will serve us well far into the future.

One needs to be careful not to fall into the trap of pretending one can make security problems disappear by simply passing laws. Some countries seem to be trying to do just this by outlawing such things as security tools (“UK government to consider hacker tool ban“, “Germany outlaws ‘hacking tools’: An impossible ban for sysadmins?“). This only results in security professionals being deprived of the very tools they need to do their jobs. In Europe, while laws get passed protecting the privacy of European citizens, the European governments ends up being exempt. “Europeans reserve their deepest distrust for corporations, while Americans are far more concerned about their government invading their privacy,” writes Bob Sullivan in his article, “‘La difference’ is stark in EU, U.S. privacy laws.” In the end, monitoring still occurs. It is just a question of who does the monitoring.

Others might not want to make security tools illegal, but instead limited the tools to those demonstrating a certain level of professional proficiency. Deb Radcliff writes in the article, “Computer Forensics Faces Private Eye Competition,” about pending legislation in South Carolina where digital forensic evidence gathered for use in a court must be collected by a person with a PI license or through a PI licensed agency. Deb writes, “Georgia, New York, Nevada, North Carolina, Texas, Virginia and Washington are some of the states going after digital forensic experts operating in their states without a PI license.” The article goes on to quote Steve Abrams, a licensed independent PI and computer forensic examiner based in Sullivans Island, S.C., “In April [2007], the state attorney general opined that even if you never set foot in South Carolina, if you’re collecting evidence to be used in court here, you still need a South Carolina [PI] license. Licensing authorities in New York, Pennsylvania, Texas and Oregon have opined the same way.”

I can’t help but think about a recent newspaper article titled “Robbery target isn’t only one who’s packing” written by Beth Brelje. While it is not directly about IT, please bear with me. Richard Flynn, owner of American Sport Shooting, made this great statement:

A lot of people who move from metropolitan areas are not use to not having a police force. I moved here in 1990, I could walk around the streets and never considered being armed. Now I will not consider going out without being armed. Anytime people move in, you get good and bad. Unfortunately we’ve got a good amount of bad. I don’t feel safe here. I considered it being prepared.

Once upon a time, I felt safe on the Internet and did not think about security. Then the world moved in and we all became just a few milliseconds away from every creep on the planet. Do we have an Internet police force keeping us safe? Even if the non existing cyber police really wanted to hear from us when our computers becomes infected with viruses, what could they do? The 165,000 men, women, and children of Monroe County would love it if everyone could just get along. Sadly, there are some bad folks who would love to cause harm in the county. The residents also would not object to a large police force that could deal with these criminals before any crime occurred. In the end, economically that is not a viable solution. So, 10,000 Monroe County residents, feel the need for some protection. I understand.

H.P. Lovecraft wrote in “Supernatural Horror in Literature” that “the oldest and strongest emotion of mankind is fear, and the oldest and strongest kind of fear is fear of the unknown.” This is a lesson repeated throughout history. Marie Curie, ever the scientist, wrote, “Nothing in life is to be feared. It is only to be understood.” Of course Marie Curie died of aplastic anemia. She fell victim of radiation from the many fascinating glowing substances she had learned to isolate. How could she have known? Understanding is not only important in overcoming fear, but it can be essential sometimes for life itself. Ignorance is often deadly. While the government works on writing laws that will eliminate Internet insecurity, it would be wise to keep open the option of dealing with these problems ourselves. The first step is to open our eyes and see the wolf. This is where monitoring comes into play. Know what is going on. Only then can we start working on a solution. Later, if the government can make our lives easier with another layer of protection, so much the better.

Samuel Langhorne Clemens, better known as Mark Twain, once wrote, “Rumors of my death have been greatly exaggerated!” It seems worldwide IDS/IPS revenue grew 19% in 2006. In 2007, the IDS/IPS marked was a $932 million industry. Frost & Sullivan expects the market to grow to $2.1 billion in the next five years, citing “complex attacks, ongoing vulnerability discoveries, and the need for companies to comply with new legislation” as major contributing drivers. Of course these numbers are focused on the commercial side of IDS/IPS.

“Computers are like Old Testament gods; lots of rules and no mercy,” wrote Joseph Campbell, an American mythology professor, writer, and lecturer. I often think of this quote when it comes to the use of signature based solutions in security. As I previously posted, IDS/IPS technology is moving aware from being solely signature based, to a blend of signature based, anomaly detection, and activity based methodologies. The Race to Zero contest is being held right now (August 8-10th) during Defcon 16. It is a hacking competition where known viruses will be tweaked in an attempt to foil signature-based blacklists of several major antivirus engine. The point is to demonstrate how easy it is to get around signature based solutions.

Liam Tung, over on ZDnet has written a very good article titled, “Signature-based antivirus is dead: Get over it“. In the article, Simon Clausen, founder & CEO at PC Tools, reports that the security industry has been looking beyond blacklists. “I would very much disagree that AV is dead. Really, traditional signature-based AV is going to be dead in a few years, but what every antivirus company is evolving towards, like us, is behavioural AV technology, so AV will be alive.”

Martin Roesch, CTO and Founder of Sourcefire, has posted concerning the upcoming Snort 3 architecture that has as a key component a contextually aware engine. This will add to Snort the ability to understand what it is defending. Snort 3 architecture is built around the concept of network context, which is essentially data about the environment and the composition of the hosts in the network as well as the local network composition. The software framework is called SnortSP (the Snort Security Platform) and the initial beta has been released. By leveraging network context, Roesch hopes to reduce/simplify/eliminate tuning as much as possible, be able to generate event priorities, and address network and transport layer evasion.

Peter Judge, in his ZDnet article “Sourcefire: Don’t Snort at open-source security,” quotes Roesch talking on the future of the security market:

Threat management has three phases. Before the threat, the firewall and patch management should prevent threats; during the attack, the IPS should block them; and, afterwards, network-behaviour analysis [NBA] should reveal damage and remedy it. These tend to be stove-piped technologies, where nobody talks to anyone. There are not enough people. You have got to get people out of the equation. You have to automate.

The US government, through the Department of Homeland Security, has issued a Request for Information (RFI) which highlights the analytical skills that DHS is seeking from a staffing perspective. Through this RFI, one can determine the technological focus DHS has for the administration’s requested $294 million fiscal cybersecurity budget. The 2009 cybersecurity budget is a significant increase over the 2008 enacted budget of $210 million. Ben Bain, in his article “DHS seeks cybersecurity capability info” quotes Alan Paller, research director at the SANS Institute, as saying that most DHS department employees don’t know how to do complex intrusion detections, log analysis or reverse engineering malware. DHS is looking for folks with experience with EINSTEIN data analysis, tools, techniques and network flow analysis capabilities, the TIC deployment environment, and compliance metrics. Ben Bain reports on the highly classified Comprehensive National Cybersecurity Initiative (CNCI) that DHS is “planning on implementing a new version of the intrusion detection and alert system — EINSTEIN 2 — designed monitor agencies’ Internet access points for malicious activity and capture intrusion data along with data transmitted in proximity to an alert.” It sounds like the government believes there is still life in some form of intrusion detection technology.

While I have mentioned Snort, it is not the only IDS/IPS. Keeping to the world of open source, most of my IDS/IPS time is split between Snort and Bro. I am very interested in Roesch’s work on the contextually aware engine. It will make Snort a more powerful tool. Now Bro has advance features giving it the ability to discern network anomalies that are caused by hostile activity. Bro also has some ability to detect violations of expected traffic rules to defend against previously-unknown attack techniques. For additional information on Bro, there are three blogs particularly helpful. Seth Hall of The Ohio State University has started the “A Bro Blog.” The ICSI Networking Group has a blog with contributions from the big names in Bro: Mark Allman, Sally Floyd, Christian Kreibich, Vern Paxson, Robin Sommer, and Nicholas Weaver. C.S.Lee (geek00L) on his site “When {Puffy} Meets ^RedDevil^” will frequently do postings involving Bro.

I started this post asking if Gartner was correct and are IDS/IPS a dying technology? By looking at both the IDS/IPS market and the government’s cyber security focus, we see the technology is very much alive. Not so much much because Gartner was wrong about IDS/IPS producing false positives and negatives, that monitoring puts a burden on the organization, that incident response can be a taxing process, nor that being able to monitor an increasingly higher bandwidth is challenging. Those problems still exist. Because of that, there is much work being done to help the IDS/IPS industry handle data more flexibly, efficiently, or accurately. Gartner was wrong with the assumption that the technology would remain stagnant and if something is difficult, it will not be implemented. IDS/IPS exist because organizations need the detection and prevention capabilities. Companies face an ever increasing reliance on information technology for competitive advantage while dealing with increasingly complex attacks, ongoing vulnerability discoveries, and the need for companies to comply with new legislation. IDS/IPS solutions continue to be part of organizations’ security programs because the technology continues to evolve while integrating new solutions. In the world of evolution, adaptability will trump better design if the better design is incomplete and inflexible in an ever changing environment.

Last week I spent Monday driving through a few states. It was an eight hour drive. When possible, I prefer driving over flying. While it may take longer, I use the time to listen to podcasts. Since I had taken the SANS System Forensics, Investigation & Response course (SEC 508), I had access to their lectures in MP3 format. The lecture on Computer Investigative Law for Forensic Analysts was prepared and taught by Richard P. Salgado. I had taken the course at a Community SANS event, close to where my brother lives. Yes, I was trying to keep my expenses down, and my brother and his family were kind enough to put me up for the week. While the course was well taught, knowledge of the legal issues of forensics was not the instructors strong point. This was reflected by the fact that the students hated that day. If only they had Richard P. Salgado. He did an amazing job.

Why am I mentioning this on a blog posting on intrusion detection systems (IDS)? The law has an ever increasing role in IT. This is especially true in the area of forensics, incident response, and intrusion detection/prevention. Before you setup any IDS system, make sure you are authorized and legally clear to do so.

With that disclaimer out of the way, I spent the weekend beginning to develop a network monitoring system. Sure, for years I have worked with Snort, but I am doing something different. For those unfamiliar with Snort, to quote their site:

Snort® is an open source network intrusion prevention and detection system utilizing a rule-driven language, which combines the benefits of signature, protocol and anomaly based inspection methods. With millions of downloads to date, Snort is the most widely deployed intrusion detection and prevention technology worldwide and has become the de facto standard for the industry.

It is a great product. Along with Snort, I have used the Basic Analysis and Security Engine (BASE), which is based on the Analysis Console for Intrusion Databases (ACID) project. BASE provides a web front-end to query and analyze the alerts coming from a SNORT IDS system. If you are pulling down software, I also suggest checking out Sguil.

Richard Bejtlich recently posted on his blog, TaoSecurity, an entry titled “DHS Einstein Demonstrates Value of Session Data.” Richard makes the statement, in relation to collecting session data:

This is just the sort of project I’d like to roll out at my new job, possibly combining Argus with ArgusEye, or maybe just Sguil without Snort.

An intriguing project. This weekend was about setting up an IDS system using Bro. To understand the importance of Bro, you need to first review the different styles of intrusion detection.

• Signature Based – looks for specific, known attacks.
  • Pros: good attack libraries, easy to understand results.
  • Cons: unable to detect new attacks or even just variants.
• Anomaly Detection – build/infer a profile of “normal use” and flag deviations.
  • Pros: potentially detects wide rand of attacks, including previously unknown types of attacks.
  • Cons: can be “trained” to accept attacks as normal, and potentially misses a wide rand of attacks including known attacks.
• Activity Based – inspect traffic and construct “events,” look for patterns of activity that deviate from a site policy.
  • Pros: potentially detects wide range of attacks (including novel), framework can accommodate signatures and anomalies.
  • Cons: policies/specification require significant development and maintenance and harder to construct attack libraries

Snort is a signature based IDS. Bro is an activity based IDS, though it does include a signature engine for matching specific patterns in packet streams. Bro is compatible with Snort. somewhat. With Bro analysis, signature matches generate events which are amenable to high level policy script processing rather than direct alerts. Other difference include that Snort is user friendly and Bro is a beast to learn. Worse still, there are no good guides for Bro. Sure, you can subscribe to the mailing list and there is a Bro Wiki. Geek00l has done some very good postings:

• Regex – Magic for NetSe[x|c]Anal(yst)?
• Bro-IDS: Enable Full Content Data Logging
• Time Machine – Payload Centric
• Bro-IDS v1.2
• Bro-IDS – Signature Matching
• FreeBSD – IDS Sensor Tweaking
• Bro-IDS – The learning process
• Multipurposes post :]
• Bro-IDS – Be Loved
• Bro-IDS – Installation Experience

Geek00l convinced Richard Bejtlich take a second look at Bro, and Richard posted:

• Bro Basics Follow-Up
• Bro Basics

That will get you started.

My interest in Bro comes from the fact that a design goal of Bro was to handle high speed, large volume monitoring. Snort, on a security appliance, can handle such traffic. Force10 released such a box, the P10, which can handle up to 1000 signatures. I have worked with the open source version of Snort on high volume networks, and it has not been pleasant. While the P10 might work well, I am interested in different capabilities.

Bro offers an interesting solution to handling monitoring on 10G traffic. If you are working with FreeBSD, there are ways to tune the kernel. While I have previously run into problems with Bro, my past problems were more likely due to trying to work under the Apple environment. Supported 10G Ethernet cards drivers had not yet been developed. Fortunately, that appears to have changed. I’ll post more as I make progress.