The Suricata Engine and the HTP Library are available to use under the GPLv2. The new engine supports “Multi-Threading, Automatic Protocol Detection (IP, TCP, UDP, ICMP, HTTP, TLS, FTP and SMB! ), Gzip Decompression, Fast IP Matching and coming soon hardware acceleration on CUDA and OpenCL GPU cards”. GPU integration allows the use of graphic cards to accelerate operations. Mike Cloppert in his post, “Detection, Bandwidth, and Moore’s Law” pointed out:

It appears the authors well understand the point in this post, and the corresponding state of the art in solving parallel computing problems. GPU’s are emerging as a good commodity solution to parallel processing. This is covered in depth by a number of recent publications discussing parallelism, and I am by no means an expert in this field, so I will simply leave follow-up on this point as an exercise for the reader.

The HTP Library is an HTTP normalizer and parser written by Ivan Ristic, creator of Mod Security and author of the soon to be released book “ModSecurity Handbook“. This integrates and provides very advanced processing of HTTP streams. The HTP library is required by the engine, but may also be used independently in a range of applications and tools. Additional details have been provided by Ivan in his post, “HTTP parser for intrusion detection and web application firewalls.” Ivan writes concerning the development, “For the first release of the parser the goal is to be able to parse HTTP streams reliably. In the subsequent versions I will work in the parser’s security properties (such as the ability to see through evasion attacks).”

New Ideas and Concepts

Quoting from the OISF announcement, some of the next generation capabilities include:

• Multi-Threading: so very necessary.
• Automatic Protocol Detection: the engine has keywords for IP, TCP, UDP, ICMP, HTTP, TLS, FTP and SMB. Users can write rules to detect a match within a stream regardless of the port the stream occurs on. This is important for malware detection and control. Detections for more layer 7 protocols are being developed.
• Gzip Decompression: the HTP Parser will decode Gzip compressed streams.
• Independent HTP Library: the HTP Parser will be usable by other applications such as proxies, filters, etc. The parser is available as a library under GPLv2 for easy integration ito other tools.
• Standard Input Methods: support for NFQueue, IPFRing, and the standard LibPcap to capture traffic. IPFW support will be available soon.
• Unified2 Output: support for standard output tools and methods.
• Flow Variables: it is possible to capture information out of a stream and save that in a variable which can then be matched against later.
• Fast IP Matching: the engine will automatically use a special fast matching preprocessor on rules that are IP matches only (such as the RBN and compromised IP lists at Emerging Threats).
• HTTP Log Module : HTTP requests can be automatically output into an apache-style log format file for monitoring and logging activity completely independent of rulesets and matching.

A few features to look forward to in a few weeks:

• Global Flow Variables: the ability to store more information from a stream or match (actual data, not just setting a bit) over a period of time allowing comparing values across many streams and time.
• Graphics Card Acceleration: using CUDA and OpenCL to make use of the processing power of even old graphics cards to accelerate the IDS. Offloading the very computationally intensive functions of the sensor will greatly enhance performance.
• IP Reputation: will allow sensors and organizations to share intelligence and eliminate many false positives.
• Windows Binaries: will be released once there is a reasonably stable body of code.

Folks Behind It

The team is listed on the OISF site. It is an all star cast including Matt Jonkman, Victor Julien, Will Metcalf, Nathan Jimerson, Margaret Skinner, Josh Smith, Brian Rectanus, Breno Silva Pinto, Anoop Saldanha, Gurvinder Singh Dahiya, Jason MacLulich, Jason Ish, Kirby Kuehl, Dennis Henderson, Martin Solum, Ivan Ristic, Pablo Rincon, and Gerardo Iglesias Galvan.

I also wanted to point out some of the heavy hitting organizations involved. The initial funding for OISF comes from the US Department of Homeland Security (DHS), the US Navy’s Space and Warfare Command (SPAWAR), and a number of private companies that participate in the OISF Consortium. The OISF is a part of the DHS Homeland Open Security Technology (HOST) program. OISF works with Open Source Software Institute and has received legal guidance from the Software Freedom Law Center.

OISF is a US nonprofit, a 501c(3) and will not commercialize, sell, patent, copyright, or profit from the engine. OISF Consortium members are donating coders, equipment, and financial support in exchange for the ability to commercialize the engine. The important take away is that OISF has long term support for future development of Suricata.

Final Thoughts

Suricata is a very exciting and promising IDS/IPS engine. It has a great group of people behind it and future development appears secured. It is a project that is in the early stages. Do not expect to download it and simply install on a production environment. For testing the software and providing feedback, the engine and the HTP Library are available for download. To keep apprised of the latest developments join the oisf mailing lists where you discuss and share feedback. The blog of Victor Julien, Suricata’s lead developer, is another great source for the latest news and information.

To finally answer the burning question: why the name Suricata? According to the OISF site, Suricata comes from the Latin genus name for the meerkat and “the Meerkat takes security and vigilance as a life or death responsibility. There is always at least one individual on guard, watching, ready to alert the entire organization. Very much like an IDS sensor. It is always watching, always ready to alert you to danger. Or something like that…”

This past Monday, portions of the plan dealing with the counterintelligence, supply chain security, and research and development, were discussed with industry group. Up until now, disclosures have been limited to information regarding effort to improve the security of government network. The Deputy Secretary for DHS, Paul Schneider, discussed the three focus areas:

1. Establishing the front lines of defense against cyber attacks and reducing current vulnerabilities.
2. Defending against a full spectrum of threats by using intelligence.
3. Shaping the future through research and investment in new technologies.

It is interesting that Schneider cited the conflict between Russia and Georgia as “perhaps the first instance of military actions containing a clear cyber element.” There is no doubt that the government is very concern about cyber’s role in future warfare. Jack M. Germain wrote an article for TechNewsWorld titled “The Winds of Cyber War.” Tom Stracener, Sr. Security Analyst for Cenzic, told Germain, “The attack on Georgia shows an economy of scale. It was massive attacks on multiple levels. This is not just a U.S. problem. Hamas and Hezbollah have been doing this for years against Israeli Web sites. These types of attacks against opponents’ Web sites are also very common in South America. All of this points to a future of widespread information warfare. It is becoming one more big weapon in the war arsenal.”

Germain’s article goes into further explanation of the government’s attempts to address these concerns. Patrick Peterson, Vice President of Technology at IronPort Systems, stated that the U.S. government decided 12 months ago to spend 30 million to prepare for cyber attacks by establishing the Comprehensive National Cybersecurity Initiative (CNCI). Germain reports that “CNCI was commissioned by two different executive orders to proactively harden government computer systems against intruders rather than reacting to intrusions after the fact.” Peterson goes on to explain, “The activities of the CNCI are so secretive that it functions as an underground agency. Even Senator [Joe] Lieberman, after hounding the administration for an explanation, only received an official letter that was heavily redacted, indicating that the CNCI is a super top secret agency that operates on a need-to-know basis.” Keep in mind that DHS has been designated to play a significant role in implementation of CNCI.

Schneider went on to say, “In research and development we will be spending a significant amount of resources in the private sector and that’s because that’s where the technology’s going to come from.” Industry has a vital role to play in the initiative, as Schneider points out, “We don’t own the nation’s information technology networks or communications infrastructure. What we are faced with is the absolute need for a very unique partnership in order to defend this network.”

The National Science Foundation FY 2009 budget request included $116.9 million for cybersecurity research and education, with $30.0 million specifically devoted respectively to research in usability ($10 million), theoretical foundations ($10 million), and privacy ($10 million) to support the CNCI. NSF stated, “These investments in cybersecurity and information security and privacy will produce research results that allow society to more fully exploit the potential benefits of an increasingly networked world. In addition, the Scholarship for Service program, which funds scholarships to build a cadre of federal professionals with skills required to protect the nation’s critical information infrastructure, increases by 30 percent to $15 million.”

Concerning the the intrusion detection component, Einstein, Schneider stated, “We’ll be deploying a much more aggressive system that will allow us to look for patterns of malicious code–to shut them down before they do real harm.” Schneider did not elaborate further on how these aggressive systems would shut down malicious code. Stephanie Condon, of CNET News, reports that DHS’ Under Secretary for the National Protection and Programs Directorate, Robert Jamison said the department is currently working closely with three different vendors to test “Einstein 2″ in different environments.

On Captol Hill yesterday, there was a hearing before the Subcommittee on Emerging Threats, Cybersecurity and Science and Technology called “Cybersecurity Recommendations for the Next Administration.” There is a live/recorded video feed of the hearing available.

Schneider expressed confidence in continuation of the cyber initiatives stating “The majority of the people running these programs will be running these programs on January 21.” Schneider continued to explain while “any administration can come in with new policies,” he said the elements of the Cybersecurity Initiative, like common situational awareness, “are foundation pieces of any cybersecurity strategy.” One might argue that Schneider comments may have been also addressing critics that are questioning DHS’ future role in cybersecurity. Dennis Fisher, Executive Editor for SearchSecurity, provides additional details in his article “DHS should lose cybersecurity authority, experts say.” Condon also provides insight in the article, “Critics: Homeland Security unprepared for cyberthreats.”

“Our view is that any improvement in the nation’s cybersecurity must go outside of DHS to be effective,” stated James Lewis, Director and Senior Fellow, Technology and Public Policy Program. Lewis appeared on behalf of CSIS’s Commission on Cybersecurity for the 44th Presidency, a group made up of 40 cybersecurity and government experts. A final report is expected in November and will contain recommendations for the next administration.

Government Accountability Office (GAO) released two reports (No. 1 and No. 2) adding to the public criticism of DHS. The GAO has been reporting on DHS’ cybersecurity efforts since 2005 and has made 30 recommendations to the department. David Powner, GAO’s director of information management issues, stated, “Clearly our work has demonstrated that DHS has been completely ineffective in fulfilling their role as the cybersecurity focal point.” The GAO’s new reports include descriptions of the department’s failure to fully address 15 key cyberanalysis and warning attributes related to activities such as monitoring government networks for unusual activity. “Congress has to be involved with this,” Lewis said, “to support building the infrastructure that will keep us secure.”

Paul Kurtz is a partner at Good Harbor Consulting (which is lead by Richard A. Clarke), and a former adviser to President Bush on cybersecurity issues. Kurtz reports that during a late June briefing for private-sector executives about the new cybersecurity initiative, senior DHS officials had disagreed openly about how to move ahead. “What was so discouraging about that day, and I’ll never forget it, is that we had infighting between DHS leaders as to how to proceed,” Kurtz said. “It demonstrated in spades the lack of leadership, and that no one is in charge at DHS. It was a travesty. We had 70 or so private sector people in the room who had spent a lot of time and once again been asked to come up with some ways that we could better work together and the department basically threw it overboard. It was incredibly discouraging to witness.” Kurtz also stated DHS’ problems stems from the fact that, “you have several people with their hands on the steering wheel.” Echoing Kurtz concerns is subcommittee member Rep. William Pascrell, D-N.J, “The last time I checked, we had at least four people at DHS who claim to be in charge of cybersecurity.”

Kurtz stressed that “there is good work being done.” Lewis agrees and describes the major problem being that the department, “really doesn’t have the authority to direct other departments and agencies. If anything, its authority has probably declined as other departments have moved out on this issue.” Lewis went on to say, “The conclusion we reached is only the White House has the authority and oversight for cybersecurity. This is now a serious national security problem and should be treated as such.” Lewis also expressed the opinion that strengthening the department’s authority was no longer a viable option at this point. “I began in this effort by thinking that we should strengthen DHS,” he told the hearing. “We did not receive much encouragement when we put that forward.” In the end, Lewis reports that his suggestion that the problems could be solved by strengthening DHS’ authority was “shot down by my own commission.”

Of course, this is Washington and other explanations for DHS’ criticism are possible. “Rearranging the deck chairs is a classic inside-the-Beltway pastime, but all that it ensures is that in two years the government’s cyber efforts will be in the same place,” Laura Keehner, DHS Press Secretary, stated. Michael Smith in his must read post, “Cage Match: OMB Report V/S GAO Report, Only One Comes Out Alive,” provides some great insight into the different perspectives and motives government agencies might have. In government, where a great deal of money is involved along with secrecy shrouding most of the operations, who knows what is real? Still, it is fun to watch and speculate. As promised, below are the links to publicly available articles from which the information used in this post was obtain.

• Government elaborates, slightly, on cybersecurity plan by Stephanie Condon
• Agencies discuss new cybersecurity plan by Ben Bain
• The Winds of Cyber War by Jack M. Germain
• Cybersecurity Recommendations for the Next Administration
• Critics: Homeland Security unprepared for cyberthreats by Stephanie Condon
• DHS should lose cybersecurity authority, experts say by Dennis Fisher, Executive Editor