In today’s work environment there is a need to show changes, potential risks, improved performance, etc. in all areas of the company’s operations. Security professionals need to be prepared to answer the basic question, “why should the CIO or CEO care about security?” CSO Online has a great quote from the post, “From the CIO: Why You Didn’t Get the CISO Job” that challenges us to consider our views when it comes to security. The post states, “laser focus on your speciality is great in middle management. It’s what we want. One of the really hard things about jumping from management to executive is a focus on the whole of the business. It’s a rare person who manages it quickly or easily.” That is basically the problem with metrics. It is a battle between generalization to the point of uselessness and details to the point of not being understandable or collectible. At the end of the day, something needs to be done because the security industry is currently leaving upper management in the position of not understanding what is going on within their business. That is a risk that not acceptable.

Andrew’s article discusses what kind of security metrics should be used. Additional sources of information on security metrics can be found in a previous post entitled “Security Metrics.” The post provides links to wonderful sources on security metric information. You might also want to take a look at the CIS Consensus Security Metrics v1.0.0 guide, NIST Special Publication (SP) 800-55 Rev 1 “Security Metrics Guide for Information Technology Systems”, NIST IR-7564 “Directions in Security Metrics Research”, “Twenty Most Important Controls and Metrics for Effective Cyber Defense and Continuous FISMA Compliance,” and “Metrics, measures & Myths.” Once you have start gathering metrics, you will want to present them in an easy to understand format. This is where Google Visualization can help.

Today’s post walks through an example using the data from the National Institute of Standards and Technology (NIST) National Vulnerability Database (NVD) Common Vulnerabilities and Exposures (CVE) database. The purpose is to provide a working example from which you can learn and apply to the various metrics gathered at your organization.

Data Source

A previous post, “Standardization and Interoperability in Security,” discussed how the Security Content Automation Protocol (SCAP) is an attempt to help defenders by providing a collection of XML schemas/standards that allow technical security information to be exchanged between tools. SCAP components consists of:

• Common Configuration Enumeration (CCE): provide unique identifiers to system configuration issues in order to facilitate fast and accurate correlation of configuration data across multiple information sources and tools.
• Common Platform Enumeration (CPE): a structured naming scheme for information technology systems, platforms, and packages.
• Common Vulnerability Enumeration (CVE): a dictionary of publicly known information security vulnerabilities and exposures.
• Common Vulnerability Scoring System (CVSS): a vulnerability scoring system designed to provide an open and standardized method of rating IT vulnerabilities. NIST has even provided a calculator for creating CVSS vulnerability severity scores.
• eXtensible Checklist Configuration Description Format (XCCDF): a specification language for writing security checklists, benchmarks, and related kinds of documents. NIST has released the NIST Interagency Report 7275 Revision 3 “Specification for Extensible Configuration Checklist Description Format (XCCDF) Version 1.1.4.”
• Open Vulnerability Assessment Language (OVAL): an information security community standard to promote open and publicly available security content, and to standardize the transfer of this information across security tools and services.

We are going to make use of the data from NVD/CVE XML feed with the Common Vulnerability Scoring System (CVSS) mappings (version 2.0). NIST documentation states:

CVSS provides an open framework for communicating the characteristics and impacts of IT vulnerabilities. Its quantitative model ensures repeatable accurate measurement while enabling users to see the underlying vulnerability characteristics that were used to generate the scores. Thus, CVSS is well suited as a standard measurement system for industries, organizations, and governments that need accurate and consistent vulnerability impact scores. Two common uses of CVSS are prioritization of vulnerability remediation activities and in calculating the severity of vulnerabilities discovered on one’s systems.

NVD provides CVSS ‘base scores‘ representing the innate characteristics of each vulnerability. ‘Temporal scores,’ which change over time due to events external to the vulnerability, are not provided though NVD does provide a CVSS score calculator. This allows an organization to add temporal data and even factor in ‘environmental scores‘ customized to reflect the impact of the vulnerability on the organization. Please refer to the CVSS standards guide and the OWASP Risk Rating Methodology concerning factors involved in estimating the severity of risks to your business.

NVD CVE XML Schema

For our example, we will be using the data feeds nvdcve-2.0-2010.xml and nvdcve-2.0-2009.xml. Examining the CVE XML 2.0 Schema, we are particularly interested in certain vulnerability and CVSS scoring information. For example, for CVE-2010-1228, we will parse and pull the following kind of information:

<entry id="CVE-2010-1228">
  <vuln:cve-id>CVE-2010-1228</vuln:cve-id>
  <vuln:published-datetime>2010-04-01T18:30:00.453-04:00
  </vuln:published-datetime>
  <vuln:last-modified-datetime>2010-04-05T00:00:00.000-04:00
  </vuln:last-modified-datetime>
  <vuln:cvss>
    <cvss:base_metrics>
      <cvss:score>10.0</cvss:score>
      <cvss:access-vector>NETWORK</cvss:access-vector>
      <cvss:access-complexity>LOW</cvss:access-complexity>
      <cvss:authentication>NONE</cvss:authentication>
      <cvss:confidentiality-impact>COMPLETE</cvss:confidentiality-impact>
      <cvss:integrity-impact>COMPLETE</cvss:integrity-impact>
      <cvss:availability-impact>COMPLETE</cvss:availability-impact>
      <cvss:source>http://nvd.nist.gov</cvss:source>
    </cvss:base_metrics>
  </vuln:cvss>
</entry>

Using Perl to Retrieve the CVE File

Initially we will read the nvdcve-2.0-2010.xml and nvdcve-2.0-2009.xml files. If we start retrieving the file regularly, we would want to change this to nvdcve-2.0-recent.xml. Of course, previous years can also be read in to provide a longer perspective on vulnerabilities. A simple example of a Perl subroutine to read the NVD CVE file and save it locally would be:

sub readpage {
   my($url,$nvd_file) = @_;
   my($proxy) = "http://your-proxy-server:proxy-port";
   my $ua = new LWP::UserAgent;
   $ua->proxy(http  => $proxy);
   $ua->proxy(ftp => $proxy);
   $ua->proxy(https => $proxy);
   # Go out and retrieve page
   my $req = new HTTP::Request('GET', $url);
   my $res = $ua->request($req);
   my $pjstatus = 1;
   # Check if the requested webpage is there and return results
   if ($res->is_success) { # Request successful
       open(OUTFILE,">$nvd_file") || ($pjstatus = 0);
       if ($pjstatus) {
          print OUTFILE $res->content;
       }
       close(OUTFILE);
   }
   else {
      $pjstatus = 0;
   }
   return($pjstatus);
}

Please substitute “http://your-proxy-server:proxy-port” with your site’s proxy server and port, if applicable.

Creating a MYSQL Table to Hold the Data

There is a great deal of information in the NVD CVE file. You will need to determine what information your organization will be interested in storing and graphing. For better or worse, folks have come to expect vulnerabilities to have a “Low,” “Medium,” or “High” score. NIST has stated concerning the NVD Vulnerability Severity Ratings:

NVD provides severity rankings of “Low,” “Medium,” and “High” in addition to the numeric CVSS scores but these qualitative rankings are simply mapped from the numeric CVSS scores:
1. Vulnerabilities are labeled “Low” severity if they have a CVSS base score of 0.0-3.9.
2. Vulnerabilities will be labeled “Medium” severity if they have a base CVSS score of 4.0-6.9.
3. Vulnerabilities will be labeled “High” severity if they have a CVSS base score of 7.0-10.0.

While preferring quantitative over qualitative values, for this example I would like to create a stacked column chart. We will add a severity column which is based on the CVSS score. An example table follows:

CREATE DATABASE vulnerabilities;
USE vulnerabilities;
DROP TABLE IF EXISTS `nvdcve`;
CREATE TABLE `nvdcve` (
  `cve_id` varchar(13) NOT NULL,
  `published` datetime default NULL,
  `modified` datetime default NULL,
  `score` DECIMAL(5,2) default '0.0',
  `severity` varchar(6) default 'LOW',
  `vector` varchar(25) default NULL,
  `complexity` varchar(25) default NULL,
  `authentication` varchar(25) default NULL,
  `confidentiality` varchar(25) default 'NONE',
  `integrity` varchar(25) default 'NONE',
  `availability` varchar(25) default 'NONE',
  `summary` varchar(512) default NULL,
  PRIMARY KEY  (`cve_id`),
  INDEX (score),
  INDEX (vector)
)

Using Perl Populating the Database

Populating the database table is simply a matter of reading the file and adding the entries to the table. An example Perl subroutine follows:

sub readxml {
   my($nvd_file, $dbh) = @_;
   my $parser = XML::LibXML-> new();
   my $doc    = $parser-> parse_file($nvd_file);
   my $xc     = XML::LibXML::XPathContext-> new( $doc->documentElement() );
   $xc-> registerNs(
      def  => 'http://scap.nist.gov/schema/feed/vulnerability/2.0' );
   $xc-> registerNs(
     vuln => 'http://scap.nist.gov/schema/vulnerability/0.4' );
   $xc-> registerNs( cvss => 'http://scap.nist.gov/schema/cvss-v2/0.2' );
   for my $entry ($xc-> findnodes("/def:nvd/def:entry")) {
      my $cve = $xc-> find('vuln:cve-id',$entry);
      my $published = $xc-> find('vuln:published-datetime', $entry);
      my $modified = $xc-> find('vuln:last-modified-datetime', $entry);
      my $summary = $xc-> find('vuln:summary', $entry);
      my $skip = 0;
      my ($metrics) = $xc-> findnodes('vuln:cvss/cvss:base_metrics', $entry) or ($skip = 1);
      if (! $skip) {
         my $score = $xc-> find('cvss:score', $metrics);
         my $vector = $xc-> find('cvss:access-vector', $metrics);
         my $complexity = $xc-> find('cvss:access-complexity', $metrics);
         my $authentication = $xc-> find('cvss:authentication', $metrics);
         my $confidentiality =
            $xc-> find('cvss:confidentiality-impact', $metrics);
         my $integrity = $xc-> find('cvss:integrity-impact', $metrics);
         my $availability = $xc-> find('cvss:availability-impact', $metrics);
         my $severity = "LOW";
         if (int($score) >= 7) {
            $severity = "HIGH";
         }
         elsif (int($score) >= 4) {
            $severity = "MEDIUM";
         }
         my $sql = qq{ SELECT count(*) FROM nvdcve WHERE cve_id=? };
         my $sth = $dbh->prepare( $sql );
         my $rc = $sth->execute($cve);
         if ( $rc) {
            my($exist) = $sth->fetchrow_array();
            if (! $exist) {
                $sql = qq{ INSERT INTO nvdcve SET cve_id=?,
published=?, modified=?, score=?, severity=?, vector=?, complexity=?,
authentication=?, confidentiality=?, integrity=?,availability=?, summary=? };
               $sth = $dbh->prepare( $sql );
               $rc = $sth->execute($cve,$published,$modified,$score,
$severity,$vector,$complexity,$authentication,
$confidentiality,$integrity,$availability,$summary);
            }
         }
      }
   }
}

The Perl Program to Pull It All Together

The above subroutines use the Perl modules LWP::UserAgent, XML::LibXML, XML::LibXML::XPathContext, and DBI. A sample Perl program that calls the above subroutines to pull down the NVD CVE data and load it into a MySQL table would be:

#!/usr/local/bin/perl -w
use LWP::UserAgent;
use XML::LibXML;
use XML::LibXML::XPathContext;
use DBI;
BEGIN{push @INC, "/home/jgerber/projects/nvd/perl"}
use nvdsubs qw($db_host $db $mysql_user $mysql_passwd $mysql.sock
readpage readxml );
# Main
my $datadir = "/home/johngerber/projects/nvd/data";
my @timeData = localtime(time);
my $year = 1900 + $timeData[5];
my $prev_year = 1900 + $timeData[5] - 1;
my $url = "http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-" .
    $year . ".xml";
my $prev_url = "http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-" .
    $prev_year . ".xml";
my $nvd_file = $datadir  . "/nvdcve-". $year . ".xml";
my $prev_nvd_file = $datadir  . "/nvdcve-". $prev_year . ".xml";
$db = "vulnerabilities";
local($dbh) = DBI->connect("DBI:mysql:mysql_socket=$mysql.sock;$db:$db_host",
$mysql_user, $mysql_passwd) || die "ERROR: Connecting: $DBI::errstr\n";
my ($pjstatus) = &readpage($prev_url,$prev_nvd_file);
if ($pjstatus) {
   &readxml($prev_nvd_file,$dbh);
}
$pjstatus = &readpage($url,$nvd_file);
if ($pjstatus) {
   &readxml($nvd_file,$dbh);
}
exit;

The nvdsubs.pm file will not be included in this post. The subroutines are defined and the only pieces missing are the MySQL database username and password. You don’t need mine. Add your own. At this point, we have everything we need to finally use Google Visualization to create a graph.

Google Visualization

We are going to create a Perl program that will read our MySQL nvdcve table and generate the JavaScript that will render our charts on the client’s browser. First, we want to define the JavaScript we want to produce. Just to alleviate some concerns, with Google Visualization your data is only shared between your server and the client connecting. This is unlike Google Charts where your data is sent to Google where it is made into a chart and the result is sent back. Google states concerning the logging of chart data (via Google Charts), “The chart data included in the HTTP request is saved in temporary logs for no longer than two weeks for internal testing and debugging purposes.” Every example in the Google Visualization Gallery will state the data policy. For Google Charts, stated at the bottom of the page for each gadget description the data policy:

While Google Visualization gadgets will have the following stated data policy:

Loading Google Libraries

The first thing the JavaScript needs to do is load the required libraries. This is accomplished with the lines:

<script type="text/javascript" src="http://www.google.com/jsapi"></script>

Area Chart and Table

In this example we are going to create an column chart. In a later section, “Other Charting Options” (see below) we define different Google Visualization charting options.

JavaScript code for a sample column chart would be:

    <script type='text/javascript'>
      google.load('visualization', '1', {packages:['columnchart']});
      google.setOnLoadCallback(drawChart);
      function drawChart() {
        var data = new google.visualization.DataTable();
        data.addColumn('date', 'Date');
        data.addColumn('number', 'High');
        data.addColumn('number', 'Medium');
        data.addColumn('number', 'Low');
        data.addRows([
           [new Date(2009, 0, 30),92,97,3],
           [new Date(2009, 1, 27),168,142,25],
           [new Date(2009, 2, 31),141,165,9],
           [new Date(2009, 3, 30),132,203,12],
           [new Date(2009, 4, 29),158,153,8],
           [new Date(2009, 5, 30),200,199,22],
           [new Date(2009, 6, 31),190,195,11],
           [new Date(2009, 7, 31),127,139,14],
           [new Date(2009, 8, 30),233,208,14],
           [new Date(2009, 9, 30),163,167,18],
           [new Date(2009, 10, 30),129,172,8],
           [new Date(2009, 11, 31),200,211,19],
           [new Date(2010, 0, 29),157,139,14],
           [new Date(2010, 1, 26),137,143,12],
           [new Date(2010, 2, 31),252,242,18],
           [new Date(2010, 3, 13),92,118,17]
        ]);
        var chart = new google.visualization.ColumnChart(document.getElementById('s4graph'));
        chart.draw(data, {displayAnnotations:true, is3D: true, isStacked: true, min: 0,
          allowHtml: true, colors:[{color:'#E41B17', darker:'#C11B17'}, {color:'#FFA500', darker:'#E56717'}, {color:'#FFE87C', darker:'#C8B560'}]});
      }
    </script>

The resulting image would be the following column chart:

Rendering the Table

When providing qualitative results, I like to back them up with more accurate numeric values. Let us include a table with links to the CVSS scores for each vulnerability.

    <script type='text/javascript'>
      google.load('visualization', '1', {packages:['table']});
      google.setOnLoadCallback(drawChart);
      function drawChart() {
        var data2 = new google.visualization.DataTable();
        data2.addColumn('date', 'Date');
        data2.addColumn('number', 'High');
        data2.addColumn('number', 'Medium');
        data2.addColumn('number', 'Low');
        data2.addRows([
           [{v:new Date(2009, 0, 30),
              f:'<a href="/nvd/cvealerts.php?date=2009-01">2009-01-30</a>'}, 92,97,3],
           [{v:new Date(2009, 1, 27),
              f:'<a href="/nvd/cvealerts.php?date=2009-02">2009-02-27</a>'}, 168,142,25],
           [{v:new Date(2009, 2, 31),
              f:'<a href="/nvd/cvealerts.php?date=2009-03">2009-03-31</a>'}, 141,165,9],
           [{v:new Date(2009, 3, 30),
              f:'<a href="/nvd/cvealerts.php?date=2009-04">2009-04-30</a>'}, 132,203,12],
           [{v:new Date(2009, 4, 29),
              f:'<a href="/nvd/cvealerts.php?date=2009-05">2009-05-29</a>'}, 158,153,8],
           [{v:new Date(2009, 5, 30),
              f:'<a href="/nvd/cvealerts.php?date=2009-06">2009-06-30</a>'}, 200,199,22],
           [{v:new Date(2009, 6, 31),
              f:'<a href="/nvd/cvealerts.php?date=2009-07">2009-07-31</a>'}, 190,195,11],
           [{v:new Date(2009, 7, 31),
              f:'<a href="/nvd/cvealerts.php?date=2009-08">2009-08-31</a>'}, 127,139,14],
           [{v:new Date(2009, 8, 30),
              f:'<a href="/nvd/cvealerts.php?date=2009-09">2009-09-30</a>'}, 233,208,14],
           [{v:new Date(2009, 9, 30),
              f:'<a href="/nvd/cvealerts.php?date=2009-10">2009-10-30</a>'}, 163,167,18],
           [{v:new Date(2009, 10, 30),
              f:'<a href="/nvd/cvealerts.php?date=2009-11">2009-11-30</a>'}, 129,172,8],
           [{v:new Date(2009, 11, 31),
              f:'<a href="/nvd/cvealerts.php?date=2009-12">2009-12-31</a>'}, 200,211,19],
           [{v:new Date(2010, 0, 29),
              f:'<a href="/nvd/cvealerts.php?date=2010-01">2010-01-29</a>'}, 157,139,14],
           [{v:new Date(2010, 1, 26),
              f:'<a href="/nvd/cvealerts.php?date=2010-02">2010-02-26</a>'}, 137,143,12],
           [{v:new Date(2010, 2, 31),
              f:'<a href="/nvd/cvealerts.php?date=2010-03">2010-03-31</a>'}, 252,242,18],
           [{v:new Date(2010, 3, 13),
              f:'<a href="/nvd/cvealerts.php?date=2010-04">2010-04-13</a>'}, 92,118,17],
        ]);
        var table = new google.visualization.Table(document.getElementById('s4graph_tab'));
        table.draw(data2, {showRowNumber: true, sortAscending: false, sortColumn: 0, allowHtml: true});
      }
    </script>

The JavaScript code assumes there is a PHP program called cvealerts.php under the /nvd directory on your web server. Adjust to your environment. A sample PHP program that could be used for cvealerts.php is provided below. The resulting table chart would look like:

Handling Events: Interactions Between Graphs

We now have two different types of graphs representing the same data. We want to add interaction between the graphs so the viewer can see the relationship. With tables rows are selected when the user clicks, which correspond to the whole column of the stacked column chart. It is not a perfect fit, but it does demonstrate nicely use of adding interactions.

        // Set a 'select' event listener for the table.
        // When the table is selected,
        // we set the selection on the line graph.
        google.visualization.events.addListener(table, 'select', function() {
          chart.setSelection([{row: table.getSelection()[0].row, column: 1}]);
         });
        // Set a 'select' event listener for the graph.
        // When the graph is selected,
        // we set the selection on the table.
        google.visualization.events.addListener(chart, 'select', function() {
           table.setSelection([{row: chart.getSelection()[0].row}]);
        });

Providing Detailed Information

When the table chart link is clicked, we would like to provide some detailed information about the vulnerability. For this example, we will do this with a simple PHP program placed in the /nvd directory on the web server. The program is called cvealerts.php.

<?
session_start();
function db_connect($table) {
   $result = mysql_pconnect("<dbhost>:<dbport>", "<username>", "<password>");
   if (!$result) return false;
   if (!mysql_select_db($table)) return false;
   return $result;
}
function do_html_header($title,$checkuser,$logpage) {
?>
  <html> <head> <title><?=$title?></title></head>
  <body bgcolor="#FFFFFF">
<?
}
function do_html_footer() {
?>
<table>
<tr><td ALIGN=CENTER NOWRAP WIDTH="590"></font>
<font face="Verdana, Arial, Helvetica" size=-2>Notice to Users: Use
of this system constitutes consent to security monitoring and testing.
<br>All activity is logged with your host name and IP address.</font>
</td></tr>
</table>
</body>
 </html>
<?
}
// Main
$dates= array();
$stringlist = "";
if (isset($_GET['date'])) {
    $passdates = explode(",",$_GET['date']);
    for ($index=0; $index<count($passdates); $index++) {
       array_push($dates, $passdates[$index]);
       $stringlist .= $passdates[$index] . " ";
    }
}
else {
  print("Confusion over how you arrived at this page.<P>\n");
  exit;
}
$stringlist = preg_replace("/ $/", "",$stringlist);
do_html_header("Review NVD CVE Announcements for Month Ending $stringlist",1,1);
$nvd_host = "http://web.nvd.nist.gov/view/vuln/detail?vulnId=";
$conn = db_connect("vulnerabilities");
if (!$conn)
   logit("Could not connect to database vulnerabilities - please try later.\n",1);
for ($index=0; $index<count($dates); $index++) {
   $rule = $dates[$index];
   $sql = "SELECT cve_id,score,published,vector,severity,complexity,left(summary,50)
    FROM vulnerabilities.nvdcve
      WHERE date_format(published,'%Y-%m')='$rule'
       ORDER BY (score+0)";
   $result = mysql_query($sql,$conn);
   if (!$result)
       logit("Problem with $sql\n",1);
   print("<table border=1><tr><td><table border=0><tr><th bgcolor=\"#727D96\">
<font color=\"#ffffff\" face=\"arial,helvetica,sanserif\">Bulletin</font></th><th bgcolor=\"#727D96\">
<font color=\"#ffffff\" face=\"arial,helvetica,sanserif\">Impact</font></th><th bgcolor=\"#727D96\">
<font color=\"#ffffff\" face=\"arial,helvetica,sanserif\">Date</font></th><th bgcolor=\"#727D96\">
<font color=\"#ffffff\" face=\"arial,helvetica,sanserif\">Vector</font></th><th bgcolor=\"#727D96\">
<font color=\"#ffffff\" face=\"arial,helvetica,sanserif\">Severity</font></th><th bgcolor=\"#727D96\">
<font color=\"#ffffff\" face=\"arial,helvetica,sanserif\">Complexity</font></th><th bgcolor=\"#727D96\">
<font color=\"#ffffff\" face=\"arial,helvetica,sanserif\">Short Summary</font></th></tr>\n");
   for ($count = 1; list($cve_id, $score, $date, $vector, $severity,$complexity,$shortsum) =
     mysql_fetch_array ($result, MYSQL_NUM); ++$count) {
?>
      <tr><td CLASS="plfieldhdrleft" WIDTH="20%" BGCOLOR='#F0F5FF'>
      <?  print("<a href=\"$nvd_host$cve_id\">$cve_id</a>"); ?>
      </td>
      <td CLASS="plfieldhdrleft" BGCOLOR='#F9FCFF'>
      <?  print($score); ?>
      </td>
      <td CLASS="plfieldhdrleft" BGCOLOR='#F0F5FF'>
      <?  print($date); ?>
      </td>
      <td CLASS="plfieldhdrleft" BGCOLOR='#F9FCFF'>
      <?  print($vector); ?>
      </td>
      <td CLASS="plfieldhdrleft" BGCOLOR='#F0F5FF'>
      <?  print($severity); ?>
      </td>
      <td CLASS="plfieldhdrleft" BGCOLOR='#F9FCFF'>
      <?  print($complexity); ?>
      </td>
      <td CLASS="plfieldhdrleft" BGCOLOR='#F0F5FF'>
      <?  print($shortsum); ?>
      </td>
      </tr>
<?
   }
}
print("</table></td></tr></table>");
do_html_footer();

The PHP program would generate a HTML table displaying the NVD CVE alerts for that month. The table would look like:

When the CVE link is clicked on, the user is taken to the NIST NVD site where additional information is available.

Using Perl to Create the JavaScript

The Perl code is rather simple now that we have the MySQL tables defined and the JavaScript we want to generate. Much of the code consists of the JavaScript listed above.

#!/usr/local/bin/perl -w
use DBI;
use Time::Local;
use POSIX qw(strftime);
use LWP::UserAgent;
BEGIN{push @INC, "/home/jgerber/projects/nvd/perl"}
use ornl_feds qw($db_host $db $mysql_user $mysql_passwd );
sub slide_nvd_alerts {
  my($min_date,$graph_name,$web_link,$dbh) = @_;
  my $slide = "";
  my $slide_head = qq!
    <script type='text/javascript'>
      google.load('visualization', '1', {packages:['columnchart,table']});
      google.setOnLoadCallback(drawChart);
      function drawChart() {
        var data = new google.visualization.DataTable();
        data.addColumn('date', 'Date');
        data.addColumn('number', 'High');
        data.addColumn('number', 'Medium');
        data.addColumn('number', 'Low');
        data.addRows([
!;
   my $slide_head_table = qq!
        var data2 = new google.visualization.DataTable();
        data2.addColumn('date', 'Date');
        data2.addColumn('number', 'High');
        data2.addColumn('number', 'Medium');
        data2.addColumn('number', 'Low');
        data2.addRows([
!;
   my $table_div = $graph_name . "_tab";
   my $slide_tail = qq!
        var chart = new google.visualization.ColumnChart(document.getElementById('$graph_name'));
        chart.draw(data, {displayAnnotations:true, is3D: true, isStacked: true, min: 0, allowHtml: true,
 colors:[{color:'#E41B17', darker:'#C11B17'}, {color:'#FFA500', darker:'#E56717'},
{color:'#FFE87C', darker:'#C8B560'}]});
        var table = new google.visualization.Table(document.getElementById('$table_div'));
        table.draw(data2, {showRowNumber: true, sortAscending: false, sortColumn: 0, allowHtml: true});
            // Set a 'select' event listener for the table.
        // When the table is selected,
        // we set the selection on the line graph.
        google.visualization.events.addListener(table, 'select', function() {
          chart.setSelection([{row: table.getSelection()[0].row, column: 1}]);
         });
      // Set a 'select' event listener for the graph.
        // When the graph is selected,
        // we set the selection on the table.
        google.visualization.events.addListener(chart, 'select', function() {
           table.setSelection([{row: chart.getSelection()[0].row}]);
        });
      }
    </script>
!;
   if ($min_date eq "") {
      my $sql2 = qq{ SELECT min(published) FROM vulnerabilities.nvdcve };
      my $sth2 = $dbh->prepare( $sql2 );
      my $rc2 = $sth2->execute();
      if ($rc2) {
         $min_date = $sth2->fetchrow_array();
      }
   }
   my $table_data = "";
   my $graph_data = "";
   my $sql2 = qq{ select date_format(published,'%Y-%m'),severity,count(severity)
      FROM vulnerabilities.nvdcve where published >= ? group by date_format(published,'%Y-%m'),severity };
   my $sth2 = $dbh->prepare( $sql2 );
   my $rc2 = $sth2->execute($min_date);
   if ($rc2) {
      my ($change,$virgin,$ht,$mt,$lt,$mmax_date) = ("",1,0,0,0,"");
      while (my($snapshot_date, $severity, $pcount) = $sth2->fetchrow_array()) {
         my $sql3 = qq{ SELECT max(published) FROM vulnerabilities.nvdcve where
date_format(published,'%Y-%m')=? };
         my $sth3 = $dbh->prepare( $sql3 );
         my $rc3 = $sth3->execute($snapshot_date);
         $max_date =  $sth3->fetchrow_array();
         $max_date =~ s/ \S+$//;
         if ($change ne $snapshot_date) {
            if (! $virgin) {
                my($year,$month,$day) = split("-",$mmax_date);
                my $mmonth = $month;
                $month--;
                $graph_data .= qq!           [new Date($year, $month, $day),$ht,$mt,$lt],
!;
                $table_data .= qq!           [{v:new Date($year, $month, $day),
              f:'<a href="$web_link/cvealerts.php?date=$year-$mmonth">$mmax_date</a>'}, $ht,$mt,$lt],
!;
                ($ht,$mt,$lt) = (0,0,0);
             }
             $change = $snapshot_date;
          }
          if ($severity eq "HIGH") { $ht = $pcount; }
          elsif ($severity eq "MEDIUM") { $mt = $pcount; }
          elsif ($severity eq "LOW") { $lt = $pcount; }
          if ($mmax_date eq "") { $mmax_date = $max_date; }
          if ($mmax_date lt $max_date) { $mmax_date = $max_date; }
          $virgin = 0;
      }
      my($year,$month,$day) = split("-",$mmax_date);
      my $mmonth = $month;
      $month--;
      $graph_data .= qq!           [new Date($year, $month, $day),$ht,$mt,$lt]
!;
     $table_data .= qq!           [{v:new Date($year, $month, $day),
              f:'<a href="$web_link/cvealerts.php?date=$year-$mmonth">$mmax_date</a>'}, $ht,$mt,$lt],
!;
   }
   $table_data .= "        ]);\n";
   $graph_data .= "        ]);\n";
   $slide = $slide_head .  $graph_data . $slide_head_table . $table_data . $slide_tail;
   return($slide);
}
sub slide_body {
  my($graph_name,$title,$style) = @_;
  my $table_name = $graph_name . "_tab";
  my $table_text = "div id=\"$table_name\"";
  if ($style ne "") {
     $table_text .= " style=\'$style\'";
  }
  my $slide2 = "<h3>$title</h3>\n";
  my $itext = "div id=\"$graph_name\"";
  if ($style ne "") {
     $itext .= " style=\'$style\'";
  }
  $slide2 .= qq{
    <table><tr>
    <td valign="top"><$itext></div></td>
    <td valign="top"><$table_text></div></td>
    <td valign="top">   </td>
    <td valign="top"><div id="labels"></div></td>
    </tr></table>
  };
  return($slide2);
}
# Main
my $web_link = "/nvd";
my $results_dir = "/data/html" . $web_link;
my $result_file = $results_dir . "/nvdcve_stats.html";
my $debug = 1;
my $db = "vulnerabilities";
local($dbh) = DBI->connect("DBI:mysql:$db:$db_host", $mysql_user, $mysql_passwd) ||
   die "ERROR: Connecting: $DBI::errstr\n";
$slides_data .= &slide_body("s4graph","NVD CVE Alerts","width:700px; height:400px;");
$slides_head .= &slide_nvd_alerts("","s4graph",$web_link,$dbh);
open(OUTFILE,">$result_file");
print OUTFILE "<HTML>\n<HEAD><TITLE>NVD CVE Statistics</TITLE>\n";
print OUTFILE "<script type=\"text/javascript\" src=\"http://www.google.com/jsapi\"></script>\n";
print OUTFILE $slides_head;
print OUTFILE "</HEAD>\n<BODY>\n";
print OUTFILE $slides_data;
print OUTFILE "</BODY>\n";
close(OUTFILE);
exit;

Other Charting Options

Google, Google users, and other companies have shared some JavaScript visualizations built on the Google Visualization API to help you get started. Below are some example:

	Annotated Time Line (GWT Integrated) An animated time series chart. By: Google
	Area Chart (GWT Integrated) Interactive area chart. By: Google
	Bar Chart (GWT Integrated) Interactive bar chart. By: Google
	Bars of Stuff Fun bar charts using images of trains, chocolate, worms, and more. By: The visapi project
	Bio Heat Map Heatmaps are a useful way to visualize matricies of data. Scientists often use green-black-red heatmaps to visualize gene expression data from microarrays. This visualization supports both three color heatmaps (ex: green to black to red) and two color heatmaps (ex: white to yellow). By: Institute for Systems Biology
	Column Chart (GWT Integrated) Interactive column chart. By: Google
	Drastic Treemap A dynamic treemap in Flash. By: DrasticData
	Dygraphs The dygraphs JavaScript library produces interactive, zoomable charts of time series. By: Dan Vanderkam
	Filters A Visualization that acts as a control over other visualizations. It is rendered within the browser using HTML. This visualization offers the ability to select some criteria to filter the DataTable used by the controlled visualizations. By: Institute for Systems Biology
	Gauge (GWT Integrated) Each numeric value is displayed as a gauge. By: Google
	Geo Map (GWT Integrated) A map of a country, continent, or region map, with colors and values assigned to specific regions. Values are displayed as a color scale, and you can specify optional hovertext for regions. By: Google
	Intensity Map (GWT Integrated) An intensity map that highlights regions or countries based on relative values. By: Google
	Line Chart (GWT Integrated) Interactive line chart. By: Google
	Magic-Table The Magic Table is a JavaScript library that allows you to see more in your data by applying some simple visual techniques to transform a table. The table is displayed in the browser by the canvas element. Internet Explorer is not supported. By: Greg Ross
	Map (GWT Integrated) An interactive map that uses the Google Maps API. By: Google
	Motion Chart (GWT Integrated) Motion Chart: A dynamic flash based chart to explore several indicators over time. Required columns: bubble name, time and 2 columns of numeric values. Optional columns: Numeric values or categories. By: Google
	Organizational Chart A GWT Integrated simple organizational chart. By: Google
	Parallel Coordinates Chart Parallel Coordinates is a method of visualizing multivariate data. An n-dimensional space is represented as n parallel lines. Works for browsers based on Gecko or Presto (does not work in IE). This is written in Javascript, no Flash required. By: Sri Harsha Allamraju
	Pie Chart (GWT Integrated) Interactive pie chart. By: Google
	Piles of Money Column chart made of of money bills. By: The visapi project
	Scatter Chart (GWT Integrated) Interactive scatter chart. By: Google
	Table (GWT Integrated) A highly customizable table with sorting, paging and selection capabilities. By: Google
	TermCloud A list of terms, where the size and color of each word is determined by a specified frequency value (typically the number of times it appears in some text). By: The visapi project
	Thematic Mapping API Enables visualization of data in Google Earth or other geobrowsers through the use of the Google Visualization API and KML. By: Thematicmapping.org
	WordCloud Displays all words in text with size and color based on the number of time each word appears. By: The visapi project

Additional Information

Below is the talk that Itai Raz, the lead engineer for the Visualization API product at Google, gave at Google I/O 2009 titled “Using the Visualization API with GWT:”

Additional Possibilities

The work above is meant only to serve as a starting point. There is a great deal more information to expand upon. For example, we began this post pulling some information from the XML schema for CVE-2010-1228. One field we did not pull out from the XML file is:

    <vuln:cwe id="CWE-362" />

The Common Weakness Enumeration (CWE) represents vulnerability types and NIST provides a CWE Cross Section Mapped into by NVD table. In the above example, we see an entry:

Clicking on the link will take us to the MITRE site that provides a great deal more information on CWE entries. It is easy enough to expand on the above program to harvest this information for a richer information database.

Another possibility is to expand the above program to pull additional information on the CVE entry. In additional to the data in the NVD CVE XML file, we could pull information from the NVD site. Using CVE-2010-1228 as an example, we could have the program pull down the page:

http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2010-1228

Notice the line:

CVSS v2 Base Score:10.0 (HIGH) (AV:N/AC:L/Au:N/C:C/I:C/A:C) (legend)

The (AV:N/AC:L/Au:N/C:C/I:C/A:C) provides values that were used in determining the base score. If you follow the link, you will see the values used in the calculations:

• CVSS Base Score: 10
  • Impact Subscore: 10
  • Exploitability Subscore: 10
• CVSS Temporal Score: Undefined
• CVSS Environmental Score: Undefined
• Overall CVSS Score: 10

NVD has made available the equations used in calculating the CVSS base score, temporal score, and environmental score.

Three other pieces of information that might provide interesting groupings are:

• Access Complexity: Low **NOTE: Access Complexity scored Low due to insufficient information
• Authentication: Not required to exploit
• Impact Type: Allows unauthorized disclosure of information; Allows unauthorized modification; Allows disruption of service

What information is of interest and how it is used will be dependent on your organization. There is a great deal of information available and many directions you start examining.

Final Thoughts

I am often reminded of the old phrase, “Trust us, we are from the government.” No one really trusts anyone, especially when it comes to matters they do not understand. Just because you are from the security group at your organization, is that reason enough for the CEO to give you unlimited money and authority to do what you see fit? Of course not. While management might trust you, they may not believe that you are capable of seeing the big picture. That is after all their job.

Another great old saying is that “the devil is in the details.” Those details will likely fall in the security domain. In organization across the planet there is a tug of war between the details and the big picture with multiple groups adding in their opinions and views. You need to make the details understandable to your higher management to effectively argue your view. Finding effective metrics and finding clear representation is essential in today’s business. Google Visualization can be a useful tool in accomplishing this task.

Challenges

Johnathan Ham and Sherri Davidoff from SANS Internet Storm Center (ISC) and Raul Siles from InGuardians have created two recent, still open, security challenges. Sherri, co-author with Jonathan of the SANS’ Network Forensics course, has posted “Network Forensics Puzzle Contest!” (8-14-2009). The most elegant solution wins a free SANS On-Demand class (worth up to $3500 depending on the course). Raul wrote a new hacking challenge on the Ethical Hacker Network site, titled “Prison Break – Breaking, Entering & Decoding” (7-27-2009). Three winners will be selected based on: the best technical answer, creativity (while also technically correct), and a random drawing. Winners will receive signed copies of Ed’s book, “Counter Hack Reloaded.”

Ed Skoudis, of Counter Hack Reloaded fame along with various SAN’s hacking and penetration testing courses (see Ed in Virginia Beach teaching “Network Penetration Testing: Planning, Scoping, and Recon” August 30th – September 4th), is the host bringing monthly new challenges created and managed by the fine folks of InGuardians. The great thing about past challenges is that they allow you to try the problems and check your solutions immediately. Check out Ed’s Counter Hack Reloaded site for a few additional, older challenges.

UPDATE: For a challenge in the forensic’s realm, check out the series of posts by Dave Hull (trustedsignal) on the SANS Forensics Blog. This series discusses the FAT file system. Dave provided the following description: “I’ve provided a copy of the disk image used during the series and have ended almost every post with a challenge question and have been giving away a forensics related title from the Syngress Publishing group. We’ve had a great time and the series is in the archives for anyone who wants to check it out.” Dave is working on a series for NTFS, which he should start posting in the next few weeks. The series is very informative and a great hands-on way to learn.

Data Sets

Greg Conti, author of Security Data Visualization, co-authored the paper, “Toward Instrumenting Network Warfare Competitions to Generate Labeled Datasets.” The paper was done for the CSET ‘09 Worksop on Cyber Security Experimentation and Test. The authors demonstrates how the network warfare competitions can be instrumented to generate modern labeled data sets. They have made available the archived data capture and log files from the 2009 Inter-Service Academy Cyber Defense Competition. The annual competition pits the service academies, including West Point, against an actual National Security Agency Red Team. There is a great deal to be learned by examining this data. A blog has been setup to discuss the data. They are hoping to do a few data captures of network warfare games, as well as, data captures of red-on-blue events at the US Military Academy at West Point.

There are a few additional sites where you can obtain data captures. JJC, from the “Security – The Global Perspective” blog, manages the OpenPacket.org site. The site’s mission is to “provide quality network traffic traces to researchers, analysts, and other members of the digital security community.” The site pcapr, powered by Mu Dynamics, calls itself a “social nOtworking site.” Go to the site to learn about networks and protocols from packet captures.

UPDATE: The folks from pcapr wrote in and pointed out that they just made available the “Collaborative Network Forensics” area where they “took the recently published ITOC dataset and the CCTF captures from the Shmoo group, indexed them for real-time browsing and contextual search/extract.” As they point out, “with over 15.0 GBytes and 26.3 million packets, this now represents the largest collection of indexed pcaps online.” Really nice.

The VizSEC site maintains links to various repositories of data sets. SourceForge, as part of the NetworkMiner tool, has links to publically available PCAP files. The wireshark site also has a few links and sample PCAP files.

Practicing

Practice can be done by installing software, using disk images, or by going to sites/training grounds. Installing software will create a vulnerable site. Make sure to do it onto a local machine inside your LAN which is used solely for testing. For ISO images, make sure you set the VM to use the IP addresses that are only available from the local host OS (NAT or Host-only). If you go to a site, take caution and remember the site could be hostile. In other words, be properly paranoid.

Software

In my post “WebGoat, Lua, and ModSecurity verses Password Guessing,” I go through the steps of setting up WebGoat. WebGoat is a deliberately insecure J2EE web application maintained by OWASP and is intended to teach a structured approach to test and exploiting vulnerabilities within an application security assessment. WebGoat is written in Java and installs on any platform with a Java virtual machine. The YGN Ethical Hacker Group has made available a series of video on walking through WebGoat v5.2. There are currently over 30 lessons.

Damn Vulnerable Web App (DVWA) is a PHP/MySQL Web application that is light weight, easy to use and full of vulnerabilities to exploit. Ryan Dewhurst, developer of DVWA, created a video showing the installation process:

If you prefer PHP scripts, Mutillidae is a set that implements the OWASP Top 10 vulnerabilities. Adrian Crenshaw posted the presentation he gave to the Louisville Chapter of OWASP about the Mutillidae project titled “OWASP Top 5 and Mutillidae: Intro to common web vulnerabilities like Cross Site Scripting (XSS), SQL/Command Injection Flaws, Malicious File Execution/RFI, Insecure Direct Object Reference and Cross Site Request Forgery (CSRF/XSRF).”

Owasp Louisville 2nd Meeting from Adrian Crenshaw on Vimeo.

ISO Disk Images

On the ISO disk image side, there are few interesting options. Badstore demo helps in the understanding of Web application vulnerabilities and shows how to reduce exposure.

For full scaled lesson based environment, there is the Linux-based distribution Damn Vulnerable Linux (DVL). Mayank Sharma writes in the article “Securing Linux by breaking it with Damn Vulnerable Linux:”

“Damn Vulnerable Linux (DVL) is everything a good Linux distribution isn’t. Its developers have spent hours stuffing it with broken, ill-configured, outdated, and exploitable software that makes it vulnerable to attacks. DVL isn’t built to run on your desktop — it’s a learning tool for security students.”  “The one thing that sets DVL apart the most,” Josh Sweeney says, “is the focus on buffer overflows and disassembly.” Disassembly, he says, is often talked about in conjunction with buffer overflows and reverse engineering. “Disassembling is when someone breaks down a program into the assembly language for further analysis. By doing this, users can analyze code at a very low level and look for security issues. There have been many excellent papers on the subject over the years, but these generally don’t come with learning tools in a self-contained, easy-to-use environment.”

Thomas Wilhelm is the author of “Professional Penetration Testing: Creating and Operating a Formal Hacking Lab” and the creator of both the Hackerdemia project and the De-ICE.net Pentest LiveCDs project. Hackerdemia is a LiveCD that containing several vulnerabilities, including un-patched software, mis-configured services, default passwords and a few other surprises. Paul Asadoorian posted “Scanning Vulnerable Linux Distributions With Nessus” where he walks through using Nessus to determine the vulnerabilities within Hackerdemia. The De-ICE.net Pentest LiveCDs are disk images that are fully-functioning server. The Security Aegis site has an interview with Thomas where he discusses these projects along with the Heorot.net pentest video training and his recently published book.

One more Linux VM intentionally configured with exploitable services pWnOS. It was created by Brady Bloxham, a.ka bond00. Below is a nice introduction video.

The Web Application Attack and Audit Framework (w3af) project has created a VMware image, called Moth, which is a set of vulnerable Web Applications and scripts. The w3af core and it’s plugins are fully written in Python, has more than 130 plugins checking for SQL injection, cross site scripting (xss), local and remote file inclusion, etc. What is really interesting about Moth is that it allows for testing of web application scanners and learning how web application firewalls work by providing a way access web applications and vulnerable scripts directly, through mod_security, and through PHP-IDS.

On the system side, LAMPSecurity has been creating a series of capture the flag exercise that uses a full Linux virtual machine that is vulnerable to remote root compromise due to a number of vulnerabilities. The most recent exercise is Capture the Flag 6 and was released 7/17/2009. The documentation will take you through the steps of the exercise.

Training Ground

The Mighty Seek Podcast did a Hands On Series and setup the NTO Hackme Test site, which includes the podcasts with the opportunity to test what is discussed out. Dan Kuykendall did two episodes: “Episode #01 – SQL Injection Part 1 [Intro]” and “Episode #02 – Cross Site Scripting (XSS) Part 1 [Intro].”

Hack This Site (HTS) is a website to test and expand one’s hacking skills. You will need to register with the site to access the hacker challenges.  There are various lessons and missions.  User cwade12c has posted the several video tutorial covering missions. Below is “Hack This Site – Basic 1 Tutorial” to give you an feel for the simplest of challenges:

HellBound Hackers (HBH) is another site offering a large resource consisting of challenges, articles, forums, etc. The LifeofaHacker site has published some challenge tutorials/walkthrough guides for both Hack This Site (HTS) and HBH.

Enigma Group is similar to HTS and HBH in terms of tutorials, articles, and hacker challenges. There are some education and humorous short tutorial videos.

The Bright Shadows site also offers challenges on JavaScript, cyptography, cracking, steganography, Flash, Java, various programming exploits, etc. Registration is required. The challenges get voted on by the members in terms of difficulty, creativity, education, and presentation.

Smash The Stack (StS) Wargaming Network has a progression of challenges where each challenge is dependent on the completion of the previous challenge. The challenges are *nix based. To get started you ssh into one one of the wargame servers on port 2224 using password “level1″, at which point you receive a message letting you know how to get started. The password for the next level will be located in different placed, depending on the game. Questions can be asked on their forums area. OverTheWire offer similar wargame challenges.

A Few Final Thoughts

The above list represents a few source I have experience with. Duncan Alderson on his site Webantix has done a great job of listing war games/hacking simulators in his post, “War Games. Current and past hacking simulators and challenges. The New Order site also has a much more comprehensive list.

Just remember, it is good to be paranoid. Even HTS, with a user base of over 1,300,000 can still have problems with disgruntled and past employees. We are talking very skilled, intelligent, and disgruntle employees. In the last major attack, root-level access to the website was gained and HTS was taken down for months.

It is a dangerous world. That is exactly why skilled ethical hackers are needed. One of my college professors would always say, “Repetition is the key to learning.” He repeated it so many time, I finally learned that lesson. The above links help provide a challenging way to practice and learn. Give them a try and have some fun.

The Security Content Automation Protocol (SCAP) is an attempt to help defenders by providing a collection of XML schemas/standards that allow technical security information to be exchanged between tools. For example, SCAP can help organizations looking for a way to respond appropriately to new vulnerabilities and threats by helping prioritize, allowing the most significant ones to be addressed sooner. It can also benefit those looking to provide interoperability across system security tools. There is even an effort to “encouraging the use of SCAP as a de-facto standard across the ICT industry for deploying trusted cloud computing services.”

Background

To help understand what exactly SCAP is, let us turn to the U.S. National Institute of Standards and Technology (NIST) Special Publications (SP) 800-117, “DRAFT Guide to Adopting and Using the Security Content Automation Protocol (SCAP):”

SCAP comprises a suite of specifications for organizing and expressing security-related information in standardized ways, as well as related reference data, such as identifiers for software flaws and security configuration issues. SCAP can be used for maintain the security of enterprise systems, such as automatically verifying the installation of patches, checking systems security configuration settings, and examining systems for signs of compromise.

NIST this month is looking for public comments on the first public draft of SP 800-126, “The Technical Specification for the Security Content Automation Protocol (SCAP).” Back in May, NIST released the draft for SP 800-117.

SCAP components consists of:

• Common Configuration Enumeration (CCE): provide unique identifiers to system configuration issues in order to facilitate fast and accurate correlation of configuration data across multiple information sources and tools.
• Common Platform Enumeration (CPE): a structured naming scheme for information technology systems, platforms, and packages.
• Common Vulnerability Enumeration (CVE): a dictionary of publicly known information security vulnerabilities and exposures.
• Common Vulnerability Scoring System (CVSS): a vulnerability scoring system designed to provide an open and standardized method of rating IT vulnerabilities. NIST has even provided a calculator for creating CVSS vulnerability severity scores.
• eXtensible Checklist Configuration Description Format (XCCDF): a specification language for writing security checklists, benchmarks, and related kinds of documents. NIST has released the NIST Interagency Report 7275 Revision 3 “Specification for Extensible Configuration Checklist Description Format (XCCDF) Version 1.1.4.”
• Open Vulnerability Assessment Language (OVAL): an information security community standard to promote open and publicaly available security content, and to standardize the transfer of this information across security tools and services.

The National Checklist Program (NCP), outlined in NIST SP 800-70, is the repository for SCAP-expressed checklists. The checklists provide detailed low level guidance on setting the security configuration of operating systems and applications.

In June, MITRE hosted the Security Automation Developer Days conference, which focused on the U.S. National Institute of Standards and Technology’s (NIST) Security Content Automation Protocol (SCAP). MITRE has made the minutes available, which includes discussion on NIST SP 800-126. Michael Smith has provided some great highlights from the conference in his post, “Security Automation Developers Conference Slides.” The problem with Michael is that it is difficult not to quote his whole blog, which is bad web etiquette. Please follow the link for some real insight concerning the slides. You can also view below Michael’s presentation, “Security Content Automation Protocol and Web Application Security:”

Back in September 2008, NIST sponsored the Fourth Annual Security Automation Conference. The Presentations are available. Ian Charters attended and posted his thoughts, “NIST and SCAP; Busting a cap on intruders Part 1.” The 5th Annual IT Security Automation Conference will be held October 26-30th, 2009 at the Baltimore Convention Center.

Make sure to check out below the OWASP video talk from SnowFROC 2009 by Ed Bellis (from Orbits) on vulnerability management titled “Doing more with less? Automate or die.”

Ed’s has also written an article for CSO Online, “How SCAP Brought Sanity to Vulnerability Management.”

Possible Problems

Some may argue that SCAP is overly complicated and people are better off relying solely on their vendor’s products and reports. That assumes that a single vendor product is sufficient to meet tomorrow’s security needs. Some organizations buy into the platform simplification model where basically they purchase a single vendor line of products in order to avoid interoperability problems. The problems is that one vendor frequently only does a few things well. The agility of the organization to adapt to changes in the security world becomes dependent solely on that single vendor. After investing so much into that one vendor, organizations find that they are completely locked in. Probably that is not the best position to be in when facing a very volatile IT environment.

Consider the below list where NIST outlines areas SCAP validation will cover (Source: NIST Interagency Report 7511 “Security Content Automation Protocol (SCAP) Version 1.0 Validation Program Test Requirements (DRAFT)):”

• FDCC Scanner: the capability to audit and assess a target system to determine its compliance with the FDCC requirements.
• Authenticated Configuration Scanner: the capability to audit and assess a target system to determine its compliance with a defined set of configuration requirements using target system logon privileges.
• Authenticated Vulnerability and Patch Scanner: the capability to scan a target system to locate and identify the presence of known vulnerabilities and evaluate the software patch status to determine compliance with a defined patch policy using target system logon privileges.
• Unauthenticated Vulnerability Scanner: the capability of determining the presence of known vulnerabilities by evaluating the target system over the network.
• Intrusion Detection and Prevention System (IDPS): the capability to monitor a system or network for unauthorized or malicious activities. An intrusion prevention system actively protects the target system or network against these activities.
• Vulnerability Remediation: the capability to install patches on a target system in compliance with a defined patching policy.
• Misconfiguration Remediation: the capability to alter the configuration of a target system to bring it into compliance with a defined set of configuration recommendations.
• Asset Scanner: the capability to actively discover, audit, and assess asset characteristics including: installed and licensed products; location within the world, a network or enterprise; ownership; and other related information on IT assets such as workstations, servers, and routers.
• Asset Database: the capability to store and report on asset characteristics including: installed and licensed products; location within the world, a network or enterprise; ownership; and other related information on IT assets such as workstations, servers, and routers.
• Vulnerability Database: a catalog of security-related software flaws labeled with CVEs where applicable. This data is made accessible to users through a search capability or data feed and contains descriptions of software flaws, references to additional information (e.g., links to patches or vulnerability advisories), and impact scores. The user-to-database interaction is provided independent of any scans, intrusion detection, or reporting activities. Thus, a product that only scans to find vulnerabilities and then stores the results in a database does not meet the requirements for an SCAP vulnerability database (such a product would map to a different SCAP capability). A product that presents the user general knowledge about vulnerabilities, independent of a particular environment, would meet the definition of an SCAP vulnerability database.
• Misconfiguration Database: a catalog of security-related configuration issues labeled with CCEs where applicable. This data is made accessible to users through a search capability or data feed and contains descriptions of configuration issues and references to additional information (e.g., configuration guidance, mandates, or other advisories). The user-to-database interaction is provided independent of any configuration scans or intrusion detection activities. Thus, a product that only scans to find misconfigurations and then stores the results in a database does not meet the requirements for an SCAP misconfiguration database (such a product would map to a different SCAP capability). A product that presents the user general knowledge about security-related configuration issues, independent of a particular environment, would meet the definition of an SCAP vulnerability database.
• Malware Tool: the capability to identify and report on the presence of viruses, worms, Trojan horses, spyware, or other malware on a target system.

It is difficult to imagine a single security product that is capable of doing all the above services well. There is a need to be able to share information between various systems performing these functions.

Game Changing Technology

Considering past statements by Aneesh Chopra, the first Chief Technology Officer of the United States, does not SCAP sound like an area that will be getting additional support by the U.S. government? ZDnet has posted a very interesting podcast of Chopra talking at the Computer History Museum. Chopra wrote a few months back:

If confirmed, I would emphasize a research program on “game-changing” ideas in cybersecurity, to find new ideas that might transform the nation’s information infrastructure to be more secure and simpler to understand and use. The goal is to make it “easy to do the right thing, hard to do the wrong things and easy to recover when the wrong thing happens anyway.”

Tim O’Reilly, one of the most insightful person around in respect to IT, wrote back in April “Why Aneesh Chopra is a Great Choice for Federal CTO.” Tim’s points out items that Chopra has accomplished in Virginia:

1. the first officially-approved open source textbook in the country, the Physics Flexbook;
2. integrating iTunes U with Virginia’s state education assessment framework;
3. the Learning Apps Development Challenge, a competition for the best iPhone and iPod Touch applications for middle-school math teaching;
4. a Ning-based social network to connect clinicians working in small health care offices in remote locations;
5. a state-funded “venture capital fund” to allow government agencies to try out risky but promising new approaches to delivering their services or improving their productivity;
6. a lightweight approval and testing process that allows the government to try out new technologies before making a full, expensive commitment.

Back in April 2007, Chopra was behind Virginia’s 95 agencies opening up their databases to the Google search engine, in order to make them widely accessible to the public. Chopra at that time stated the top priority of the state’s strategic plan for information technology, which was adopted last year, is increased access to government information. A great thing to do, provided security is insuring only the information you want is being accessed in the manner intended.

John Dvorak offers a different opinion of Chopra in his post “Special Report: Is US Chief Information Officer (CIO) Vivek Kundra a Phony?” Dvorak states, “It would be logical to assume that Kundra managed to get his buddy Chopra the CTO job despite the fact that Chopra’s technology background is essentially nil.” Whether O’Reilly or Dvorak is correct, Chopra needs to start reading the Guerilla CISO for great insight into security solutions. Michael outlines a plan on fixing government patch and vulnerability management through SCAP in the post, “Federated Vulnerability Management.” Here are a few of the ideas discussed in the post:

• Every IT asset reports into a patch management system of some sort. Group the assets allowing for identification of who is responsible when something has a problem.
• Do periodic network scanning.
• The orchestrator will correlate network scans with patch management status and gives a ticketing/alert/whatever where unmanaged devices are identified.
• The NVD feed is pushed down to the agencies/departments which are sent out as vulnerability alerts along with the checks to see if systems are vulnerable.
• Hardening guides are pushed from the agencies/departments in SCAP form and audit information is pulled of IT assets. Differences are automatically entered into a workflow and reporting system.

Imagine the additional possibilities when intrusion detection/prevention systems, patch remediation, asset scanner, and malware tools start sharing information.

SCAP and the Cloud

Aneesh Chopra should also read Christofer Hoff’s rational Survivability blog. In Hoff’s post, “Extending the Concept: A Security API for Cloud Stacks“, he considers building on the capabilities of SCAP to embed a “standardized and open API layer into each IaaS, PaaS and SaaS offering (see the API blocks in the diagram below) to provide not only a standardized way of scanning for network vulnerabilities, but also configuration management, asset management, patch remediation, compliance, etc.” Hoff goes on to write, “Further (HT to @davidoberry who reminded me about my posts on the topic) we could use TCG IF-MAP as a comms. protocol for telemetry.”

Hoff is another person who is difficult to quote without including his complete post. He makes the point that you gain “automated audit and security management capability for the customer/consumer and a a streamlined, cost effective, and responsive way of automating the validation of said controls in relation to compliance, SLA and legal requirements for service providers.” By doing so, Hoff points out, you are “not reinventing the wheel and we have lots of technology and standardized solutions we can already use to engineer into the stack.”

Update: Hoff pointed out (see comments area) some of the excellent work done by Iron Frog (Ben) in not only his post “Some thoughts for addressing the Assurance component of A6,” but also his series of post “Can we do the Security Stack API RESTfully? (parts 1, 2, 3, 4, and 5).”

Peter Mell, who recently changed positions at NIST from the SCAP validation program manager to the leader of the agency’s Cloud computing project, will likely agree with Hoff’s points. Expect NIST efforts in the Cloud to take SCAP into consideration.

Final Thoughts

As Michael Smith points out, in the Cloud one faces the same problems as a managed service provider, mainly how to allow the auditing of systems and the underlying infrastructure. An API could allow a managed services environment making security tasks much easier to customers. To quote Michael Smith, “we have in SCAP is Common Platform Enumeration (CPE) which allows you to specify the hardware and software (ie, how the infrastructure that you don’t know about is built) and eXtensible Configuration Checklist Description (XCCDF) which specifies the audit/compliance checks. Package them together and you have a way of describing what the infrastructure looks like and the technical auditing standard to go along with it.” Sounds like some game changing ideas that could transform the nation’s information infrastructure, helping it be more secure. I hope you are listening, Aneesh Chopra.

The original slides made heavy use of the Microsoft PowerPoint animation feature. Unfortunately, SlideShare does not currently support animation. You can download the presentation and the animations will work, but I ended up modifying the slides so they are more viewable online. SlideBoom will keep the animation, but it does it by creating a video of the presentation. I decided to stick with SlideShare and spare you the resulting nine minute video. While I should add audio and make a SlideCast, this post might never be completed if I wait until I have time to create a really nice web presentation.

Merriam-Webster defines a totem as any supposed entity that watches over or assists a group of people, such as a family, clan, or tribe. In this presentation I focused on how TOTEM assists in watching over and evaluating the threat an IP represents. The idea behind TOTEM is simple: compare threat information from sources such as watchlists (DShield, Emerging Threats, SenderBase, etc.) to activities with the organization (IDS/IPS, flow logs, etc.) and other locations (SANS ISC, DOE federated model, etc.). As new threat information and activity sources are added, a better evaluation can be rendered.

The purpose of this presentation has been to share the basic ideas behind TOTEM with the hope that others may provide helpful insight. So far I have not disappointed. I wanted to thank everyone for I have received some very intriguing ideas.

To deal with administrative accounts at Twitter, Adam O’Donnell provides some great advices to corporate CSOs in his article, “A roadmap for the Twitter CSO.” Dave Goldsmith post, “My Pentest Secret: Password Guessing,” provides more advice to mitigate risk of password guessing attacks.

Today’s post focuses on Dave’s point:

FAILED LOGIN DELAYS. What to do when someone is grinding passwords on the same account? Account lockout is pretty unpopular as it can lead to a denial of service attack. Doing nothing is pretty unpopular because attackers can grind forever. Enter the exponentially increasing login delay. Every failed login on an account causes the system to delay more and more on that account until a reset on that counter after a reasonable period of time or a valid login.

The Open Web Application Security Project (OWASP) has begun a podcast focused on web application security. The podcast is hosted by Jim Manico, a Web Application Architect and Security Engineer for Aspect Security. In Podcast #2, Stephen Craig Evans, an independent software security consultant, talks about Lua and the OWASP Summer of Code project wiki, Securing WebGoat using ModSecurity. The project goes through the steps involved in securing WebGoat using the combination of ModSecurity and Lua. To quote Ivan Ristic, creator of ModSecurity, the project “stretched the boundaries of what ModSecurity could do.”

To help address the problem of dictionary attacks against your web server, today’s post will be using ModSecurity with the scripting language Lua. First, let’s setup WebGoat for testing purposes.

WebGoat

For those unfamiliar with OWASP WebGoat project, WebGoat is:

WebGoat is a deliberately insecure J2EE web application maintained by OWASP designed to teach web application security lessons. In each lesson, users must demonstrate their understanding of a security issue by exploiting a real vulnerability in the WebGoat application. For example, in one of the lessons the user must use SQL injection to steal fake credit card numbers. The application is a realistic teaching environment, providing users with hints and code to further explain the lesson.

Why the name “WebGoat”? Developers should not feel bad about not knowing security. Even the best programmers make security errors. What they need is a scapegoat, right? Just blame it on the ‘Goat!

WebGoat is intended to teach a structured approach to test and exploiting vulnerabilities within an application security assessment. Also being developed is the OWASP Testing Project, which provides a full application security assessment testing methodology. Through WebGoat, companies have a way to teach web application security lessons to their developers. There are over 30 lessons dealing with such issues as:

The YGN Ethical Hacker Group has made available a series of video on walking through WebGoat v5.2. The videos consist of:

Please recognize that WebGoat is a vulnerable server and therefor you will want to set it up so no one but you can access the WebGoat server. By default WebGoat only listens on the loopback address. Below are the steps to pull down WebGoat and install it on a Linux server. Since this is not a production system, we will be installing the WebGoat developer release.

Installing JDK

WebGoat will require Sun JDK 6 to be installed. Get the Sun JDK 6 from Sun’s website. Sun requires you to agree to terms, so you’ll need to go there and agree. Run the installer which gets downloaded. Agree again to the terms. The installer will install a few rpms and jars.

root# /bin/sh jdk-6u11-linux-i586-rpm.bin
root# ls -la /usr/java
default  jdk1.5.0_17  jdk1.6.0_11  latest
root# declare -x JAVA_HOME="/usr/java/latest"
root# declare -x PATH="${JAVA_HOME}/bin:${PATH}
root# java -version
java version "1.6.0_11"
Java(TM) SE Runtime Environment (build 1.6.0_11-b03)
Java HotSpot(TM) Server VM (build 11.0-b16, mixed mode)

Running WebGoat Standard Release

While the documentation for WebGoat says to install Tomcat, the WebGoat zip file will come with its own version of Tomcat. Running WebGoat in this manner can prove to be the easiest path allowing the avoidence of Java software version problems. We will go through both deployments. First, installing WebGoat with Tomcat.

 root# cd /usr/local/src
/usr/local/src root# wget \

http://webgoat.googlecode.com/files/WebGoat-OWASP_Standard-5.2.zip

/usr/local/src root# /usr/bin/openssl sha1 WebGoat-OWASP_Standard-5.2.zip
SHA1(WebGoat-OWASP_Standard-5.2.zip)=
1e8950d8af0a1726ee1c4509cb64ee4ee6da7584
/usr/local/src root# unzip WebGoat-OWASP_Standard-5.2.zip
/usr/local/src root# cd WebGoat-5.2

At this point, a slight modification needs to be made to webgoat.sh. It checks if the java version is 1.5. This is an odd check, since WebGoat was compiled under 1.6 and will not run under 1.5. Find where webgoat.sh has grep ‘version \”1.5′ and change 1.5 to 1.6. At that point, you are read to start WebGoat.

/usr/local/src/WebGoat-5.2 root# /bin/sh webgoat.sh start8080
Using CATALINA_BASE:   ./tomcat
Using CATALINA_HOME:   ./tomcat
Using CATALINA_TMPDIR: ./tomcat/temp
Using JAVA_HOME:       /usr/java/latest

  Open http://127.0.0.1:8080/WebGoat/attack
  Username: guest
  Password: guest
  Or try http://guest:guest@127.0.0.1:8080/WebGoat/attack

This is running WebGoat accessible only to 127.0.0.1. From your web browser, go to http://127.0.0.1:8080/WebGoat/attack and log in with username guest and password guest. At this point, WebGoat is running and you are set to start going through the lesson plans and exercise.

Installing WebGoat.war

If you have Tomcat on your server, you will want to install only the WebGoat.war file. I am going to make this a little more complicated by going through the steps to install Tomcat. In previous posts, I have stepped through installation of Apache and ModSecurity. Walking through the installation of Tomcat will help get us on the same page as far as configuration and installation.

There is a known issue with the latest stable release of Tomcat, 6.0.18. It requires JDK 5 at the moment, due to incompatibilities introduced by Sun JDK 6. Sun changed the JDBC spec in an incompatible fashion that was discovered after Tomcat 6 went out. There are changes in the trunk to replace the DB connection pooling mechanism with one that isn’t impacted by the 1.6 change. Unfortunately, WebGoat required JDK 6. To get a round this problem, we will use the subversion release of Tomcat.

We first need to install Apache Ant, which is a software tool for automating software build processes. It is similar to make but is implemented using the Java language.

root# cd  /usr/local/src/
/usr/local/src root# wget http://www.uniontransit.com/apache/ant/binaries/apache-ant-1.7.1-bin.tar.gz
/usr/local/src root# md5sum apache-ant-1.7.1-bin.tar.gz
cc5777c57c4e8269be5f3d1dc515301c  apache-ant-1.7.1-bin.tar.gz
/usr/local/src root# tar xzf apache-ant-1.7.1-bin.tar.gz
/usr/local/src root# mv apache-ant-1.7.1 /work/software
/usr/local/src root# cd  /work/software
/work/software root# ln -s apache-ant-1.7.1 ant
/work/software root# declare -x ANT_HOME="/work/software/ant"
/work/software root# declare -x PATH="${PATH}:${ANT_HOME}/bin"
/work/software root# ant
Buildfile: build.xml does not exist!
Build failed

The error above indicate that ant command is recognized by shell but it did not find build.xml file that needed to compile ant projects. So, it’s absolutely normal and the installation was successful.

We are now ready to download and build Tomcat.

root# cd  /usr/local/src/
/usr/local/src root# svn checkout \
http://svn.apache.org/repos/asf/tomcat/trunk tomcat-7
/usr/local/src root# cd tomcat-7
/usr/local/src/tomcat-7 root# ant download
/usr/local/src/tomcat-7 root# ant
/usr/local/src/tomcat-7 root# cd output
/usr/local/src/tomcat-7/output root# mv build \
/work/software/tomcat-7
/usr/local/src/tomcat-7 root# cd /work/software/
/work/software root# ln -s tomcat-7 tomcat
/work/software root# declare -x CATALINA_HOME="/work/software/tomcat"
/work/software root# chmod u+x $CATALINA_HOME/bin/*
/work/software root# mkdir $CATALINA_HOME/logs

Configuring and Using Tomcat

The following modifications can be done to configuration Tomcat files that came down as part of WebGoat or the latest version from the Tomcat site. If you make the modifications to the Tomcat under Webgoat, keep in mind:

1. Set $CATALINA_HOME appropriately
2. Modify webgoat.sh, removing the line “cp -f $CATALINA_HOME/conf/server_8080.xml $CATALINA_HOME/conf/server.xml”.
3. Adjust paths based on where WebGoat is installed.
4. Add to the $CATALINA_HOME/conf/tomcat-users.xml file. Do not replace content.
5. You can use $CATALINA_HOME/bin/shutdown.sh to shutdown Tomcat.
6. You can start with $CATALINA_HOME/bin/startup.sh instead of webgoat.sh, provided you have $PATH, $JAVA_HOME, and $CATALINA_HOME set.

Before starting, create a manager username and password. This is set in the $CATALINA_HOME/conf/tomcat-users.xml file. The following is an example only:

/work/software root# vi $CATALINA_HOME/conf/tomcat-users.xml
<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
  <role rolename="tomcat"/>
  <role rolename="manager"/>
  <user username="jerry" password="mousepower" roles="tomcat"/>
  <user username="tom" password="catpower" roles="manager"/>
</tomcat-users>

We can setup secure web authentication through the use of digital certificates using SSL. First step is to use the keytool utility, which is included in the Sun Java Standard Edition JDK, to create a keystore file. Use “changeit” as the password. (If you don’t use “changeit” you will have to state the password in with the keystorePass setting in server.xml). For example:

/work/software root# mkdir /work/software/tomcat/keystore
/work/software root# cd /work/software/tomcat/keystore
/work/software/tomcat/keystore root# keytool -genkey -alias tomcat \
-keyalg RSA -keysize 2048 -keystore /work/software/tomcat/keystore/keystore
Enter keystore password:  changeit
What is your first and last name?
  [Unknown]:  John Gerber
What is the name of your organizational unit?
  [Unknown]:  SecurityMonks
What is the name of your organization?
  [Unknown]:  OrderOfUnix
What is the name of your City or Locality?
  [Unknown]:  Knoxville
What is the name of your State or Province?
  [Unknown]:  TN
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=John Gerber, OU=SecurityMonks, O=OrderOfUnix, L=Knoxville, ST=TN, C=US correct?
  [no]:  yes

Enter key password for 
        (RETURN if same as keystore password):
/work/software/tomcat/keystore root# ls  /work/software/tomcat/keystore
/work/software/tomcat/keystore root# keytool -list -keystore keystore

The keystore created will not be trusted by JVM until the certificate is imported into JVM’s trusted certificate keystore. We will export the SSL certificate we just generated and import it into the JVM’s keystore.

/work/software/tomcat/keystore root# keytool -export \
-alias tomcat -keystore keystore -file tomcat.cer
/work/software/tomcat/keystore root# keytool -import \
-trustcacerts -keystore /usr/java/jdk1.5.0_17/jre/lib/security/cacerts  \
-alias tomcat -file /work/software/tomcat/keystore/tomcat.cer
/work/software/tomcat/keystore root# keytool -list -alias tomcat \
-keystore /usr/java/jdk1.5.0_17/jre/lib/security/cacerts

Modify the $CATALINA_HOME/conf/server.xml section which defines a SSL HTTP/1.1 Connector on port 8443. It should go with the other connectors in the Service section and looks something like this:

    <Connector port="8443" minSpareThreads="5" maxSpareThreads="75"
           enableLookups="true" disableUploadTimeout="true"
           acceptCount="100"  maxThreads="200"
           scheme="https" secure="true" SSLEnabled="true"
           keystore="/work/software/tomcat/keystore/keystore"  keypass="changeit"
           clientAuth="false" sslProtocol="TLS" />

In order to require SSL on a specific site configure a security constrant for that app. Do this by editing the $CATALINA_HOME/conf/web.xml file and adding the following section just before the ending </web-app> tag:

    <security-constraint>
    <web-resource-collection>
    <web-resource-name>Automatic SSL Forwarding</web-resource-name>
    <url-pattern>/*</url-pattern>
    </web-resource-collection>
    <user-data-constraint>
    <transport-guarantee>CONFIDENTIAL</transport-guarantee>
    </user-data-constraint>
    </security-constraint>

Now startup Tomcat.

/work/software root# $CATALINA_HOME/bin/startup.sh
Using CATALINA_BASE:   /work/software/tomcat
Using CATALINA_HOME:   /work/software/tomcat
Using CATALINA_TMPDIR: /work/software/tomcat/temp
Using JRE_HOME:       /usr/java/jdk1.5.0_17
/work/software root# ps awx | grep tomcat
  783 pts/14   Sl     0:04 /usr/java/jdk1.5.0_17/bin/java
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=/work/software/tomcat/conf/logging.properties
-Djava.endorsed.dirs=/work/software/tomcat/endorsed
-classpath :/work/software/tomcat/bin/bootstrap.jar
-Dcatalina.base=/work/software/tomcat
-Dcatalina.home=/work/software/tomcat
-Djava.io.tmpdir=/work/software/tomcat/temp org.apache.catalina.startup.Bootstrap start
/work/software root# /usr/sbin/lsof -iTCP -n -P | grep java
java      6175   root   10u  IPv6 327556       TCP *:8080 (LISTEN)
java      6175   root   12u  IPv6 327557       TCP *:8443 (LISTEN)
java      6175   root   21u  IPv6 327562       TCP *:8009 (LISTEN)
java      6175   root   22u  IPv6 327565       TCP 127.0.0.1:8005 (LISTEN)

Tomcat is now built and running. You can access it by going to:

http://localhost:8080

which will redirect you to:

https://localhost:8443

When you goto https://localhost:8443 you will be asked to accept the certificate. If you have problems, make sure to clear you browser’s cache. Now that we have Tomcat server running, we are ready to download and setup our WebGoat server.

 root# $CATALINA_HOME/bin/shutdown.sh
 root# cd /usr/local/src
/usr/local/src root# wget http://webgoat.googlecode.com/files/WebGoat-5.2.war
/usr/local/src root# /usr/bin/openssl sha1 WebGoat-5.2.war
SHA1(WebGoat-5.2.war)= c5aab7c5496625777a3b9e21b9888cddee5b649c
/usr/local/src root# mv WebGoat-5.2.war /work/software/tomcat/webapps/WebGoat.war
/usr/local/src root# $CATALINA_HOME/bin/startup.sh
/usr/local/src root# ls -la /work/software/tomcat/webapps/WebGoat
/usr/local/src root# ls -la /work/software/tomcat/conf/tomcat-users.xml

Add WebGoat users and roles to $CATALINA_HOME/conf/tomcat-users.xml file. Start Tomcat back up.

 root# $CATALINA_HOME/bin/shutdown.sh
 root# vi $CATALINA_HOME/conf/tomcat-users.xml
    <tomcat-users>
      <role rolename="webgoat_basic"/>
      <role rolename="webgoat_admin"/>
      <role rolename="webgoat_user"/>
      <role rolename="tomcat"/>
      <user password="webgoat" roles="webgoat_admin" username="webgoat"/>
      <user password="basic" roles="webgoat_user,webgoat_basic" username="basic"/>
      <user password="tomcat" roles="tomcat" username="tomcat"/>
      <user password="guest" roles="webgoat_user" username="guest"/>
    </tomcat-users>
 root# $CATALINA_HOME/bin/startup.sh

At this point, WebGoat should be running. Pretend you are an 18-year hacker, and use your penetration skills to break into one of the accounts. Check out the Access Control Flaws material and the Remote Admin Access section. Being aware of what is possible, and that the threats are real, helps motivate a person to defend against them.

Now it is time to examine the work of OWASP Summer of Code project wiki, Securing WebGoat using ModSecurity. With a vulnerable server to test out the vulnerabilities against, we will move on to the required software that will help defend against brute force password attacks. First step, install Lua to use with ModSecurity.

Lua

The Lua language combines “simple procedural syntax with powerful data description constructs based on associative arrays and extensible semantics. Lua is dynamically typed, runs by interpreting bytecode for a register-based virtual machine, and has automatic memory management with incremental garbage collection, making it ideal for configuration, scripting, and rapid prototyping.”

For a complete discussion of the benefits of Lua, Soumya has written an article “10 Reasons Why You Should Make Lua (A New Programming Language) Your Coding Friend – A Detailed Review.” Erik Wrenholt has also done an interesting benchmark again popular languages to compute the Mandelbrot. We are focusing on Lua is because it can be used with ModSecurity. Plus, we want to take advantage of the work done by Stephen Craig Evans and others who worked on securing web applications.

To install Lua on a linux server by source:

root# cd /usr/local/src/ /usr/local/src root# wget http://www.lua.org/ftp/lua-5.1.4.tar.gz /usr/local/src root# md5sum lua-5.1.4.tar.gz d0870f2de55d59c1c8419f36e8fac150 lua-5.1.4.tar.gz /usr/local/src root#tar xzf lua-5.1.4.tar.gz /usr/local/src root# cd lua-5.1.4 /usr/local/src/lua-5.1.4 root# make linux /usr/local/src/lua-5.1.4 root# make install /usr/local/src/lua-5.1.4 root# cd /usr/local/lib /usr/local/lib root# gcc -shared -o liblua.5.1.4.so /usr/local/lib/liblua.a /usr/local/lib root# ln -s liblua.5.1.4.so liblua.so

ModSecurity

If you are running a version of ModSecurity older than version 2.5, you will need to upgrade. As of ModSecurity 2.5, Lua can be used:

The new SecRuleScript directive allows for the execution of Lua scripts which provide an even more flexible and powerful interface into ModSecurity. When is Lua needed? ModSecurity chained rules can easily implement AND logic to create complex rules that evaluate that specific variables are present and have certain data, however they can not easily create proper OR logic. This is where Lua can help.

A previous blog post, “Implementing a Web Application Firewall with ModSecurity,” goes through the step of installing ModSecurity with an Apache Web Server. Following that post, your Apache httpd.conf configuration files should load the mod_security2.so module and include the modsecurity.conf file. It is the modsecurity.conf file where the additional rules will be added.

Problem with Usernames and Passwords

WebGoat demonstrates a few security issues that need to be addressed. From OWASP’s “OWASP ModSecurity Securing WebGoat Section4 Sublesson 04.2” Forgot Password section, the following points are made:

• Attackers who are attempting to enumerate valid usernames. If you submit an invalid one the html response text includes info stating as such. We can track this.
• Once a specific valid username is identified, the attacker then starts a targeted attack to guess answers to the password hint (favorite color).
• Attackers can initiate reverse brute force attacks – this is when an attacker cycles through different valid user accounts and submits the same common answer to the question (submitting Blue as the answer to different usernames).

The OWASP ModSecurity Securing WebGoat document does such a good job outlining the security issues along with possible solutions, I am going to leave it to the reader to decide what solutions are appropriate for their systems. If I continued stepping through to a more secure implementation, I would end up copying everything in the document. Playing around with the rules, testing the results, is great fun in a geeky security kind of way. Do drop down and read the reviewer’s comments. A very good job by Stephen Craig Evans and all who worked on OWASP Summer of Code project. Of course, a special thank to Ivan Ristic, who gave us ModSecurity.

Recently, I listened to a IT Conversation podcast, from the O’Reilly Media Emerging Technology Conference. Tim O’Reilly spoke about hackers. Not the black hatters, but those folks who work tirelessly to bring about the kind of software and services that make the Internet possible. While the Internet may be at times a dangerous place, thanks to the efforts of these hackers who work out of love for the challenge, often with little regard to financial factors, we have these great tools that go a long way towards helping people secure their applications.

Final Thoughts

In today’s post we examined a security breach that occurred involving a major player in the Internet community. To help understand that problem, and others, we setup WebGoat. Sun Tzu wrote in The Art of War, “So it is said that if you know your enemies and know yourself, you will fight without danger in battles. If you only know yourself, but not your opponent, you may win or may lose. If you know neither yourself nor your enemy, you will always endanger yourself.” WebGoat helps us understand the attack vectors against web applications a little better. Once identifying a possible problems, we walked through a solution that can help reduce the risk.

It is easy to find humor in an employee with administrative access using the password “happiness.” The reaction from the Internet community to Twitter’s problems might be a little schadenfreude at play. Theodor Adorno, philosopher and sociologist, defined schadenfreude as largely unanticipated delight in the suffering of another which is cognized as trivial and/or appropriate. Or, maybe it is more like whistling past the graveyard, where folks are a bit cheerful or joyful in the face of a situation that doesn’t warrant it.

When very public security incidents occur, companies need to take a little stock. Not all employees will take training seriously nor follow all policies. That includes people with important roles. Employees make mistakes and it is difficult to guard against every possible mistake that could occur. That is why a layered approach to security is constantly preached. While each layer cost money, security groups at organizations are in a constant battle to monitor and prevent intrusions in a cost effective way. Fortunately, we have the work of many hackers (the good kind), helping us develop solutions to deal with daily new challenges. There are no guarantees. As Albert Einstein once said, “Anyone who has never made a mistake has never tried anything new.” Wisdom comes when we learn from these mistakes. CIOs need to ask themselves, “how safe is my company from being next week’s security headline?” Security groups within an organization must be able to learn and adapt. At the end of the day, the question is how different is your company from Twitter? Insanity is doing the same thing and expecting different results. I’ll close this post with the wise words of Sam Levenson, “You must learn from the mistakes of others. You can’t possibly live long enough to make them all yourself.”

OpenSSL 0.9.8c-1 up to 0.9.8g-9 on Debian-based operating systems uses a random number generator that generates predictable numbers, which makes it easier for remote attackers to conduct brute force guessing attacks against cryptographic keys.

Hubert Seiwert, Internet Security Specialist at Westpoint Ltd., released debian_ssh_scan.py on May 16th. The code does remote check for weak Debian sshd host keys as identified in CVE-2008-0166. The fingerprints are taken from keys generated by HD Moore’s common and uncommon keys. Mr. Seiwert also used Justin Azoff’s multi-threading code. While it is not the only scanner, Mr. Seiwert did a very nice job.

For those who might be less familiar with Python, I thought I would walk through getting debian_ssh_scan.py installed. Most distributions of Linux and Unix have Python installed and with a few additional steps you will be ready to scan your hosts for vulnerabilities.

Set HTTP_PROXY

If you need to access the Internet through a proxy server, the HTTP_PROXY environment variables should be set. This will allow wget, Python’s urllib module, and other applications (yum, apt-get etc) to use this environment variable to access http/https through the proxy server.

# export HTTP_PROXY="http://<proxy-server-ip>:<port>"

Replace “<proxy-server-ip>” with your proxy server name/ip and “<port>” with the proxy’s port.

Install setuptools

The setuptools module is a way to allow developers an easy way to build and distribute Python packages in a single-file archive called an “egg.” The steps to get setuptools installed are:

1. Download the appropriate egg for your version of Python (e.g. setuptools-0.6c8-py2.3.egg). Do NOT rename it.
2. Run it. Setuptools will install itself using the matching version of Python (e.g. python2.3), and will place the easy_install executable in the default location for installing Python scripts (as determined by the standard distutils configuration files, or by the Python installation).

To install:

# cd /home/ger/software
# wget http://pypi.python.org/packages/2.3/s/setuptools/setuptools-0.6c8-py2.3.egg
# sh setuptools-0.6c8-py2.3.egg

Install paramiko

The python module paramiko implements SSH2 protocol for secure (encrypted and authenticated) connections to remote machines. Below, the easy_install executable is used. The Python module easy_install is bundled with setuptools and allows for automatically download, build, install, and management of Python packages.

# cd /home/ger/software
# wget http://www.lag.net/paramiko/download/paramiko-1.7.3.tar.gz
# tar xzf paramiko-1.7.3.tar.gz
# cd paramiko-1.7.3
# easy_install ./

Pull Down debian_ssh_scan_v4

The python script debian_ssh_scan_v4 can now be installed.

# cd /home/ger/software
# wget http://itsecurity.net/debian_ssh_scan_v4.tar.bz2
# bzip2 -cd debian_ssh_scan_v4.tar.bz2 | tar xvf -
 # cd debian_ssh_scan_v4

Start Scanning

You are now ready to start scanning. The below IP is used only for demonstration purposes. Use your own site’s IPs.

#  ./debian_ssh_scan_v4.py 127.0.0.1:22
201691 fingerprints loaded.
127.0.0.1:22 sshd fingerprint 97382c98fe3d45fa779abd34bb65fb73 VULNERABLE (RSA 2048 bit key, pid 5214)

Modify targets.txt, if you want to create a file of IPs. Run the file of IPs through the scan program using the command:

# cat targets.txt | ./debian_ssh_scan_v4.py

Final Words

Debian has issued an update for OpenSSL. For affected systems, the software packages need to be updated and all cryptographic key material must be recreated. Please see Security Focus references for more details.