To deal with administrative accounts at Twitter, Adam O’Donnell provides some great advices to corporate CSOs in his article, “A roadmap for the Twitter CSO.” Dave Goldsmith post, “My Pentest Secret: Password Guessing,” provides more advice to mitigate risk of password guessing attacks.

Today’s post focuses on Dave’s point:

FAILED LOGIN DELAYS. What to do when someone is grinding passwords on the same account? Account lockout is pretty unpopular as it can lead to a denial of service attack. Doing nothing is pretty unpopular because attackers can grind forever. Enter the exponentially increasing login delay. Every failed login on an account causes the system to delay more and more on that account until a reset on that counter after a reasonable period of time or a valid login.

The Open Web Application Security Project (OWASP) has begun a podcast focused on web application security. The podcast is hosted by Jim Manico, a Web Application Architect and Security Engineer for Aspect Security. In Podcast #2, Stephen Craig Evans, an independent software security consultant, talks about Lua and the OWASP Summer of Code project wiki, Securing WebGoat using ModSecurity. The project goes through the steps involved in securing WebGoat using the combination of ModSecurity and Lua. To quote Ivan Ristic, creator of ModSecurity, the project “stretched the boundaries of what ModSecurity could do.”

To help address the problem of dictionary attacks against your web server, today’s post will be using ModSecurity with the scripting language Lua. First, let’s setup WebGoat for testing purposes.

WebGoat

For those unfamiliar with OWASP WebGoat project, WebGoat is:

WebGoat is a deliberately insecure J2EE web application maintained by OWASP designed to teach web application security lessons. In each lesson, users must demonstrate their understanding of a security issue by exploiting a real vulnerability in the WebGoat application. For example, in one of the lessons the user must use SQL injection to steal fake credit card numbers. The application is a realistic teaching environment, providing users with hints and code to further explain the lesson.

Why the name “WebGoat”? Developers should not feel bad about not knowing security. Even the best programmers make security errors. What they need is a scapegoat, right? Just blame it on the ‘Goat!

WebGoat is intended to teach a structured approach to test and exploiting vulnerabilities within an application security assessment. Also being developed is the OWASP Testing Project, which provides a full application security assessment testing methodology. Through WebGoat, companies have a way to teach web application security lessons to their developers. There are over 30 lessons dealing with such issues as:

The YGN Ethical Hacker Group has made available a series of video on walking through WebGoat v5.2. The videos consist of:

Please recognize that WebGoat is a vulnerable server and therefor you will want to set it up so no one but you can access the WebGoat server. By default WebGoat only listens on the loopback address. Below are the steps to pull down WebGoat and install it on a Linux server. Since this is not a production system, we will be installing the WebGoat developer release.

Installing JDK

WebGoat will require Sun JDK 6 to be installed. Get the Sun JDK 6 from Sun’s website. Sun requires you to agree to terms, so you’ll need to go there and agree. Run the installer which gets downloaded. Agree again to the terms. The installer will install a few rpms and jars.

root# /bin/sh jdk-6u11-linux-i586-rpm.bin
root# ls -la /usr/java
default  jdk1.5.0_17  jdk1.6.0_11  latest
root# declare -x JAVA_HOME="/usr/java/latest"
root# declare -x PATH="${JAVA_HOME}/bin:${PATH}
root# java -version
java version "1.6.0_11"
Java(TM) SE Runtime Environment (build 1.6.0_11-b03)
Java HotSpot(TM) Server VM (build 11.0-b16, mixed mode)

Running WebGoat Standard Release

While the documentation for WebGoat says to install Tomcat, the WebGoat zip file will come with its own version of Tomcat. Running WebGoat in this manner can prove to be the easiest path allowing the avoidence of Java software version problems. We will go through both deployments. First, installing WebGoat with Tomcat.

 root# cd /usr/local/src
/usr/local/src root# wget \

http://webgoat.googlecode.com/files/WebGoat-OWASP_Standard-5.2.zip

/usr/local/src root# /usr/bin/openssl sha1 WebGoat-OWASP_Standard-5.2.zip
SHA1(WebGoat-OWASP_Standard-5.2.zip)=
1e8950d8af0a1726ee1c4509cb64ee4ee6da7584
/usr/local/src root# unzip WebGoat-OWASP_Standard-5.2.zip
/usr/local/src root# cd WebGoat-5.2

At this point, a slight modification needs to be made to webgoat.sh. It checks if the java version is 1.5. This is an odd check, since WebGoat was compiled under 1.6 and will not run under 1.5. Find where webgoat.sh has grep ‘version \”1.5′ and change 1.5 to 1.6. At that point, you are read to start WebGoat.

/usr/local/src/WebGoat-5.2 root# /bin/sh webgoat.sh start8080
Using CATALINA_BASE:   ./tomcat
Using CATALINA_HOME:   ./tomcat
Using CATALINA_TMPDIR: ./tomcat/temp
Using JAVA_HOME:       /usr/java/latest

  Open http://127.0.0.1:8080/WebGoat/attack
  Username: guest
  Password: guest
  Or try http://guest:guest@127.0.0.1:8080/WebGoat/attack

This is running WebGoat accessible only to 127.0.0.1. From your web browser, go to http://127.0.0.1:8080/WebGoat/attack and log in with username guest and password guest. At this point, WebGoat is running and you are set to start going through the lesson plans and exercise.

Installing WebGoat.war

If you have Tomcat on your server, you will want to install only the WebGoat.war file. I am going to make this a little more complicated by going through the steps to install Tomcat. In previous posts, I have stepped through installation of Apache and ModSecurity. Walking through the installation of Tomcat will help get us on the same page as far as configuration and installation.

There is a known issue with the latest stable release of Tomcat, 6.0.18. It requires JDK 5 at the moment, due to incompatibilities introduced by Sun JDK 6. Sun changed the JDBC spec in an incompatible fashion that was discovered after Tomcat 6 went out. There are changes in the trunk to replace the DB connection pooling mechanism with one that isn’t impacted by the 1.6 change. Unfortunately, WebGoat required JDK 6. To get a round this problem, we will use the subversion release of Tomcat.

We first need to install Apache Ant, which is a software tool for automating software build processes. It is similar to make but is implemented using the Java language.

root# cd  /usr/local/src/
/usr/local/src root# wget http://www.uniontransit.com/apache/ant/binaries/apache-ant-1.7.1-bin.tar.gz
/usr/local/src root# md5sum apache-ant-1.7.1-bin.tar.gz
cc5777c57c4e8269be5f3d1dc515301c  apache-ant-1.7.1-bin.tar.gz
/usr/local/src root# tar xzf apache-ant-1.7.1-bin.tar.gz
/usr/local/src root# mv apache-ant-1.7.1 /work/software
/usr/local/src root# cd  /work/software
/work/software root# ln -s apache-ant-1.7.1 ant
/work/software root# declare -x ANT_HOME="/work/software/ant"
/work/software root# declare -x PATH="${PATH}:${ANT_HOME}/bin"
/work/software root# ant
Buildfile: build.xml does not exist!
Build failed

The error above indicate that ant command is recognized by shell but it did not find build.xml file that needed to compile ant projects. So, it’s absolutely normal and the installation was successful.

We are now ready to download and build Tomcat.

root# cd  /usr/local/src/
/usr/local/src root# svn checkout \
http://svn.apache.org/repos/asf/tomcat/trunk tomcat-7
/usr/local/src root# cd tomcat-7
/usr/local/src/tomcat-7 root# ant download
/usr/local/src/tomcat-7 root# ant
/usr/local/src/tomcat-7 root# cd output
/usr/local/src/tomcat-7/output root# mv build \
/work/software/tomcat-7
/usr/local/src/tomcat-7 root# cd /work/software/
/work/software root# ln -s tomcat-7 tomcat
/work/software root# declare -x CATALINA_HOME="/work/software/tomcat"
/work/software root# chmod u+x $CATALINA_HOME/bin/*
/work/software root# mkdir $CATALINA_HOME/logs

Configuring and Using Tomcat

The following modifications can be done to configuration Tomcat files that came down as part of WebGoat or the latest version from the Tomcat site. If you make the modifications to the Tomcat under Webgoat, keep in mind:

1. Set $CATALINA_HOME appropriately
2. Modify webgoat.sh, removing the line “cp -f $CATALINA_HOME/conf/server_8080.xml $CATALINA_HOME/conf/server.xml”.
3. Adjust paths based on where WebGoat is installed.
4. Add to the $CATALINA_HOME/conf/tomcat-users.xml file. Do not replace content.
5. You can use $CATALINA_HOME/bin/shutdown.sh to shutdown Tomcat.
6. You can start with $CATALINA_HOME/bin/startup.sh instead of webgoat.sh, provided you have $PATH, $JAVA_HOME, and $CATALINA_HOME set.

Before starting, create a manager username and password. This is set in the $CATALINA_HOME/conf/tomcat-users.xml file. The following is an example only:

/work/software root# vi $CATALINA_HOME/conf/tomcat-users.xml
<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
  <role rolename="tomcat"/>
  <role rolename="manager"/>
  <user username="jerry" password="mousepower" roles="tomcat"/>
  <user username="tom" password="catpower" roles="manager"/>
</tomcat-users>

We can setup secure web authentication through the use of digital certificates using SSL. First step is to use the keytool utility, which is included in the Sun Java Standard Edition JDK, to create a keystore file. Use “changeit” as the password. (If you don’t use “changeit” you will have to state the password in with the keystorePass setting in server.xml). For example:

/work/software root# mkdir /work/software/tomcat/keystore
/work/software root# cd /work/software/tomcat/keystore
/work/software/tomcat/keystore root# keytool -genkey -alias tomcat \
-keyalg RSA -keysize 2048 -keystore /work/software/tomcat/keystore/keystore
Enter keystore password:  changeit
What is your first and last name?
  [Unknown]:  John Gerber
What is the name of your organizational unit?
  [Unknown]:  SecurityMonks
What is the name of your organization?
  [Unknown]:  OrderOfUnix
What is the name of your City or Locality?
  [Unknown]:  Knoxville
What is the name of your State or Province?
  [Unknown]:  TN
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=John Gerber, OU=SecurityMonks, O=OrderOfUnix, L=Knoxville, ST=TN, C=US correct?
  [no]:  yes

Enter key password for 
        (RETURN if same as keystore password):
/work/software/tomcat/keystore root# ls  /work/software/tomcat/keystore
/work/software/tomcat/keystore root# keytool -list -keystore keystore

The keystore created will not be trusted by JVM until the certificate is imported into JVM’s trusted certificate keystore. We will export the SSL certificate we just generated and import it into the JVM’s keystore.

/work/software/tomcat/keystore root# keytool -export \
-alias tomcat -keystore keystore -file tomcat.cer
/work/software/tomcat/keystore root# keytool -import \
-trustcacerts -keystore /usr/java/jdk1.5.0_17/jre/lib/security/cacerts  \
-alias tomcat -file /work/software/tomcat/keystore/tomcat.cer
/work/software/tomcat/keystore root# keytool -list -alias tomcat \
-keystore /usr/java/jdk1.5.0_17/jre/lib/security/cacerts

Modify the $CATALINA_HOME/conf/server.xml section which defines a SSL HTTP/1.1 Connector on port 8443. It should go with the other connectors in the Service section and looks something like this:

    <Connector port="8443" minSpareThreads="5" maxSpareThreads="75"
           enableLookups="true" disableUploadTimeout="true"
           acceptCount="100"  maxThreads="200"
           scheme="https" secure="true" SSLEnabled="true"
           keystore="/work/software/tomcat/keystore/keystore"  keypass="changeit"
           clientAuth="false" sslProtocol="TLS" />

In order to require SSL on a specific site configure a security constrant for that app. Do this by editing the $CATALINA_HOME/conf/web.xml file and adding the following section just before the ending </web-app> tag:

    <security-constraint>
    <web-resource-collection>
    <web-resource-name>Automatic SSL Forwarding</web-resource-name>
    <url-pattern>/*</url-pattern>
    </web-resource-collection>
    <user-data-constraint>
    <transport-guarantee>CONFIDENTIAL</transport-guarantee>
    </user-data-constraint>
    </security-constraint>

Now startup Tomcat.

/work/software root# $CATALINA_HOME/bin/startup.sh
Using CATALINA_BASE:   /work/software/tomcat
Using CATALINA_HOME:   /work/software/tomcat
Using CATALINA_TMPDIR: /work/software/tomcat/temp
Using JRE_HOME:       /usr/java/jdk1.5.0_17
/work/software root# ps awx | grep tomcat
  783 pts/14   Sl     0:04 /usr/java/jdk1.5.0_17/bin/java
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=/work/software/tomcat/conf/logging.properties
-Djava.endorsed.dirs=/work/software/tomcat/endorsed
-classpath :/work/software/tomcat/bin/bootstrap.jar
-Dcatalina.base=/work/software/tomcat
-Dcatalina.home=/work/software/tomcat
-Djava.io.tmpdir=/work/software/tomcat/temp org.apache.catalina.startup.Bootstrap start
/work/software root# /usr/sbin/lsof -iTCP -n -P | grep java
java      6175   root   10u  IPv6 327556       TCP *:8080 (LISTEN)
java      6175   root   12u  IPv6 327557       TCP *:8443 (LISTEN)
java      6175   root   21u  IPv6 327562       TCP *:8009 (LISTEN)
java      6175   root   22u  IPv6 327565       TCP 127.0.0.1:8005 (LISTEN)

Tomcat is now built and running. You can access it by going to:

http://localhost:8080

which will redirect you to:

https://localhost:8443

When you goto https://localhost:8443 you will be asked to accept the certificate. If you have problems, make sure to clear you browser’s cache. Now that we have Tomcat server running, we are ready to download and setup our WebGoat server.

 root# $CATALINA_HOME/bin/shutdown.sh
 root# cd /usr/local/src
/usr/local/src root# wget http://webgoat.googlecode.com/files/WebGoat-5.2.war
/usr/local/src root# /usr/bin/openssl sha1 WebGoat-5.2.war
SHA1(WebGoat-5.2.war)= c5aab7c5496625777a3b9e21b9888cddee5b649c
/usr/local/src root# mv WebGoat-5.2.war /work/software/tomcat/webapps/WebGoat.war
/usr/local/src root# $CATALINA_HOME/bin/startup.sh
/usr/local/src root# ls -la /work/software/tomcat/webapps/WebGoat
/usr/local/src root# ls -la /work/software/tomcat/conf/tomcat-users.xml

Add WebGoat users and roles to $CATALINA_HOME/conf/tomcat-users.xml file. Start Tomcat back up.

 root# $CATALINA_HOME/bin/shutdown.sh
 root# vi $CATALINA_HOME/conf/tomcat-users.xml
    <tomcat-users>
      <role rolename="webgoat_basic"/>
      <role rolename="webgoat_admin"/>
      <role rolename="webgoat_user"/>
      <role rolename="tomcat"/>
      <user password="webgoat" roles="webgoat_admin" username="webgoat"/>
      <user password="basic" roles="webgoat_user,webgoat_basic" username="basic"/>
      <user password="tomcat" roles="tomcat" username="tomcat"/>
      <user password="guest" roles="webgoat_user" username="guest"/>
    </tomcat-users>
 root# $CATALINA_HOME/bin/startup.sh

At this point, WebGoat should be running. Pretend you are an 18-year hacker, and use your penetration skills to break into one of the accounts. Check out the Access Control Flaws material and the Remote Admin Access section. Being aware of what is possible, and that the threats are real, helps motivate a person to defend against them.

Now it is time to examine the work of OWASP Summer of Code project wiki, Securing WebGoat using ModSecurity. With a vulnerable server to test out the vulnerabilities against, we will move on to the required software that will help defend against brute force password attacks. First step, install Lua to use with ModSecurity.

Lua

The Lua language combines “simple procedural syntax with powerful data description constructs based on associative arrays and extensible semantics. Lua is dynamically typed, runs by interpreting bytecode for a register-based virtual machine, and has automatic memory management with incremental garbage collection, making it ideal for configuration, scripting, and rapid prototyping.”

For a complete discussion of the benefits of Lua, Soumya has written an article “10 Reasons Why You Should Make Lua (A New Programming Language) Your Coding Friend – A Detailed Review.” Erik Wrenholt has also done an interesting benchmark again popular languages to compute the Mandelbrot. We are focusing on Lua is because it can be used with ModSecurity. Plus, we want to take advantage of the work done by Stephen Craig Evans and others who worked on securing web applications.

To install Lua on a linux server by source:

root# cd /usr/local/src/ /usr/local/src root# wget http://www.lua.org/ftp/lua-5.1.4.tar.gz /usr/local/src root# md5sum lua-5.1.4.tar.gz d0870f2de55d59c1c8419f36e8fac150 lua-5.1.4.tar.gz /usr/local/src root#tar xzf lua-5.1.4.tar.gz /usr/local/src root# cd lua-5.1.4 /usr/local/src/lua-5.1.4 root# make linux /usr/local/src/lua-5.1.4 root# make install /usr/local/src/lua-5.1.4 root# cd /usr/local/lib /usr/local/lib root# gcc -shared -o liblua.5.1.4.so /usr/local/lib/liblua.a /usr/local/lib root# ln -s liblua.5.1.4.so liblua.so

ModSecurity

If you are running a version of ModSecurity older than version 2.5, you will need to upgrade. As of ModSecurity 2.5, Lua can be used:

The new SecRuleScript directive allows for the execution of Lua scripts which provide an even more flexible and powerful interface into ModSecurity. When is Lua needed? ModSecurity chained rules can easily implement AND logic to create complex rules that evaluate that specific variables are present and have certain data, however they can not easily create proper OR logic. This is where Lua can help.

A previous blog post, “Implementing a Web Application Firewall with ModSecurity,” goes through the step of installing ModSecurity with an Apache Web Server. Following that post, your Apache httpd.conf configuration files should load the mod_security2.so module and include the modsecurity.conf file. It is the modsecurity.conf file where the additional rules will be added.

Problem with Usernames and Passwords

WebGoat demonstrates a few security issues that need to be addressed. From OWASP’s “OWASP ModSecurity Securing WebGoat Section4 Sublesson 04.2” Forgot Password section, the following points are made:

• Attackers who are attempting to enumerate valid usernames. If you submit an invalid one the html response text includes info stating as such. We can track this.
• Once a specific valid username is identified, the attacker then starts a targeted attack to guess answers to the password hint (favorite color).
• Attackers can initiate reverse brute force attacks – this is when an attacker cycles through different valid user accounts and submits the same common answer to the question (submitting Blue as the answer to different usernames).

The OWASP ModSecurity Securing WebGoat document does such a good job outlining the security issues along with possible solutions, I am going to leave it to the reader to decide what solutions are appropriate for their systems. If I continued stepping through to a more secure implementation, I would end up copying everything in the document. Playing around with the rules, testing the results, is great fun in a geeky security kind of way. Do drop down and read the reviewer’s comments. A very good job by Stephen Craig Evans and all who worked on OWASP Summer of Code project. Of course, a special thank to Ivan Ristic, who gave us ModSecurity.

Recently, I listened to a IT Conversation podcast, from the O’Reilly Media Emerging Technology Conference. Tim O’Reilly spoke about hackers. Not the black hatters, but those folks who work tirelessly to bring about the kind of software and services that make the Internet possible. While the Internet may be at times a dangerous place, thanks to the efforts of these hackers who work out of love for the challenge, often with little regard to financial factors, we have these great tools that go a long way towards helping people secure their applications.

Final Thoughts

In today’s post we examined a security breach that occurred involving a major player in the Internet community. To help understand that problem, and others, we setup WebGoat. Sun Tzu wrote in The Art of War, “So it is said that if you know your enemies and know yourself, you will fight without danger in battles. If you only know yourself, but not your opponent, you may win or may lose. If you know neither yourself nor your enemy, you will always endanger yourself.” WebGoat helps us understand the attack vectors against web applications a little better. Once identifying a possible problems, we walked through a solution that can help reduce the risk.

It is easy to find humor in an employee with administrative access using the password “happiness.” The reaction from the Internet community to Twitter’s problems might be a little schadenfreude at play. Theodor Adorno, philosopher and sociologist, defined schadenfreude as largely unanticipated delight in the suffering of another which is cognized as trivial and/or appropriate. Or, maybe it is more like whistling past the graveyard, where folks are a bit cheerful or joyful in the face of a situation that doesn’t warrant it.

When very public security incidents occur, companies need to take a little stock. Not all employees will take training seriously nor follow all policies. That includes people with important roles. Employees make mistakes and it is difficult to guard against every possible mistake that could occur. That is why a layered approach to security is constantly preached. While each layer cost money, security groups at organizations are in a constant battle to monitor and prevent intrusions in a cost effective way. Fortunately, we have the work of many hackers (the good kind), helping us develop solutions to deal with daily new challenges. There are no guarantees. As Albert Einstein once said, “Anyone who has never made a mistake has never tried anything new.” Wisdom comes when we learn from these mistakes. CIOs need to ask themselves, “how safe is my company from being next week’s security headline?” Security groups within an organization must be able to learn and adapt. At the end of the day, the question is how different is your company from Twitter? Insanity is doing the same thing and expecting different results. I’ll close this post with the wise words of Sam Levenson, “You must learn from the mistakes of others. You can’t possibly live long enough to make them all yourself.”

Before going any further, I did want to give credit to the Doctor, who has been good enough to post over on his ITIL Service Management blog a mind mapping of the incident management process under ITIL. That mind map is the image used in today’s blog. While it has nothing directly to do with RTIR, it is interesting in terms of incident management and I wanted to give proper credit.

Installation

Below are the basic steps involved with installing RTIR.

1. Download and install required Perl modules.

There are several Perl packages, which are dependent on other packages, and so the cycle goes. You can install there packages using cpan, with commands like:

/usr/local/src root# perl -MCPAN -e 'install Business::Hours' /usr/local/src root# perl -MCPAN -e 'install Net::Whois::RIPE'

Sometimes you can run into problems installing modules in that manner. Another alternative is to pull down the package, untar, configure, compile, and install in a manner similar to:

root# cd /usr/local/src /usr/local/src root# wget http://search.cpan.org/CPAN/authors/\ id/M/MR/MRSAM/Net-CIDR-0.11.tar.gz /usr/local/src root# tar xzf Net-CIDR-0.11.tar.gz /usr/local/src root# cd Net-CIDR-0.11 /usr/local/src/Net-CIDR-0.11 root# perl Makefile.PL /usr/local/src/Net-CIDR-0.11root# make test /usr/local/src/Net-CIDR-0.11 root# make install

To make installation of RTIR as easy as possibly, you may want to make sure the following packages are installed.

2. Download, configure, and install RTFM.

The stable RTIR package was released back to 2004, way before the most recent RT 3 release. Pull down the most recently updated for RT 3 release of the RT FAQ Manager (RTFM). To quote from the FAQ, “RTFM lets you open, categorize and search for ‘articles.’ Like RT, RTFM lets your users contribute additional information to existing articles and makes sure that each article’s full history is preserved for future inspection. RTFM makes it easy to quickly search the knowledge base and find critical information.”

root# cd /usr/local/src /usr/local/src root# wget http://download.bestpractical.com/pub/rt/devel/\ RTIR_M3/RTFM-2.2.2.tar.gz /usr/local/src root# tar xzf RTFM-2.2.2.tar.gz /usr/local/src root# cd RTFM-2.2.2 /usr/local/src/RTFM-2.2.2 root# perl Makefile.PL /usr/local/src/RTFM-2.2.2 root# make /usr/local/src/RTFM-2.2.2 root# make install /usr/local/src/RTFM-2.2.2 root# make initdb /usr/local/src/RTFM-2.2.2 root# cd .. /usr/local/src root# wget http://download.bestpractical.com/pub/rt/devel/\ RTIR_M3/RTFM-Extension-ArticleTemplates-0.01.tar.gz /usr/local/src root# tar xzf RTFM-Extension-ArticleTemplates-0.01.tar.gz /usr/local/src root# cd RTFM-Extension-ArticleTemplates-0.01 /usr/local/src/RTFM-Extension-ArticleTemplates-0.01root# perl Makefile.PL /usr/local/src/RTFM-Extension-ArticleTemplates-0.01root# make /usr/local/src/RTFM-Extension-ArticleTemplates-0.01root# make install /usr/local/src/RTFM-Extension-ArticleTemplates-0.01root# cd ..

3. Download, configure, and install RTIR.

Pull down the most recently updated for RT 3 release of RTIR.

root# cd /usr/local/src /usr/local/src root# wget http://download.bestpractical.com/pub/rt/devel/\ RTIR_M3/RT-IR-2.3.17.tar.gz /usr/local/src root# tar xzf RT-IR-2.3.17.tar.gz /usr/local/src root# cd RT-IR-2.3.17 /usr/local/src/RT-IR-2.3.17 root# perl Makefile.PL /usr/local/src/RT-IR-2.3.17 root# make install

4. Edit RT configuration file.

Edit the RT configuration file /opt/rt3/etc/RT_SiteConfig.pm adding the RTIR configuration file /opt/rt3/local/plugins/RT-IR/etc/RTIR_Config.pm.

/usr/local/src/RT-IR-2.3.17 root# vi /opt/rt3/etc/RT_SiteConfig.pm

Add the lines:

# The RTIR config file $RTIR_CONFIG_FILE = "/opt/rt3/local/plugins/RT-IR/etc/RTIR_Config.pm"; require $RTIR_CONFIG_FILE || die ("Couldn't load RTIR config file '$RTIR_CONFIG_FILE'\n$@"); Set(@Plugins, 'RT::FM', 'RT::IR');

5. Initialize the database.

Update the RT database.

/usr/local/src/RT-IR-2.3.17 root# make initdb

6. Stop and restart the Apache server.

For good measure, restart the Apache server.

/usr/local/src/RT-IR-2.3.17 root# /usr/local/apache/bin/apachectl stop /usr/local/src/RT-IR-2.3.17 root# /usr/local/apache/bin/apachectl start

7. Configure RTIR.

Configuration of RT and RTIR is a topic for another post; maybe even a book. I will include below basic instructions from RTIR readme. Please see the next section for additional documentation.

1) Using RT’s configuration interface, add the email address
of the Network Operations Team (the people who will handle
activating and removing Blocks) as AdminCC on the Blocks queue.
RT -> Configuration -> Queues -> Blocks -> Watchers

2) You may want to modify the email messages that are automatically
sent on the creation of Investigations and Blocks.
RT -> Queues -> <select RTIR’s Queue> -> Templates.
RT -> Global -> Templates.

3) By default, RT ships with a number of global Scrips. You should use
RT’s configuration interface to look through them, and disable any
that aren’t apropriate in your environment.
RT -> Queues -> </select><select RTIR’s Queue> -> Scrips.
RT -> Global -> Scrips.

4) Add staff members who handle incidents to the DutyTeam group.
RT -> Configuration -> Groups -> DutyTeam -> Members.

5) You can override values in the RTIR_Config.pm in your
RT_SiteConfig.pm file. Just add your customizations after the “require”
line mentioned above.

Additional Information

RT has a nice user interface. In order to figure out and use the program, you need to read the documentation. We may come back and do a post on configuration. How you configure RTIR software depends on the environment and your plans for using the software. Below is a listing of several information sources.

• DocIndex.pod – Can also view DocIndex.pod with the command:
  perldoc lib/RT/IR/DocIndex.pod
• AdministrationTutorial.pod – Can also view AdministrationTutorial.pod with the command:
  perldoc lib/RT/IR/AdministrationTutorial.pod
• Config.pm – Can also view Constituencies.pod with the command:
  perldoc lib/RT/IR/Config.pm
• Constituencies.pod – Can also view Constituencies.pod with the command:
  perldoc lib/RT/IR/Constituencies.pod
• Ticket.pm – Can also view Ticket.pm with the command:
  perldoc lib/RT/IR/Tutorial.pod
• Tutorial.pod – Can also view Tutorial.pod with the command:
  perldoc lib/RT/IR/Tutorial.pod

If you are going to be using RT, you need to get the “RT Essentials” book written by Jesse Vincent, Robert Spier, Dave Rolsky, Darren Chamberlain, and Richard Foley. It is a good reference and a quick read. For up-to-date information, see the RT Wiki and the Best Practical Solutions blog site.

Prerequisites

To start, please review the following posts:

1. An Apache Implementation
2. Apache and OpenSSL
3. PHP Implementation
• PHP as a Module
• PHP as a CGI
• PHP Configuration Modifications
4. Introduction to MySQL
5. Setting Up and Securing MySQL: References
6. Implementing a Web Application Firewall with ModSecurity

Install Software

With Apache, MySQL, PHP, OpenSSL, and ModSecurity installed, we are now ready to focus on software packages required by RT.

1. Installing expat.

Different operating systems will vary on whether expat, the XML parser, is installed. Expat is needed to complete the cpan install for XML::RSS. Check your particular operating system.

 root# cd /usr/local/src
 /usr/local/src root# wget http://downloads.sourceforge.net/expat/expat-2.0.1.tar.gz
 /usr/local/src root# tar xzf expat-2.0.1.tar.gz
 /usr/local/src root# cd expat-2.0.1
 /usr/local/src/expat-2.0.1 root# ./configure
 /usr/local/src/expat-2.0.1 root# make
 /usr/local/src/expat-2.0.1 root# make check
 /usr/local/src/expat-2.0.1 root# make install

2. Install FastCGI

For RT, you can install mod_perl or mod_fastcgi. In this posting, we are going to walks through the installation of FastCGI. Information concerning mod_perl will be provided below so the reader can chose what fits best in their environment. FastCGI is much simpler to install and allows the core Apache process to stay small in size. With FastCGI, RT runs as a separate process from Apache allowing RT to be stopped and restarted without affecting the Apache server. In general, FastCGI programs are easier to manage.

The Apache module mod_fastcgi allows a web server to run CGI scripts via a separate, persistent program. PHP comes with FastCGI support compiled in by default, so nothing needs to be done to the PHP installation.

You can have the Apache program call FastCGI, and have it run as the same user as the Apache server or use suexec to have FastCGI switch to a different user. Under some operating systems, suexec may not get compiled and installed when installing Apache. Check if suexec is installed, and if not go back to the Apache source, compile it, and install it. Initially, we are not going to use the suexec program. Instead we will create the group “rt”, add user httpd to group rt, and set permissions that way. You may choose later to use suexec.

 root# ls -la /usr/local/apache/bin/suexec
ls: /usr/local/apache/bin/suexec: No such file or directory
 root# cd /usr/local/src/httpd-2.2.8
 /usr/local/src/httpd-2.2.8 root# make suexec
 /usr/local/src/httpd-2.2.8 root# cp ./support/suexec /usr/local/apache/bin/suexec

Now, we are ready to get mod_fastcgi installed.

 root# cd /usr/local/src
 /usr/local/src root# wget http://www.fastcgi.com/dist/mod_fastcgi-2.4.6.tar.gz
 /usr/local/src root# tar xzf mod_fastcgi-2.4.6.tar.gz
 /usr/local/src root# cd mod_fastcgi-2.4.6
 /usr/local/src/mod_fastcgi-2.4.6 root# cp Makefile.AP2 Makefile
 /usr/local/src/mod_fastcgi-2.4.6 root# make top_dir=/usr/local/apache
 /usr/local/src/mod_fastcgi-2.4.6 root# make top_dir=/usr/local/apache install
 /usr/local/src/mod_fastcgi-2.4.6 root# /usr/local/apache/bin/apachectl stop
 /usr/local/src/mod_fastcgi-2.4.6 root# vi /usr/local/apache/conf/httpd.conf

Add the following lines to the Apache httpd.conf file:

# Load the mod_fastcgi module.
LoadModule fastcgi_module modules/mod_fastcgi.so

Check if installation and configuration is working.

 /usr/local/src/mod_fastcgi-2.4.6 root# /usr/local/apache/bin/apachectl configtest
Syntax OK
 /usr/local/src/mod_fastcgi-2.4.6 root# /usr/local/apache/bin/apachectl start
 /usr/local/src/mod_fastcgi-2.4.6 root# cat /var/www/logs/error_log | grep -i fastcgi
[Fri Aug 01 12:17:22 2008] [notice] FastCGI: process manager initialized (pid 15221)
[Fri Aug 01 12:17:22 2008] [notice] Apache/2.2.8 (Unix) mod_ssl/2.2.8
OpenSSL/0.9.7a mod_fastcgi/2.4.6 configured -- resuming normal operations

For in depth coverage of mod_perl, Stas Bekman and Eric Cholet have written the book, “Practical mod_perl.” They have made the complete book available online in both HTML and PDF format under the Creative Commons Attributes Share-Alike License. Stas Bekman and Jim Brandt have also written the “mod_perl2 User’s Guide Book” where 50% of the book’s proceeds go to The Perl Foundation.

If you are installing under Mac OS X, mod_perl may complain about Perl 5.8.8 being built without threads and you will get a message about building perl with -Duserthreads. If you are determined to use mod_perl, consider dropping back to Apache 1.3.x and using mod_perl 1.x. While Apache 1.3.x is legacy code, and I tend to want to use the code that is being actively developed, there is an argument for using Apache 1.3.x. One major feature of Apache 2.x is threading. On Windows, where most basic libraries are and must be threadsafe, Apache 2 is really the only choice. Earlier Mac OS X releases did not include a completely thread-safe libc, so threading is still not fully supported in Perl. This is why the Perl version that comes with Mac OS X is not compiled to use threads. To use Apache2.x, Perl will need to be configured to use threads. The code is available from the Perl web site.

Rather than getting bogged down in compiling Perl to use thread, we will move ahead and use FastCGI. By the time this post, I will have worked on getting RT installed under Linux, Mac OS X, and FreeBSD. Figuring out what software works best in a multi OS environment can be challenging.

3. Configure RT

Let us start by adding the group RT. Under many operating systems, this would be done with the simple command “groupadd rt.” Things are always more interesting under Mac OS X, where you would have to first look at what group ids (gid), choose an unused gid, and then create the rt group using that gid. Under Mac OS X Leopard, group rt would be created with the commands:

 root# dscl . list /groups PrimaryGroupID | sort -k 2,2 -n
 root# dscl . create /groups/rt gid gid-of-rt
 root# dscl . create /groups/rt passwd '*'
 root# dscl . read /groups/rt
AppleMetaNodeLocation: /Local/Default
Password: *
PrimaryGroupID: gid-of-rt
RecordName: rt
RecordType: dsRecTypeNative:groups

RT’s primary maintenance and documentation site is http://www.bestpractical.com. Documentation can be found at the Best Practical Solutions RT Wiki located at http://wiki.bestpractical.com/. The latest TAR/GZ is located at http://download.bestpractical.com/pub/rt/release/rt.tar.gz. The lack of any version numbers means the version can be updated at any time. The latest version, as of this writing, is 3.8.0.

The following are the steps for downloading and configuring RT:

 root# cd /usr/local/src
 /usr/local/src root# wget http://download.bestpractical.com/pub/rt/release/rt.tar.gz
 /usr/local/src root# tar xzf rt.tar.gz
 /usr/local/src root# cd rt-3.8.0
 /usr/local/src/rt-3.6.5 root# ./configure \
  --with-web-user="httpd" \
  --with-web-group="httpd" \
  --with-rt-user="httpd" \
  --with-rt-group="rt"

4. Install Apache::TEST

Perl module Apache::TEST will not allow you to run the test check as root. You can download the module separately as a non root user and after configuring, compiling, and testing the program, you install it as root.

 root# su - goofy
 ~$ cd src
 ~/src goofy$ wget http://search.cpan.org/CPAN/authors/id/P/PH/PHRED/Apache-Test-1.30.tar.gz
 ~/src goofy$ tar xzf Apache-Test-1.30.tar.gz
 ~/src goofy$ cd Apache-Test-1.30
 ~/src goofy$ perl Makefile.PL
 ~/src goofy$ make
 ~/src goofy$ make test
 ~/src goofy$ sudo su root
 root# make instal

5. Run fixdeps Command and Install Perl Modules

Now you are ready to utilize the fixedeps utility that comes with RT to install required Perl modules. There is also the testdeps utility to test if all dependencies are installed and RT is ready to be installed. You may need to run fixdeps multiple times before testdeps reports that you have all required software packages. The first time through, it can take awhile (depending on your installation). Be aware that some perl modules may need to be installed manually. It various depending on OS and your environment. You will be able to tell which modules need manual installation by the final message provided by the fixdeps program.

 root# cd /usr/local/src/rt-3.8.0
 /usr/local/src/rt-3.8.0 root# make fixdeps
 /usr/local/src/rt-3.8.0 root# make fixdeps
 /usr/local/src/rt-3.8.0 root# make testdeps

6. Install RT

The final installation of RT is the easy part.

 /usr/local/src/rt-3.8.0 root# make install

7. Configure RT_SiteConfig.pm

We now will configure /opt/rt3/etc/RT_SiteConfig.pm. In the next step a database user and a database will be setup. We are only adding those values to the configuration file in this step. I am going to set up a hostname (rt.securitymonks.com) for my current machine. Please do not copy blindly. Change this to your environment. We will create the hostname so it only exists locally by adding an entry into the machines /etc/hosts file. Right now, I am only going to access the Apache server from this machine. In other words, the client and server will be on the same box.

 /usr/local/src/rt-3.8.0 root# vi /etc/hosts

Add the following line, adapting it to your organization:

 /usr/local/src/rt-3.8.0 root# vi /etc/hosts
##
127.0.0.1       localhost
10.1.218.202   rt.securitymonks.com

We are now ready to modify the RT_SiteConfig.pm file.

 /usr/local/src/rt-3.8.0 root# vi /opt/rt3/etc/RT_SiteConfig.pm

At minimum, add the following linesto /opt/rt3/etc/RT_SiteConfig.pm:

Set($rtname, 'BRORT');
Set($Organization, 'securitymonks');

Set($CorrespondAddress , 'john@securitymonks.com');
Set($CommentAddress , 'john@securitymonks.com');

Set($Timezone , 'US/Eastern'); # obviously choose what suits you

# THE DATABASE:

Set($DatabaseType, 'mysql'); # e.g. Pg or mysql

# These are the settings we used above when creating the RT database,
# you MUST set these to what you chose in the section above.

Set($DatabaseUser , 'rtuser');
Set($DatabasePassword , 'secret');
Set($DatabaseName , 'rtdb');

# THE WEBSERVER:

Set($WebPath , '');
Set($WebBaseURL , 'https://rt.securitymonks.com');

# Logging
Set($LogToSyslog, '');
Set($LogToFile, 'debug');
Set($LogDir, '/opt/rt3/var/log');
Set($LogToFileNamed, "rt.log");

8. Initialize the Database

RT needs to create the rtdb database, the rt db users, and initialize some tables. This can be done with the command initialize-database, which should be run only once.

 /usr/local/src/rt-3.8.0 root# make initialize-database
 /usr/local/bin/perl sbin/rt-setup-database --action init --dba root --prompt-for-dba-password
In order to create or update your RT database, this script needs to connect to your mysql
instance on localhost as root.  Please specify that user's database password below. If the
user has no database

password, just press return.

Password:
Working with:
Type:   mysql
Host:   localhost
Name:   rtdb
User:   rtuser
DBA:    root
Now creating a mysql database rtdb for RT.
Done.
Now populating database schema.
Done.
Now inserting database ACLs
Granting access to rtuser@'localhost' on rtdb.
Done.
Now inserting RT core system objects
Done.
Now inserting data
Done inserting data
Done.

Check the MySQL database out.

 /usr/local/src/rt-3.8.0 root# mysql -u rtuser -p
mysql> use rtdb;

9. Modify Apache Configuration File

Edit the /usr/local/apache/conf/httpd.conf file.

 /usr/local/src/rt-3.8.0 root# /usr/local/apache/bin/apachectl stop
 /usr/local/src/rt-3.8.0 root# vi /usr/local/apache/conf/httpd.conf

We are going to have the RT server run under our secure web server. Find the “<virtualhost _default_:443>” line, change it to “<virtualhost 10.1.218.202:443>“. Add the following lines to that section (adjusting to your environment):

   ServerName rt.securitymonks.com
   DocumentRoot /opt/rt3/share/html
   ErrorLog /usr/local/apache/logs/rt.error
   LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
   CustomLog /usr/local/apache/logs/rt.access_log combined
   AddHandler fastcgi-script fcgi
   ScriptAlias / /opt/rt3/bin/mason_handler.fcgi/

Add the user the Apache server runs as (httpd by default), to the RT group. For non Mac OS X, modify group membership by editing the file /etc/group (vi /etc/group). Mac OS X users need to user the dscl command.

 root# dscl . append /groups/rt GroupMembership httpd
 root# dscl . read /groups/rt

Change the group and permission on the log area if you have told RT to log to /opt/rt3/var/log.

 root# chgrp rt /opt/rt3/var/log
 root# chmod g+w /opt/rt3/var/log

Test the configuration of the file, and if everything checks out start up Apache.

 /usr/local/src/rt-3.8.0 root# /usr/local/apache/bin/apachectl configtest
Syntax OK
 /usr/local/src/rt-3.8.0 root# /usr/local/apache/bin/apachectl start

Remember there are now three files to check for problems with RT.

• /opt/rt3/var/log/rt.log
• /usr/local/apache/logs/rt.error
• /usr/local/apache/logs/rt.access_log

There are many configuration operations. The options chosen in this post represents only the minimal to get RT running. Please see the RT Wiki’s FastCGIConfiguration page for additional information.

10. Access RT and Change the Default Password

Now it is time to log in and change the default password. Using the entry we made in our /etc/hosts file, we can now access the site by going to https://rt.securitymonks.com. This URL should be different for your site. You will see a login screen similar to the image on the left.

Log in using the username “root” and password “password“. Once logged in, you will see the screen similar to the image below (click on the image if you need to enlarge):

Over on the left menu bar, select “Configuration.” That will bring you to the “RT Administration” screen:

Select, “Users.” That will bring you to the “Select a user” screen:

Select the user “root,” which will bring you to the “Modify the user root” screen. If you look at the lower left of the screen, there is a “Access Control area.” There is a place to enter “New Password.” Do so. The screen looks like:

Make sure to hit the “Save Changes” button at the bottom of the screen. With a working copy of RT, you are not ready to start adjusting configurations and working with the program. For additional information, Please check out the “RT Essentials” and the RT Wiki and the Best Practical Solutions blog site. Look for future posts to build upon the RT installation and database.

The crime landscape is shifting. Attacks are moving up the network stack. Network-based attacks are not the prime source of security problems anymore. The attacks today are moving into the application layer: Web 2.0, instant messenger attacks, fraud, information theft, and crime-ware are just some examples of new types of attacks that generate a load of data to be collected and analyzed. Beware! Applications are really chatty and generate alot of data.

While my current post is not about security visualization (see earlier post “Security Data Visualization“), I would like to point out that DAVIX, a live CD for data analysis and visualization, is expected to be released August 6th. That should be really cool and fun.

Since application security is a topic of interest for me, I ran into the problem of having too many topics I wanted to discuss when I started trying to write a post on ModSecurity, an open source, free WAF Apache module. Today, rather than waiting for me to integrate the information, I decided to move ahead and do the post while limiting myself to only pointing out the various sources. The reader can follow the links for a more in-depth discussion and understanding on the topic.

Why You Should Care

The Risky Business podcast has come to be one of my favorite podcasts. The host, Patrick Gray and regular guest Munir Kotadia, just cracks me up. Plus the show is informative and features great guests. This week’s show had an interview with H D Moore talking about the DNS bug. Timely and informative; what else can one ask for? The 68th episode, done at the beginning of this month, had an interview with Jeremiah Grossman concerning web application firewalls. As Patrick writes in the show notes, “it takes typical organizations around 130 days to fix sequel injection bugs in code. But you can mitigate these sorts of things with a Web app firewall, and you won’t even have to deal with the development team! Hooray!.”

In Grossman’s blog post “Can WAFs protect against business logic flaws?“, he pointed out that “WAFs don’t defend against every logic flaw, or even every crazy form of SQLi or XSS. Just as white/black box scanners can’t identify every vulnerability and neither can expert pen-testers or source code auditors.” Stuart King, in his article, “Larry David and Web Application Firewalls“, builds upon this idea when he wrote:

Back to the CSO article where the point is made that we are sitting on a huge legacy of insecure code and that “we can’t rewrite history.” So, the argument is that a web application firewall mitigates the risk – note: does not solve the problem – until the code can be replaced.

How much of the risk is mitigated is open to debate, but there are lots of other things to consider too. For instance the cost of redeveloping code against the cost of purchasing and supporting a WAF. We also need to consider the value and risk profile of the product.

Today’s world consist of attackers adjusting focus from network-based attacks to the application layer. Grossman in his post “Website Security Strategies that Work” makes the claim that “9 out of 10 (or more) websites have vulnerabilities as a result of being built by those who didn’t know or appreciate the severity of today’s attacks.” There is no arguing that many organizations are sitting on a huge legacy of insecure code, much of which may have been written before the discovery of prevalent vulnerabilities such as XSS, SQL Injection, CSRF, etc. Even worse, organization often have their security groups focused on network or system security, leaving application level security to developers. Unfortunately, these developers are receiving little or no training, while remaining under pressure to produce code under short deadlines.

Andre Gironda series, starting with “Week of War on WAF’s: Day 1 — Top ten reasons to wait on WAF’s,” provides important reasons why WAFs should not be viewed as a silver bullet solution. Rich Mogull in his post “Web Application Security: We Need Web Application Firewalls To Work. Better” makes the important point:

With old school vulnerabilities we know the details of the specific vulnerability and (usually) exploit mechanism. With WAFs, we are trying to block vulnerability classes instead of specific vulnerabilities. This is a HUGE difference. The WAF doesn’t know the details of the application or any application-specific vulnerabilities, and thus is much more limited in what it can block.

Mogull goes on to state that WAFs can:

no longer be merely external boxes protecting against generic vulnerabilities; they need tighter integration into our applications. In the long term, I’ve branded this Application and Database Monitoring and Protection (ADMP) as we create a dedicated application and database security stack that links from the browser on the front end, to the DB server on the back.

ADMPs, or if you prefer WAFs + Database Activity Monitoring (WAFs+DAM), would be another step in the evolution of WAFs. As Ivan Ristic, creator of ModSecurity, points out in his blog post “What’s the Score of the Game?“:

I feel that one of areas where organizations are failing, with regards to web application security, is that there is a lack of communication between the following three groups: Development teams (who are running source code reviews), InfoSec teams (who are running vulnerability scans) and Operational Security teams (who are running web application firewalls). These three teams each have unique perspectives on the vulnerabilities of the webapps and they should share their data with each other.

Nicely stated. No one is arguing that writing secure code is not the answer. If organization began adapting secure systems development lifecycle (SDLC) models into their business operation, many security problems would go away. Building secure software will require changes in the current development culture, which will include people, processes, and technology. No small task.

Gunnar Peterson has a nice post, “WAF and XSG Risk and Effectiveness at 20,000 feet” where he discusses modeling of combination of risk and effectiveness to identify areas of focus. As Peterson points out in another post, “WAFs are not as static as network firewalls…Instead WAFs collaborate much more directly with development, which is another growth opportunity for security industry.”

This post is going to stay focused on WAFs. With it taking on average 130 days to fix sequel injection bugs, organizations need something they can implement today. WAFs have an important role to play in adding a layer of security and monitoring to a defense in depth security approach. WAFs will evolve. They are in the process of evolving now. Understanding the fundamental ideas and going through the implementation of an open source solution starts us on the path of better understanding of future technologies.

An Implementation Using ModSecurity

Building on previous posts concerning “An Apache Implementation“, “PHP Implementation“, and “Apache and OpenSSL“, we have an Apache web server setup to build upon. For additional details, please get Ivan Ristic’s book, “Apache Security.” It really is a must have book for anyone serious about running an Apache web server. Ristic also maintains the ModSecurity website and blog, which serves as a great source for up-to-date information on ModSecurity.

The Apache module mod_unique_id needs to be installed for ModSecurity to work properly. This module was not installed when we configured Apache. At that time, we did not know we needed it. While it can be somewhat inconvenient, for security reasons it is best not to install modules not needed.

1. Stop Apache Server.

# /usr/local/apache/bin/apachectl stop

2. Install mod_unique_id Module.

For non Mac OS X, do the following:

root# cd /usr/local/src/httpd-2.2.8
/usr/local/src/httpd-2.2.8 root# make clean
/usr/local/src/httpd-2.2.8 root# CC=gcc \
CFLAGS="-O3 -fno-omit-frame-pointer" \
./configure --prefix=/usr/local/apache \
--enable-rewrite \
--enable-so \
--disable-imap \
--disable-userdir --with-mpm=worker --enable-ssl --enable-unique-id --enable-unique-id
/usr/local/src/httpd-2.2.8 root# make
/usr/local/src/httpd-2.2.8 root# make install

For Mac OS X, please do:

root# cd /usr/local/src/httpd-2.2.8
/usr/local/src/httpd-2.2.8 root# make clean
/usr/local/src/httpd-2.2.8 root# CC=gcc \
CFLAGS="-O3 -fno-omit-frame-pointer" \
LDFLAGS="-L/opt/local/lib -L/usr/lib" \
./configure --prefix=/usr/local/apache --enable-rewrite \
--enable-so --disable-imap --disable-userdir \
--with-mpm=worker --enable-ssl --enable-unique-id
/usr/local/src/httpd-2.2.8 root# make
/usr/local/src/httpd-2.2.8 root# make install

3. Install PCRE.

Only under Mac OS X did I have to install Perl Compatible Regular Expressions (PCRE). You may be able to skip this step, depending on your OS.

root# cd /usr/local/src
/usr/local/src root# wget http://downloads.sourceforge.net/pcre/pcre-7.7.tar.gz
/usr/local/src root# tar xzf pcre-7.7.tar.gz
/usr/local/src root# cd pcre-7.7
/usr/local/src/pcre-7.7 root# CC=gcc \
CFLAGS="-O3 -fno-omit-frame-pointer" \
LDFLAGS="-L/opt/local/lib -L/usr/lib" \
./configure
/usr/local/src/pcre-7.7 root# make
/usr/local/src/pcre-7.7 root# make test
/usr/local/src/pcre-7.7 root# make install

4. Install the latest version of libxml2 or Lua.

To quote wikipedia, libxml is “a library for parsing XML documents” and Lua is “a lightweight, reflective, imperative and procedural programming language, designed as a scripting language with extensible semantics as a primary goal.” ModSecurity requires dynamic libraries which are not built by default in the source distribution. Binary distribution is recommended.

I will go through configuration and installation of libxml2 from source and the binary installation of lua under Mac OS X. There is a good chance if you are running a different OS, the libraries will have already been installed.

root# cd  /usr/local/src/
/usr/local/src root# wget ftp://xmlsoft.org/libxml2/libxml2-2.6.32.tar.gz
/usr/local/src root# tar xzf libxml2-2.6.32.tar.gz
/usr/local/src root# cd libxml2-2.6.32
/usr/local/src/cd libxml2-2.6.32 root#  CC=gcc \
CFLAGS="-O3 -fno-omit-frame-pointer" \
LDFLAGS="-L/opt/local/lib -L/usr/lib" \
/usr/local/src/cd libxml2-2.6.32 root# ./configure
/usr/local/src/cd libxml2-2.6.32 root# make
/usr/local/src/cd libxml2-2.6.32 root# make install
/usr/local/src/cd libxml2-2.6.32 root# cd ..
/usr/local/src root# wget http://luaforge.net/frs/download.php/3097/lua5_1_3_Darwin811x86_lib.tar.gz
/usr/local/src root# mkdir lua
/usr/local/src root# cd lua
/usr/local/src/lua root# tar xzf lua5_1_3_Darwin811x86_lib.tar.gz
/usr/local/src/lua root# cp liblua5.1.* /usr/local/lib
/usr/local/src/lua root# cp include/* /usr/local/include

5. Download, unpack, configure, and compile ModSecurity.

If you are interested in connecting a ModSecurity sensor to the central audit log repository, you will want to build the ModSecurity Log Collector below with the command “make mlogc”. Install instructions can be found under apache2/mlogc-src directory. That step will not be included below.

root# cd  /usr/local/src/
/usr/local/src root# wget http://www.modsecurity.org/download/modsecurity-apache_2.5.5.tar.gz
/usr/local/src root# tar xzf modsecurity-apache_2.5.5.tar.gz
/usr/local/src root# cd modsecurity-apache_2.5.5
/usr/local/src/modsecurity-apache_2.5.5 root# cd apache2

For non Mac OS X, configure with the command:

/usr/local/src/modsecurity-apache_2.5.5/apache2 root# ./configure \
--with-apxs=/usr/local/apache/bin/apxs \
--with-apr=/usr/local/apache/bin \
--with-apu=/usr/local/apache/bin

For Mac OS X, use the command:

/usr/local/src/modsecurity-apache_2.5.5/apache2 root# CC=gcc \
CFLAGS="-O3 -fno-omit-frame-pointer" \
LDFLAGS="-L/opt/local/lib -L/usr/lib" \
./configure --with-apxs=/usr/local/apache/bin/apxs

Continue to compile and install with the commands:

/usr/local/src/modsecurity-apache_2.5.5/apache2 root# make
/usr/local/src/modsecurity-apache_2.5.5/apache2 root# make test
/usr/local/src/modsecurity-apache_2.5.5/apache2 root# make install
/usr/local/src/modsecurity-apache_2.5.5/apache2 root# ls -la /usr/local/apache/modules

6. Configure Apache and ModSecurity.

We must now edit the httpd.conf file in order to load libxml2 or lua5.1 modules before the ModSecurity module.

/usr/local/src/modsecurity-apache_2.5.5/apache2 root# vi /usr/local/apache/conf/httpd.conf

Add the lines for non Mac OS X:

#
LoadFile /usr/lib/libxml2.so
LoadFile /usr/lib/liblua5.1.so
#
LoadModule security2_module modules/mod_security2.so

For Mac OS X, add the lines:

#
LoadFile /usr/local/lib/libxml2.2.dylib
LoadFile /usr/local/lib/liblua5.1.so
#
LoadModule security2_module modules/mod_security2.so

Create the ModSecurity configuration file. There is a file modsecurity.conf-minimal present in the /usr/local/src/modsecurity-apache_2.5.5 that can be used. There is also a a Core Rule Set that was included in the /usr/local/src/modsecurity-apache_2.5.5/rules directory courtesy of Breach Security Inc. To quote the README file, “The Core Rule Set provides generic protection from unknown vulnerabilities often found in web application that are in most cases custom coded. The Core Rule Set is heavily commented to allow it to be used as a step-by-step deployment guide for ModSecurity.” Under the rules subdirectory, there a directory “optional” which contains additional possible rules. It is left to the reader which configuration files they may want to include, though it might be wise to start with the minimal and make sure the Apache runs without problems. Then add configurations files as desired.

/usr/local/src/modsecurity-apache_2.5.5/apache2 root# cd ..
/usr/local/src/modsecurity-apache_2.5.5 root# cp  modsecurity.conf-minimal /usr/local/apache/conf/modsecurity.conf
/usr/local/src/modsecurity-apache_2.5.5 root# cp  rules/*.conf /usr/local/apache/conf/

Include the modsecurity.conf, and additional ModSecurity configurations file, in the Apache httpd.conf file.

/usr/local/src/modsecurity-apache_2.5.5 root# vi /usr/local/apache/conf/httpd.conf

Add the line:

Include /usr/local/apache/conf/modsecurity.conf
#Include /usr/local/apache/conf/modsecurity_crs_10_config.conf
#Include /usr/local/apache/conf/modsecurity_crs_21_protocol_anomalies.conf
#Include /usr/local/apache/conf/modsecurity_crs_23_request_limits.conf
#Include /usr/local/apache/conf/modsecurity_crs_30_http_policy.conf
#Include /usr/local/apache/conf/modsecurity_crs_35_bad_robots.conf
#Include /usr/local/apache/conf/modsecurity_crs_40_generic_attacks.conf
#Include /usr/local/apache/conf/modsecurity_crs_45_trojans.conf
#Include /usr/local/apache/conf/modsecurity_crs_50_outbound.conf

Edit /usr/local/apache/conf/modsecurity.conf. The modifications will be very dependent on your environment. See resources listed in the Additional Information section to help with configuration. The default configuration saves the log files relative to the configuration file directory. Change this to where the apache logs are currently being saved.

/usr/local/src/modsecurity-apache_2.5.5 root# vi /usr/local/apache/conf/modsecurity.conf

Change the values to:

SecAuditLog /var/www/logs/modsec_audit.log
SecDebugLog /var/www/logs/modsec_debug.log

Let’s create null files with the correct permissions for Apache.

/usr/local/src/modsecurity-apache_2.5.5 root# touch  /var/www/logs/modsec_audit.log
/usr/local/src/modsecurity-apache_2.5.5 root# chown httpd.httpd /var/www/logs/modsec_audit.log
/usr/local/src/modsecurity-apache_2.5.5 root# touch  /var/www/logs/modsec_debug.log
/usr/local/src/modsecurity-apache_2.5.5 root# chown httpd.httpd /var/www/logs/modsec_debug.log

7. Start Apache.

Check that the configuration file is correct and start up Apache.

/usr/local/src/modsecurity-apache_2.5.5 root# /usr/local/apache/bin/apachectl configtest
Syntax OK
/usr/local/src/modsecurity-apache_2.5.5 root# /usr/local/apache/bin/apachectl start

Check if ModSecurity if configured into running Apache server.

/usr/local/src/modsecurity-apache_2.5.5 root# cat /var/www/logs/error_log | grep ModSecurity
[Thu Jul 31 18:24:59 2008] [notice] ModSecurity for Apache/2.5.5 (http://www.modsecurity.org/) configured.

Additional Information

This post is only to get the basics down. The above information was taken from the ModSecurity documentation install section for version 2.5.5. A great deal more information is available at the ModSecurity blog site and in the book “Apache Security“.

Concluding Remarks

Ivan Ristic and Ofer Shezaf are working on an interesting paper, “Enough With Default Allow in Web Applications!” This paper demonstrates how WAFs are evolving. To quote the paper:

The default allow deployment model, which is commonly used to implement and deploy web applications, is the cause of numerous security problems. We propose a method of modeling web applications in a platform-agnostic way to adopt a default deny model instead, removing several classes of vulnerability altogether and significantly reducing the attack surface of many others. Our approach is best adopted during development, but can be nearly as efficient as an afterthought, or when used at deployment time. What they are looking to do is create a protection layer between the web servers and applications which would increase security and turn applications into verifiable components with external contracts that can be enforced.

Ristic mentions in his post the planned release of “an open source profiling tool (which I will announce next week) to help with the third use case and automate the creation of positive security models (also known as the learning feature of web application firewalls).” Breach Security has also teamed up with WhiteHat Security to add the ability to their Sentinel scanning service to automatically create custom ModSecurity rules for certain classes of vulnerabilities that are found in your web applications. This is the kind of evolution that is required in security and makes ModSecurity such an interesting software package.

If you are using Apache 2.x, which is what we installed in the previous post, Apache supports SSL already with the mod_ssl module. If you need to check if mod_ssl is part of your Apache configuration, do so with the command:

/usr/local/apache/bin/httpd -l

Having established that mod_ssl module is installed, we will now go through the steps of generating a certificate request, approving the certificate, and add to Apache configuration file required lines to create a SSL enabled Apache server.

Key Generation

We start off by generating a non password protected 1024 bit server private key using the RSA algorithm. We will have the key stored in the file server.key.

 root#  cd /usr/local/apache/conf
 /usr/local/apache/conf root# mkdir ssl
 /usr/local/apache/conf root# cd ssl
 /usr/local/apache/conf/ssl root# openssl genrsa -out server.key 1024
Generating RSA private key, 1024 bit long modulus
.....................................................................++++++
....++++++
e is 65537 (0x10001)

 /usr/local/apache/conf/ssl root# cat server.key
-----BEGIN RSA PRIVATE KEY-----
MIICXQIBAAKBgQDS1A7eGdTo39tnYdzYI3Ifl1/sA4ZiY7PzkDU68tZoA4WzJB9n
aSJHCbd0ESG0oPiyf/yqMyNR2ECwjLUFo1m1nrON2w3QdM5zFZZv6v7jDt40u3bT
95XwdFG4FYDw+gq6liuRAkAe2+B6WM1Qv5rN3IzhmZmPA8YjV/sbPzNmzIuNHrVw
FY0WkRzrEF6P36Z6RVJQSzgzx4pDeu5rRX88HxLdTm4Uz6maiwhLsxZv0QIDAQAB
hsqZuAY7esEvSlL9xxXHxHL9Ywl/EXnXSMcJ9ktSobs/T0favkQKulgq6ov9TzIQ
v2Z2vLEABQJBANeKaGm41GgDZp3yEIuNKUp0OjwnORpkuYf/DFA+ox1AAS2OPGjV
iua7aiHYPPF6O6Knb+6SiBRFVkjB6Pz1Fl0CQFs7mxCIHrvTjGN8EcHQ038IP/iu
AoGBAMLjcFLzghM7TDBHEMVkDs0RO4SKxaESFXkjZ3F0papFB0TQMY+AakVMwB80
7vlwjDVFhqU23IF97F7H01bA590DfIxg6c11w4PdlxHVb9Kv+K7P7mve3wbJEUV+
rYr8r+Hr25Fegzwg1tfgFLDDkDoeC3u1wbQNCmL/qksSrD6hAkEA+mc0g4S92Y6S
gDQ3JU/YLaYV4aw2Xk/v5RtNmk++73QtU++azuXSFeDbHHHsZdm2tXBOdRkCQQCv
vg68hRPLa1p0VjbfUk3kgzgoa+LHfnE4TeEAXNIqu1E6j8r5v4Pt9cnnpqSqT/vn
qzTyQmTBW621ioer5A4v2QocJ2R6XhgjDwFOmKTGs0mH
-----END RSA PRIVATE KEY-----
 /usr/local/apache/conf/ssl root# chmod 400 server.key

Certificate Signing Request (CSR)

The next step is to create a certificate-signing request (CSR), which is used as a message sent asking a certificate authority (CA) to sign a certificate. If you want a field to be empty, do not hit return. That would just select the default value. Instead, use “.”. Below, I have left my responses off. Enter your own appropriate values. The generated CSR will be the file server.csr.

 /usr/local/apache/conf/ssl root# openssl req -new -key server.key -out server.csr
You are about to be asked to enter information that will be
incorporated into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [GB]:
State or Province Name (full name) [Berkshire]:
Organization Name (eg, company) [My Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
 /usr/local/apache/conf/ssl root#  cat server.csr

You can now send the CSR server.csr to your public CA. The CA will generate and sign the certificate. To make things more interesting, we are now going to sign our own CSR and generate a signed certificate server.crt. You can use the command “openssl x509″ to examine the certificate.

 /usr/local/apache/conf/ssl root# openssl x509 -req -days 365 -in server.csr \
-signkey server.key -out server.crt
Signature ok
subject=/C=US/ST=New
 Jersey/L=Princetown/O=PU/OU=RS/CN=podus.pu.edu/emailAddress=jbond@pu.edu
Getting Private key
 /usr/local/apache/conf/ssl root# openssl x509 -text -in server.crt
 /usr/local/apache/conf/ssl root# chmod 400 server.crt

Configuration

Now we are going to enable SSL on our Apache server running the server off port 443. The server will use the certificate generated above. We are also going to disable SSLv2 since it has some problems and is disabled by default in Internet Explorer 7, Firefox 2, Opera 9, and Safari.

Modify the apache configuration file:

/usr/local/apache/conf/httpd.conf

adding the lines:

Listen 443
<virtualhost _default_:443>
#   SSL Engine Switch:
SSLEngine on
# Path to the server certificcate
SSLCertificateFile "/usr/local/apache/conf/ssl/server.crt"
# Path to the server private key
SSLCertificateKeyFile "/usr/local/apache/conf/ssl/server.key"
# Allow SSLv3 only
SSLProtocol All -SSLv2
#   SSL Cipher Suite:
#   Disallow ciphers that are weak
SSLCipherSuite ALL:!EXP:!NULL:!ADH:!LOW
# Make SSL work with Internet Explorer
SetEnvIf User-Agent ".*MSIE.*" \
              nokeepalive ssl-unclean-shutdown \
              downgrade-1.0 force-response-1.0
</virtualhost>

Depending on your requirements, you might want to run the server only over SSL. You can do this by stopping the server from listening on port 80. That would result in unable to connect message. Instead, it might be better to redirect traffic coming to http over to https. A great source of recipes for rewrite rule is the mod_rewrite Cookbook site.

RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^/(.*) https://%{SERVER_NAME}/$1 [R,L]

We will also add limitations to who can access the site by limiting who access the “/” directory. In the example below, we will limit it to 127.0.0.1. In the httpd.conf file, after the line “<Directory />” add the line:

SSLRequire  %{REMOTE_ADDR} =~ m/^127.0.0.1$/

Start Apache with the command:

/usr/local/apache/bin/apachectl start

You can try and access the web server with the URL:

http://yourhostname/

you will get the message:

Forbidden
You don't have permission to access / on this server.

Notice that the URL has changed from “http://yourhostname” to “https://yourhostname”. Now try and access the site using:

http://127.0.0.1/

You will get prompted whether to accept the certificate. Since you are accessing 127.0.0.1, it will complain about a domain name mismatch. The host you are trying to access needs to match the entry you provided for “Common Name” in the CSR, otherwise it will complain. If you accept the certificate, you can access the site.

Final Thoughts

At this point, we have secured communication between the client and the Apache web server. In my next post, we will discuss installing one more module, mod_security. Mod_security is a web application firewall and serves as another layer in our web defenses. It is not meant as a replacement for implementing good security in databases, web servers, or applications. That is why we have gone through all these additional steps first. Remember, once, SSLv2 was thought to be secure. Now we know otherwise. Vulnerabilities are continuously being discovered. Good security is about building up one’s defenses. It is a process, not a destination. Maybe one day some company will have a security solution that will defend systems against all threats. I know that is not today, no matter what sales folks might say. When that day comes, you can bet before that solution makes it to market, hackers will have found a way around it. So while we wait for the security rapture to take us to a secure promise land, implement security in layers. It is your best defense.

In honor of ApacheCon Europe, held this past week, we will be going over a very basic implementation of an Apache web server. Ivan Ristic, author of “Apache Security” spoke at ApacheCon where he presented “Web Intrusion Detection with ModSecurity.” Rich Bowen, author of “The Definitive Guide to Apache mod_rewrite” (and a few other titles) on his blog DrBacchus Journal did a post titled “ApacheCon EU 2008 so far.” Rich had this to say about Ivan’s talk, “His talk was fabulous.” He goes on to elaborate, “I’m sure that everything Ivan talked about is in the docs, but his talk was amazingly valuable anyway, since it showed me things in one hour that would probably have taken me months to discover. And I’ve been using mod_security for years already, and wasn’t aware of them, or didn’t quite understand the syntax.” Nick Kew agrees with Rich. On Nick’s blog NIQ’s Soapbox, his posting “Putting ones money where ones mouth is ….” had this to say, “The highlight of today was Ivan Ristic’s mod_security talk: that module is getting seriously interesting.”

If you are interested in hearing more about ApacheCon, the keynote sessions have been made available for free off the Linux Magazine web site. The available presentations consist of:

• Jim Jagielski, Chairman of the Apache Software Foundation, starts off with his talk “State of the Feather.” To quote from the program, “Join ASF Chairman Jim Jagielski for a review of events and progress over the last 12 months within the Apache Software Foundation. Jim will detail the growth of the ASF, both in members as well as projects, discuss the noteworthy achievements of the ASF during that time period, and preview what the next 12 months likely hold for the pre-eminent open source foundation.“
• Cliff Schmidt, Executive Director of Literacy Bridge, discusses “Using Audio Technology and Open Content to Reduce Global Illiteracy.” To quote from the program, “During this talk, Cliff will share his observations from Ghana and discuss Literacy Bridge’s Talking Book Project. Literacy Bridge was founded to empower children and adults with tools for scalable knowledge sharing and literacy learning. The Talking Book Project is Literacy Bridge’s major program, developing new and affordable digital audio technology to provide vital, locally generated information and literacy training to people with limited access to either.”
• Rishab Aiyer Ghosh, Open Source Initiative Board Member, presents “Apache and Steam Engines: the Magic of Collaborative Innovation.” To quote from the program, “Rishab looks at collaborative model of creativity, from 18th century steam engines to the human genome project and discusses why and how collaborative creativity works. Using data from the FLOSS studies, he shows how this makes free software a continuing source of economic value and innovation around the world.”
• Roy Fielding, Co-founder of The Apache Software Foundation, and Vice President, Apache HTTP Server, discussed “Apache 3.0 (a Tall Tale)”. To quote from the program, “Thirteen years ago, the Apache Group founders finished the first beta release of Apache httpd, having reached the end of their initial pile of small improvements, and began to look forward to a complete rewrite of the server architecture. Suddenly, our forward progress slowed to a trickle, mailing list traffic dropped by two-thirds, and our focus diverged…Today, we face a new chasm, and our past successes have only made it wider and deeper than before. This talk is about the other side.“

I am with Rich and Nick, Ivan’s work with ModSecurity is extremely interesting and we will build towards implementing it. First, we need to start simple for there are many steps in the process. This post will provide references for setting up an Apache server, followed by a simple implementation. For additional information, particularly in the area of security, see my previous post “Securing Apache: References.”

Some folks might ask, “why not simply install XAMPP or MAMPP (depending on your system)?” Installing binaries is one way to go. With something as important as the web server, taking the easiest path is not necessary the best option. Compiling from source provides the most power and flexibility to change things according to your priorities. You gain control over such issues as compile time options, modules, and when to upgrade (verses waiting for security patches and upgrades to come out in binary format). While it might not be the easiest path, it is vital in learning what is going on with your system and helping avoid integration problems in the future.

Documents and Articles

Below are a few documents and articles that are most helpful when setting up Apache.

• Apache Security by Ivan Ristic. When it comes to the Apache web server, Ivan is the man to listen to. His book, truly is the complete guide to securing your Apache web server. It is an excellent resource.
• Compiling and Installing document created by The Apache Software Foundation. A great deal of information on Apache is available, not surprisingly, off the Apache HTTP server site.
• Securing Apache 2: Step-by-Step by Ivan Ristic. This is a shorter, more compact article appearing in SecurityFocus.
• Security Tips document created by The Apache Software Foundation. A very good source of information on securing the Apache HTTP server. The best place to go to ensure you have the most up-to-date information on securing the server.

Benchmarks

The Center for Internet Security is a non-profit enterprise that helps develop security configuration benchmarks. The stated mission of CIS is to “reduce the risk of business and e-commerce disruptions resulting from inadequate technical security controls.” They have created the document “CIS Level 1 & 2 Benchmark and Scoring Tool for the Apache Web Server.” The document was last updated, as of this writing, on January 2008. The download file consists of:

• CIS_Apache_Benchmark_v2.1.pdf – the Benchmark document contains detailed instructions for implementing the steps necessary for CIS Level 1 and Level 2 sec.
• cis_score_tool_apache_v2.10.sh.gz – a Host-based Scoring Tool scores the security of a system against the Benchmark and creates a variance report.

Additional information is available off the site.

Forums and Blogs

Most of the forums and blogs that I am familiar with deal with security issues involved with web servers. See my “Securing Apache: References” post for those links.

Installing Apache

If Apache was not installed with your operating system, or if you wish to compile from source, you will need to download the latest Apache from the Apache web site. For this document, I will go through pulling down Apache version 2.2. Please consult the Apache HTTP Server Version 2.2 Compiling and Installing documentation for additional information.

 root# cd /usr/local/src
 /usr/local/src root# wget http://mirrors.isc.org/pub/apache/httpd/httpd-2.2.8.tar.gz
 /usr/local/src root# tar xzf httpd-2.2.8.tar.gz

At this point, you need to check the integrity. There as two ways to do this. First, is by calculating the MD5 sum of the source and comparing it to the signature file. Mac OS X users, please note use the command /sbin/md5 instead of md5sum.

 /usr/local/src root# md5sum httpd-2.2.8.tar.gz
39a755eb0f584c279336387b321e3dfc  httpd-2.2.8.tar.gz

 /usr/local/src root# wget -O - -q http://www.apache.org/dist/httpd/httpd-2.2.8.tar.gz.md5
39a755eb0f584c279336387b321e3dfc  httpd-2.2.8.tar.gz

The second method, uses public-key cryptography to verify the integrity of the files. This is more complicated, but more secure. The MD5 sums can be circumvented if an intruder compromises the main distribution site and replaces the signature files. Using public-key cryptography can be done using GnuPG, the free software version of the OpenPGP. Most Unix systems has it installed by default. The installation is fairly straight forward, no matter what OS you are using:

1. Install GnuPG. For Mac OS X, you would want to install Mac GnuPG. For Windows, you will need WinPT.
2. Optionally, under Unix you might want to install a graphical front-end for GnuPG.
3. Generate a pair of keys.

Apache developers use their cryptographic keys to sign the distributions digitally. We are going to download the PGP signature, fetch the GnuPG unique key ID (DE885DD3), and then check the signature.

 /usr/local/src root# wget http://www.apache.org/dist/httpd/httpd-2.2.8.tar.gz.asc
 /usr/local/src root# gpg --keyserver pgpkeys.mit.edu --recv-key DE885DD3
gpg: requesting key DE885DD3 from HKP keyserver pgpkeys.mit.edu
gpg: trustdb created
gpg: key DE885DD3: public key "Sander Striker " imported
gpg: Total number processed: 1
gpg:               imported: 1

 /usr/local/src root# gpg httpd-2.2.8.tar.gz.asc
gpg: Signature made Sat Jan 18 07:21:28 2003 PST using DSA key ID DE885DD3
gpg: Good signature from "Sander Striker "
gpg:                 aka "Sander Striker "
gpg: checking the trustdb
gpg: no ultimately trusted keys found
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Fingerprint: 4C1E ADAD B4EF 5007 579C  919C 6635 B6C0 DE88 5DD3

 /usr/local/src root# gpg --fingerprint DE885DD3
pub  1024D/DE885DD3 2002-04-10 Sander Striker 
     Key fingerprint = 4C1E ADAD B4EF 5007 579C  919C 6635 B6C0 DE88 5DD3
uid                            Sander Striker 
sub  2048g/532D14CA 2002-04-10

To verify DE885DD3 was created by the real Sander Striker, download public keys for the Apache HTTP Server developers from the Apache HTTP Server Project website. Officially, you should validate by face-to-face communication with multiple government-issued photo identification confirmations. Trust can be a complicated issue. For more information on determining what level of trust, please read the GNU Privacy Handbook section on Validating other keys on your public keyring.

Let us get back to compiling Apache.

 /usr/local/src root# cd httpd-2.2.8
 /usr/local/src/httpd-2.2.8 root# ./configure --prefix=/usr/local/apache
 /usr/local/src/httpd-2.2.8 root# make
 /usr/local/src/httpd-2.2.8 root# make install

There is more to be done. The above configuration was to help in determining the Apache modules that are compiled be default. The following modules, should not be used unless needed:

The following modules will be require for use with later postings. If you are sure you do not need theses modules, do not include them, and make sure not to include them in the below configuration.

To determine which modules would be installed by default, issue the following command.

 /usr/local/src/httpd-2.2.8 root# ./httpd -l
Compiled in modules:
  core.c
  mod_authn_file.c
  mod_authn_default.c
  mod_authz_host.c
  mod_authz_groupfile.c
  mod_authz_user.c
  mod_authz_default.c
  mod_auth_basic.c
  mod_include.c
  mod_filter.c
  mod_log_config.c
  mod_env.c
  mod_setenvif.c
  prefork.c
  http_core.c
  mod_mime.c
  mod_status.c
  mod_autoindex.c
  mod_asis.c
  mod_cgi.c
  mod_negotiation.c
  mod_dir.c
  mod_actions.c
  mod_userdir.c
  mod_alias.c
  mod_so.c

Now, we are going to change the modules that get installed. We are going to add Secure Sockets Layer (SSL) support with the “–enable-ssl” switch. In a later post, we will discuss how to use this cryptographic protocols to provide secure communications between clients and our web browser. Unless you are sure you do not want SSL support, include the “–enable-ssl” switch.

For folks compiling on Mac OS X 10.5, the “–enable-ssl” switch will give you problems. As of this writing, Leopard is using OpenSSL version 0.9.71 from September 2006. There have been some changes made since then. You could upgrade, but as mentioned before, you do not know what installed software is dependent on that library. It would be nice if Apple had upgraded with the release of a new OS, but you have to play the cards Steve Jobs has dealt. The MacPorts Project have the latest version of OpenSSL, 0.9.8g from October 2007. In order to get Apache to compile, I would recommend using the most recent OpenSSL library. If you need help with MacPorts, please see my posting “MacPorts Under Mac OS X Leopard.” Special Mac OS X installation instruction follow.

For non Mac OS X operating systems, do the following to configure, compile and install Apache:

 /usr/local/src/httpd-2.2.8 root# make clean
 /usr/local/src/httpd-2.2.8 root# /bin/rm -r /usr/local/apache
 /usr/local/src/httpd-2.2.8 root# CC=gcc \
CFLAGS="-O3 -fno-omit-frame-pointer" \
./configure --prefix=/usr/local/apache \
--enable-rewrite \
--enable-so \
--disable-imap \
--disable-userdir --with-mpm=worker --enable-ssl
 /usr/local/src/httpd-2.2.8 root# make
 /usr/local/src/httpd-2.2.8 root# make install

For Mac OS X, you need to tell the compiler which libraries to use so the more recent OpenSSL is used. Do that with the following commands:

 /usr/local/src/httpd-2.2.8 root# make clean
 /usr/local/src/httpd-2.2.8 root# /bin/rm -r /usr/local/apache
 /usr/local/src/httpd-2.2.8 root# CC=gcc \
CFLAGS="-O3 -fno-omit-frame-pointer" \
LDFLAGS="-L/opt/local/lib -L/usr/lib" \
./configure --prefix=/usr/local/apache --enable-rewrite \
--enable-so --disable-imap --disable-userdir \
--with-mpm=worker --enable-ssl
 /usr/local/src/httpd-2.2.8 root# make
 /usr/local/src/httpd-2.2.8 root# make install

Configuration

It is time to configure and make the Apache server more secure. Ivan Ristic have made available “Chapter 2: Installation and configuration.” Follow the chapter, do not just copy. For demonstration purposes, I’ll be using what is written in that chapter to configure the Apache web server. There are various configuration options and you want to configure the server for your environment. It is very important to understand what is contained in your configuration file.

Create the user and group httpd, from which the Apache web server will run. Under most versions of Unix (not Mac OS X), this is a simple matter of executing the following commands:

 /usr/local/src/httpd-2.2.8 root# cd /usr/local/apache
 /usr/local/apache root# /usr/sbin/groupadd httpd
 /usr/local/apache root# /usr/sbin/useradd httpd -g httpd -d /dev/null -s /sbin/nlogin

Under Mac OS X, there is no groupadd or useradd command. Things are always more interesting under Mac OS X. Prior to Leopard (10.5), you would have to determine which group ids (gid) have been used, choose an unused gid, and then create the httpd group using that gid. This would be accomplished with the commands:

 root# nireport . /groups gid name
 root# nicl . -create /groups/httpd
 root# nicl . -append /groups/httpd unique-gid
 root# nicl . -append /groups/httpd passwd "*"
 root# nireport . /groups gid name

Once creating the group, you would need to create a new user by finding an unused uid, create the user, fill in the user attributes, add a password, create a home area, and finally set permissions. This would be accomplished with the commands:

 root# nireport / /users name uid
 root# niutil -create / /users/httpd
 root# niutil -createprop / /users/httpd uid uid-from-above
 root# niutil -createprop / /users/httpd gid gid-from-above
 root# niutil -createprop / /users/httpd realname "Web Server"
 root# niutil -createprop / /users/httpd home "/dev/null"
 root# niutil -createprop / /users/httpd shell "/sbin/nologin"
 root# niutil -createprop / /users/httpd passwd "*"

NetInfo, the system configuration database, no longer exists in Mac OS X 10.5 (Leopard). The entire structure for managing local users, groups, and other such things has been completely replaced by Local Directory Services. In Leopard, the DirectoryService daemon does the job of the DirectoryService, lookupd, and the memberd daemons. Please see previous posting, “Backing Up Using Amanda on Mac OS X Leopard Part I” for additional details. There is now a command line utility dscl to perform some advanced functions formerly covered by NetInfo Manager. Creating the group httpd and user httpd would be done with the following commands:

 root# dscl . list /groups PrimaryGroupID | sort -k 2,2 -n
 root# dscl . create /groups/httpd gid gid-of-httpd
 root# dscl . create /groups/httpd passwd '*'
 root# dscl . read /groups/httpd
AppleMetaNodeLocation: /Local/Default
Password: *
PrimaryGroupID: gid-of-httpd
RecordName: httpd
RecordType: dsRecTypeNative:groups

 root# dscl . list /users UniqueID | sort -k 2,2 -n
 root# dscl localhost -create /Local/Default/Users/httpd
 root# dscl localhost -create /Local/Default/Users/httpd RecordName httpd
 root# dscl localhost -create /Local/Default/Users/httpd UserShell /sbin/nologin
 root# dscl localhost -create /Local/Default/Users/httpd RealName "Web Server"
 root# dscl localhost -create /Local/Default/Users/httpd UniqueID a-unique-uid
 root# dscl localhost -create /Local/Default/Users/httpd PrimaryGroupID gid-of-httpd
 root# dscl localhost -create /Local/Default/Users/httpd NFSHomeDirectory /dev/null
 root# dscl . read /users/httpd
AppleMetaNodeLocation: /Local/Default
GeneratedUID: generated-unique-id
NFSHomeDirectory: /dev/null
PrimaryGroupID: gid-of-httpd
RealName:
 Web Server
RecordName: httpd
RecordType: dsRecTypeNative:users
UniqueID: a-unique-uid
UserShell: /sbin/nologin

As part of the installation, the file /usr/local/apache/conf/httpd.conf is created. Move that configuration for safe keeping and start with an empty configuration file. Add the required functionality to ensure only the needed directives and modules are included. Also, adjust file permissions. Mac OS X users note that there is no group “root.” Please use the group “admin” instead.

 /usr/local/src/httpd-2.2.8 root# cd /usr/local/apache
 /usr/local/apache root# mv /usr/local/apache/conf/httpd.conf /usr/local/apache/conf/httpd.conf.orig
 /usr/local/apache root# chown -R root:root /usr/local/apache
 /usr/local/apache root# find /usr/local/apache -type d | xargs chmod 755
 /usr/local/apache root# find /usr/local/apache -type f | xargs chmod 644
 /usr/local/apache root# chmod u+x  /usr/local/apache/bin/*
 /usr/local/apache root# mkdir -p /var/www/logs
 /usr/local/apache root# mv /usr/local/apache/htdocs /var/www/htdocs
 /usr/local/apache root# find /var/www/ -type d | xargs chmod 755
 /usr/local/apache root# find /var/www/ -type f | xargs chmod 644
 /usr/local/apache root# chmod -R go-r /usr/local/apache/conf
 /usr/local/apache root# chmod -R go-r /usr/local/apache/logs
 /usr/local/apache root# chmod -R go-r /var/www/logs
 /usr/local/apache root# vi /usr/local/apache/conf/httpd.conf

Create a configuration file /usr/local/apache/conf/httpd.conf similar to the following file (adjust to your requirements):

# Location of the web server files
ServerRoot /usr/local/apache
# Location of the wev server tree
DocumentRoot /var/www/htdocs
# Listen on which port
Listen 80
# Store the PID of the main Apache process
PidFile /var/www/logs/httpd.pid
# Do not enables DNS lookups on client IP addresses
HostNameLookups Off
#
User httpd
Group httpd
# Deny access to the complete filesystem and then allow access
# to the document root.
<Directory />
Order Deny,Allow
Deny from all
Options None
AllowOverride None
</Directory>
<Directory /var/www/htdocs>
Order Allow,Deny
Allow from all
</Directory>
# Enable CGI Scripts
<Directory /var/www/cgi-bin>
Options ExecCGI
SetHandler cgi-script
</Directory>
# Logging
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog /var/www/logs/access_log combined
LogLevel info
ErrorLog /var/www/logs/error_log
# Setting Server Configuration Limits
# wait up to 300 seconds for slow clients
TimeOut 60
# allow connections to be reused between requests
KeepAlive On
# allow a maximum of 100 requests per connection
MaxKeepAliveRequests 100
# wait up to 15 seconds for the next
# request on an open connection
KeepAliveTimeout 15
# impose no limits on the request body
LimitRequestBody 64000
# allow up to 100 headers in a request
LimitRequestFields 100
# each header may be up to 8190 bytes long
LimitRequestFieldsize 8190
# the first line of the request can be
# up to 8190 bytes long
LimitRequestLine 8190
# limit the XML request body to 1 million bytes(Apache 2.x only)
LimitXMLRequestBody 1000000
# the maximum number of processes
ServerLimit 16
# how many processes to start with
StartServers 2
# how many threads per process to create
ThreadsPerChild 25
# minimum spare threads across all processes
MinSpareThreads 25
# maximum spare threads across all processes
MaxSpareThreads 75
# maximum clients at any given time
MaxClients 150
# Preventing Information Leaks
ServerSignature Off
ServerTokens ProductOnly
<FilesMatch "(^\.ht|~$|\.bak$|\.BAK$)">
Order Allow,Deny
Deny from all
</FilesMatch>

At this point, you are ready to bring up the Apache web server. Clean up any unnecessary files.

 /usr/local/src/httpd-2.2.8 root# /bin/rm -r /usr/local/apache/cgi-bin
 /usr/local/src/httpd-2.2.8 root# /bin/rm -r /usr/local/apache/manual
 /usr/local/src/httpd-2.2.8 root# /usr/local/apache/bin/apachectl configtest
 /usr/local/src/httpd-2.2.8 root# /usr/local/apache/bin/apachectl start

If you have any problems, take a look at /var/www/logs/error_log. This is a very basic and clean Apache web server configuration. It is a starting point from which we will build upon in future postings.

Conclusion

I started this post with a quote from William Faulkner concerning how “only when the clock stops does time come to life.” Or if you prefer the despair.com quote, “Get to work: You aren’t being paid to believe in the power of your dreams.” I’ll be honest with you, doing a post on Apache implementation is not my idea of an exciting post. I would much rather jump ahead and start talking about securing web applications at a higher level. Sometimes, one has to build up to the more exciting stuff in order to demonstrate that one is not just selling pipe dreams with no real way to make those ideas a reality. That is the difference between science and science fiction. Bernard of Chartres once wrote, “We are like dwarfs standing upon the shoulders of giants, and so able to see more and see farther than the ancients.” Only the hard work of the ancients has allowed us to see further and dream bigger. At some point, to make those dreams a reality, getting to work in the annoying details of life is a requirement.

• Apache Security by Ivan Ristic
• Securing Apache 2: Step-by-Step by Ivan Ristic
• Apache HTTP Server Version 2.2: Compiling and Installing
• Apache HTTP Server Version 2.2: Security Tips
• Center for Internet Security Benchmarks for Apache Web Server v2.1 by Ryan Barnett
• Securing Apache Step by Step by Ryan C. Barnett
• ModSecurity Reference Manual
• NIST SP 800-44 v2: Guidelines on Securing Public Web Server by Miles Tracy, Wayne Jansen, Karen Scarfone, and Theodore Winograd
• NIST SP 800-95: Guide to Secure Web Services by Anoop Singhal, Theodore Winograd, and Karen Scarfone
• OWASP WeBekci Project, a web based ModSecurity 2.x management tool.
• REMO, a project to build a graphical rule editor for ModSecurity with a positive/whitelist approach.
• Got Root, the Internets largest source of intrusion prevention signatures and comment spam blacklists for webservers, over 13,000 signatures.

There are two freely available tools for helping with the security of your Apache configuration:

• The CIS Scoring Tool for Apache
• Apache httpd Tools

A coworker was complaining that the majority of information he was finding in blogs was junk. I asked him how was he finding his information. He was doing a regular Google search; not even a Google Blog Search. I understood his pain. George Siemens makes a very interesting distinction between collective intelligence and connective intelligence. Collective intelligence is “a form of intelligence that emerges from the collaboration and competition of many individuals“. George defines connective intelligence as “individual creation of information, ideas, and concepts which are then shared with others, connected, and re-created and extended based on the interaction.”

George goes on to state, “simply, collective means blending together. Connective means connecting while retaining the original (though others may build on it in their own spaces).” Put another way, “the collective presents a melting pot of ideas. The connective represents a mosaic of ideas.” People are surprised when I tell them that I do not read blogs. I read Ivan Ristic, Jeremiah Grossman, Gunnar Peterson, Ryan Barnett, Dafydd Stuttard, etc. My coworker’s problem is that he’s drowning in the melting pot of information provided by collective intelligence. When I read an author I like or come across software I find really useful, I look to see if the authors have a blog. I will then subscribe to their RSS feed, allowing me to make use of connective intelligence.

A few blogs of interest for web application security:

• Ivan Ristic, author of “Apache Security” and principal author of ModSecurity, the open source web application firewall.
• Jeremiah Grossman, author of the CIS Scoring Tool for Apache and founder and Chief Technology Officer of WhiteHat Security.
• Ryan Barnett, author of “Preventing Web Attacks With Apache“, and Director of Application Security Training for Breach Security.
• Dafydd “PortSwigger” Stuttard, co-authof of “The Web Application Hacker’s Handbook” and Principal Security Consultant at NGS Software.
• Gunnar Peterson, Software Security Architect and CTO at Arctec Group.
• Robert “RSnake” Hansen, CEO SecTheory.
• Shreeraj Shah, founder of Blueinfy, a company that provides application security services.
• Billy Hoffman, co-author of “Ajax Security” and lead research engineer with Atlanta-based SPI Dynamics Inc.
• Bryan Sullivan, co-author of “Ajax Security” and developer and security researcher at SPI Dynamics, Microsoft.
• Chris Shiflett, author and speaker who leads the web application security practice at OmniTI.
• Ory Segal, Security Products Architect, Rational, Application Security (Watchfire), IBM.
• Anurag Agarwal, is a senior application security consultant providing expertise on secure development lifecycle and vulnerability assessment. He also manages www.attacklabs.com and www.myappsecurity.com.

I wanted to mention that I started off with the names of several web application professionals. I wanted to include links to their names in this post. As I searched out their names to add a little background blurb, I kept coming across postings from Anurag Agarwal. He has done a great job profiling many of the leaders in web application security. The above list is missing many people and that is entirely my fault. As I stated, the list is of people that I am familiar with and is not meant to be a complete list of web application security professionals.

With these resources at our disposal, we are well positioned to start our quest to secure Apache.