System Advancements at the Monastery

This week I had the pleasure of presenting two talks at the National Laboratories Information Technology (NLIT) 2009 Summit held in Oak Ridge, TN. Everyone involved was great and I had a fun time. Since the presentations have been posted to the NLIT site, I am free to post now.

The original slides made heavy use of the Microsoft PowerPoint animation feature. Unfortunately, SlideShare does not currently support animation. You can download the presentation and the animations will work, but I ended up modifying the slides so they are more viewable online. SlideBoom will keep the animation, but it does it by creating a video of the presentation. I decided to stick with SlideShare and spare you the resulting nine minute video. While I should add audio and make a SlideCast, this post might never be completed if I wait until I have time to create a really nice web presentation.

Merriam-Webster defines a totem as any supposed entity that watches over or assists a group of people, such as a family, clan, or tribe. In this presentation I focused on how TOTEM assists in watching over and evaluating the threat an IP represents. The idea behind TOTEM is simple: compare threat information from sources such as watchlists (DShield, Emerging Threats, SenderBase, etc.) to activities with the organization (IDS/IPS, flow logs, etc.) and other locations (SANS ISC, DOE federated model, etc.). As new threat information and activity sources are added, a better evaluation can be rendered.

The purpose of this presentation has been to share the basic ideas behind TOTEM with the hope that others may provide helpful insight. So far I have not disappointed. I wanted to thank everyone for I have received some very intriguing ideas.

After working through the steps involved in “Setting Up Axiis for Security Visualization,” I became interested in taking a look at Prefuse Flare. Flare is a visualization toolkit written for Flash in ActionScript. It is a creation of the UC Berkeley Visualization Lab. Sasha Dzeletovic, a person who has worked with Flare, has done a very nice post comparing the two, “Flare vs. Axiis.” Both Tom Gonzalez, one of the co-founders of Axiis, and Sasha seem to agree that one big difference between Flare and Axiis is that, to quote Tom, “Axiis allows you to describe complex layout algorithms in markup in a more concise, and I like to think, intuitive manner.”

The Flare tutorial provides some useful links to resources intended to help with ActionScript 3:

• Adobe provides an Overview of AS3, with links to additional resources.
• Essential ActionScript 3 by Colin Moock from O'Reilly publishing is a great book to help you get started. You can access it online here (some institutions, such as universities, provide access for free).
• The Adobe Flex API Reference is invaluable for understanding the different classes and methods available. We will be focused only on the classes in the flash.* packages.

Installation is very similar to Axiis. Install Flex SDK and/or Adobe Flex Builder. Installing Flex Builder is straight forward, but regrettably development for Adobe Flex Builder for Linux has been put on hold. We will be using Flex SDK. We will want Apache Ant. Flare is package with a build.xml file, so all we need to do is change the first few lines to point to out Flex SDK installation, and we are ready to use ant to compile the libraries.

While I could duplicate the steps for installing Flex SDK and Apache ant, it is simpler to point to “Setting Up Axiis for Security Visualization.” Once you complete those steps, you are ready to install Flare. Continuing from my previous post, let us make sure the required software is installed and our environmental variables are properly set:

root# declare -x JAVA_HOME="/usr/java/latest"
root# declare -x PATH="${JAVA_HOME}/bin:${PATH}"
root# java -version
java version "1.6.0_13"
Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
Java HotSpot(TM) Server VM (build 11.3-b02, mixed mode)

root# declare -x ANT_HOME="/work/software/ant"
# ant -v
Apache Ant version 1.7.1 compiled on June 27 2008
Buildfile: build.xml does not exist!
Build failed

root# declare -x PATH="${PATH}:${ANT_HOME}/bin"
root# export PATH=/work/software/flex/bin:${PATH}
root# mxmlc --version
Version 3.3.0 build 4852

At this point, we are interested in downloading the Flare file and unzipping it.

root# cd /work/software
/work/software root# mkdir flare
/work/software root# cd flare
/work/software/flare root# wget http://flare.prefuse.org/download
/work/software/flare root# unzip prefuse.flare-alpha-20090124.zip
/work/software/flare root# vi build.xml

Now we need to modify the first few lines of the build.xml file that is part of the Flare zipped file. We will want to adjust FLEX_HOME, asdoc.

<?xml version="1.0" encoding="utf-8"?>
<project name="flare" default="usage" basedir=".">
  <!-- ================================================ -->
  <!-- Import Flex Ant Tasks, Set Flex Home and asdoc   -->
  <!-- ================================================ -->
  <property name="LOCALE" value="en_US"/>
  <property name="FLEX_HOME" value="/work/software/flex/"/>
  <property name="asdoc" value="${FLEX_HOME}bin/asdoc"/>

At this point, build all Flare targets with ant.

/work/software/flare root# ant all
/work/software/flare root# ls build
DependencyGraph.swf  flare.swc        JobVoyager.swf
flare.demos.swf      flare.tests.swf  PackageMap.swf

You can now use your favorite browser to open file /work/software/flare/build/flare.demos.swf. The program will call the other Flash executables.

The Flare tutorial provides a great starting point in working with Flare. The post “Flex and Flare Installation (K)Ubuntu 8.10” will take you through building some of the examples from the tutorial.

Check out the different examples and start experimenting. Two other projects that caught my attention and might be of interest:

• JuiceKit, which is for building graphically rich and interactive information displays. JuiceKit is particularly interesting because it is based on Flare.
• BirdEye, which states “the actionscript-based library enables users to create multi-dimensional data visualization interfaces for the analysis and presentation of information.” BirdEye caught my attention because it was listed in the article “Visualizing Relational Data Using Graph Theory” by Jason Bellone and Daniel Lang.

With Axiis and Prefuse Flare, you have some powerful tools to start visualizing your security data.

Last week Axiis, an open source data visualization framework, was released. Axiis is built upon Degrafa, an open source declarative graphics framework, and Adobe Flex 3. Thanks to Nathan Yau for pointing this out in his post “Open Source Data Visualization Framework – Axiis.” There were also great comments to Nathan’s post, including a comparison of Axiis to Berkley’s Prefuse Flare. Flare is also a visualization toolkit, but it is written for Flash in ActionScript 3. For a discussion of Flash, Flex, and ActionScript 3, check out Lee Brimelow post “Flash Builder rebrand FAQ.”

Today we are going to walk through the steps of setting up Axiis on a Linux platform.

Install a Java JDK

Get the Sun JDK 6 from Sun’s website. Sun requires you to agree to terms, so you’ll need to go there and agree. Run the installer which gets downloaded. Agree again to the terms. The installer will install a few rpms and jars.

root# /bin/sh jdk-6u13-linux-i586-rpm.bin
root# ls /usr/java
default  jdk1.6.0_13  latest
root# declare -x JAVA_HOME="/usr/java/latest"
root# declare -x PATH="${JAVA_HOME}/bin:${PATH}"
root# java -version
java version "1.6.0_13"
Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
Java HotSpot(TM) Server VM (build 11.3-b02, mixed mode)

Install Another Neat Tool (ANT)

Apache Ant is a free and open source tool for automating software build processes. It is similar to make but is implemented using the Java language using XML configuration files. It is platform-neutral.

root# cd  /work/src/
/work/src root# wget http://www.uniontransit.com/\
apache/ant/binaries/apache-ant-1.7.1-bin.tar.gz
/work/src root# md5sum apache-ant-1.7.1-bin.tar.gz
cc5777c57c4e8269be5f3d1dc515301c  apache-ant-1.7.1-bin.tar.gz
/work/src root# tar xzf apache-ant-1.7.1-bin.tar.gz
/work/src root# mv apache-ant-1.7.1 /work/software
/work/src root# cd  /work/software
/work/software root# ln -s apache-ant-1.7.1 ant
/work/software root# declare -x ANT_HOME="/work/software/ant"
/work/software root# declare -x PATH="${PATH}:${ANT_HOME}/bin"
/work/software root# ant
Buildfile: build.xml does not exist!
Build failed

The error above indicates that ant command is recognized by shell but it did not find build.xml file that needed to compile ant projects. So, it’s absolutely normal and the installation was successful. We will be creating build.xml files below.

Install the Flex SDK

The Adobe Flex 3.3 Software Development Kit (SDK) includes the Flex framework (component class library) and Flex compiler along with:

• Automated testing framework.
• Memory/Performance profiler.
• Certain components, such as the charting and the AdvancedDataGrid component (the code will work, but a watermark image is layered over it).

Installation is fairly straight forward.

root# cd  /work/software
/work/software root# mkdir flex
/work/software root# cd flex
/work/software/flex root# wget \
http://download.macromedia.com/pub/flex/sdk/flex_sdk_3.zip
/work/software/flex root# unzip flex_sdk_3.zip
/work/software/flex root# export PATH=/work/software/flex/bin:${PATH}
/work/software/flex root# mxmlc --version
Version 3.3.0 build 4852

Test the installation out by creating the traditional “hello world” program. First, under your favorite editor, create the file helloworld.mxml file:

/home/ger/flex root# vi helloworld.mxml
<?xml version="1.0" encoding="utf-8"?>
<mx:Application
    xmlns:mx="http://www.adobe.com/2006/mxml"
    width="300" height="200"
    horizontalAlign="center" verticalAlign="middle"
    viewSourceURL="src/HandlingEventsEventHandler/index.html"
>
<mx:Panel title="My first Flex application" horizontalAlign="center" >
    <mx:Label fontSize="18" text="Hello, World." />
</mx:Panel>
</mx:Application>

Compile the program:

/home/ger/flex root# mxmlc --strict=true helloworld.mxml
/home/ger/flex root# ls helloworld.swf
helloworld.swf

Use your favorite browser to view the helloworld.swf flash file. You now need to copy the flexTasks.jar file to the ant’s lib folder. The flexTasks.jar file contains the ant task definitions for compiling flex applications.

root# cd  /work/software/flex
/work/software/flex root# cp /work/software/flex/ant/lib/flexTasks.jar \
/work/software/ant/lib

Create a new file build.xml with the following content (make the appropriate changes to FLEX_HOME and APP_ROOT):

/home/ger/flex root# vi build.xml
<?xml version="1.0" ?>
<project name="helloworld" default="build" >
    <property name="FLEX_HOME" value="/work/software/flex/" />
    <property name="APP_ROOT" value="/home/ger/flex/" />
    <taskdef
        resource="flexTasks.tasks"
        classpath="${FLEX_HOME}/ant/lib/flexTasks.jar" />
    <target name="build">
        <mxmlc file="${APP_ROOT}/helloworld.mxml">
            <load-config filename="${FLEX_HOME}/frameworks/flex-config.xml"/>
            <source-path path-element="${FLEX_HOME}/frameworks"/>
        </mxmlc>
    </target>
</project>

From the directory where build.xml file exist, execute ant:

/home/ger/flex root# ant
Buildfile: build.xml

build:
    [mxmlc] Loading configuration file /work/software/flex/frameworks/flex-config.xml
    [mxmlc] /home/ger/flex/helloworld.swf (181669 bytes)

BUILD SUCCESSFUL
Total time: 7 seconds

The flash file helloworld.swf was created when you ran the command ant. As with the make command, you now have a way to specify build options within the build.xml file. See “Talk:Flex Ant Tasks” for a much more detail description of setting up ant to perform Flex tasks.

Eclipse

Eclipse is an open source plug-in—based editor and IDE framework. It does a good job of code editing while allowing third parties to extend its capabilities through modules. We are going to pull down the “Eclipse IDE for C/C++ Developers” bundle, which already contains the CDT (C/C++ Development Tools). Your Java version will need to be 1.5 or higher.

/home/ger/flex root# cd /work/src
/work/src  root# wget http://mirrors.med.harvard.edu/eclipse//technology/epp/downloads/\
release/ganymede/SR2/eclipse-cpp-ganymede-SR2-linux-gtk.tar.gz
/work/src root# tar xzf eclipse-cpp-ganymede-SR2-linux-gtk.tar.gz
/work/src root# mv eclipse /work/software
/work/src root# su - ger
[ger@loxias ~]$ declare -x JAVA_HOME="/usr/java/latest"
[ger@loxias ~]$ declare -x PATH="${JAVA_HOME}/bin:${PATH}"
[ger@loxias ~]$ java -version
java version "1.6.0_13"
Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
Java HotSpot(TM) Server VM (build 11.3-b02, mixed mode)
[ger@loxias ~]$ /work/software/eclipse/eclipse

Max Berger has written a great HOWTO “Setting up Eclipse” that will walk you through using Eclipse.

Aptana

One Eclipse plug-in, Aptana Studio, comes in both commercial and Community Edition. Since you have already installed Eclipse, you can install Aptana as a plug-in directly into your current Eclipse configuration. Follow the directions from the Aptana site.

1. From the Help menu in Eclipse, select Software Updates …
2. Select the Available Software tab
3. Click the “Add Site…” button.
4. Specify the Location Url update site: http://update.aptana.com/update/studio/3.4/ and click OK
5. Select the checkbox next to the added update site.
6. Click the Install.. button.
7. Complete instruction to install from update site.

If you would like to use the Aptana perspective, navigate and select Window > Open Perspective > Aptana. You will also want to change the default editor.

Flex Builder

Using the Flex SDK is more complicated and involves more work than using Flex Builder. Flex Builder comes with many nice features, such as tag completion, code hinting, built-in API reference manual, automatic importing of libraries, wizards, visual layout and styling, and automatic builds. There is a Flex Builder version for Linux, though it is a plugin-only version and unfortunately development has been put on hold. If you work in the education field, Adobe Flex Builder 3 Pro is available for free under Microsoft Windows and Mac OS X.

To get Flex Builder running on your system is simple:

root# cd  /work/src/
/work/src root# wget http://download.macromedia.com/pub/labs\
/flex/flexbuilder_linux/flexbuilder_linux_install_a4_081408.bin
/work/src root#  /bin/sh flexbuilder_linux_install_a4_081408.bin

At which point, the installation program will walk you through the installation process asking for the Eclipse folder. If you are following this document, the Eclipse directory would be /work/software/eclipse.

/work/src root#  su - ger
[ger@loxias ~]$ declare -x JAVA_HOME="/usr/java/latest"
[ger@loxias ~]$ declare -x PATH="${JAVA_HOME}/bin:${PATH}"
[ger@loxias ~]$ /bin/sh /work/software/Adobe_Flex_Builder_Linux/Adobe_Flex_builder.sh

Once you are running Eclipse, yo may choose the traditional Flex Builder perspective by navigate and select Window > Open Perspective > Other… > Flex Development to change it to the traditional Flex Builder look.

Degrafa

Degrafa is a declarative graphics framework that allows you to draw shapes and objects in your Flex application using an MXML type syntax. Degrafa is required for Axiis. Juan Sanchez in the post “An Introduction to Degrafa” provides a nice introduction.

root# cd  /work/software/
/work/software root# mkdir  degrafa
/work/software root# cd degrafa
/work/software/degrafa root# wget http://degrafa.googlecode.com/files/Degrafa_Beta3.1_Flex3.zip
/work/software/degrafa root# unzip Degrafa_Beta3.1_Flex3.zip
/work/software/degrafa root# cp /work/software/degrafa/Degrafa_Beta3.1_Flex3.swc  \
/work/software/ant/lib

I ran into memory problems with the Linux version of Flex Builder. It may be due to the software being Alpha. For the rest of this document, I will use ant to compile my programs with the idea that I am showing the more difficult method. If Flex Builder is working well for you, the ideas below end up being the same. It is always good to have an idea of what is going on behind a nice GUI application like Eclipse.

Below we walk through pulling down sample Degrafa code and compile the code. Please note that I start using the compiler.include-libraries attribute to specify the Degrafa library needs to be used in compiling the DegrafaPieMenu.mxml program.

root# cd  /home/ger/workspace
/home/ger/workspace root# mkdir ex3
/home/ger/workspace root# cd ex3
/home/ger/workspace/ex3 root# wget http://www.finflex.fi/projects/DegrafaPieMenu/srcview\
/DegrafaPieMenu.zip
/home/ger/workspace/ex3 root# unzip DegrafaPieMenu.zip
/home/ger/workspace/ex3 root# cd src
/home/ger/workspace/ex3/src root# vi build.xml
<?xml version="1.0" ?>
<project name="DegrafaPieMenu" default="build" >
    <property name="FLEX_HOME" value="/work/software/flex/" />
    <property name="APP_ROOT" value="/home/ger/workspace/ex3/src/" />
    <taskdef
        resource="flexTasks.tasks"
        classpath="${FLEX_HOME}/ant/lib/flexTasks.jar" />
    <target name="build">
        <mxmlc file="${APP_ROOT}/DegrafaPieMenu.mxml">
            <load-config filename="${FLEX_HOME}/frameworks/flex-config.xml"/>
            <source-path path-element="${FLEX_HOME}/frameworks"/>
            <compiler.include-libraries dir="${FLEX_HOME}/ant/lib/" append="true">
               <include name="DegrafaLibrary.swc" />
            </compiler.include-libraries>
        </mxmlc>
    </target>
</project>

At this point, use the command ant to compile the program.

/home/ger/workspace/ex3/src root# ant
Buildfile: build.xml

build:
    [mxmlc] Loading configuration file /work/software/flex/frameworks/flex-config.xml
    [mxmlc] /home/ger/workspace/ex3/src/DegrafaPieMenu.swf (247701 bytes)

BUILD SUCCESSFUL
Total time: 8 seconds

Use your favorite web browser top open the file DegrafaPieMenu.swf. With the necessary software and libraries installed, we are finally ready for Axiis.

Axiis

We are going to pull down the Axiis libraries, place them with the Ant libraries, pull down sample Axiis code, setup a build.xml file, run ant, and view the results. We do everything the same as what we have done before.

root# cd  /home/ger/workspace
/work/software root# mkdir  axiis
/work/software root# cd axiis
/work/software/axiis root# wget http://axiis.googlecode.com/files/Axiis_Library.zip
/work/software/axiis root# unzip Axiis_Library.zip
/work/software/axiis root#  cp libs/* /work/software/ant/lib
/work/software/axiis root# cd /home/ger/workspace
/home/ger/workspace root# mkdir axiis
/home/ger/workspace root# cd axiis
/home/ger/workspace/axiis root# wget http://axiis.googlecode.com/files/Axiis_Examples.zip
/home/ger/workspace/axiis root# unzip Axiis_Examples.zip
/home/ger/workspace/axiis root# cd src
/home/ger/workspace/axiis/src root# vi build.xml
<?xml version="1.0" ?>
<project name="LineAreaSeriesExample" default="build" >
    <property name="FLEX_HOME" value="/work/software/flex/" />
    <property name="APP_ROOT" value="/home/ger/workspace/axiis/src/" />
    <taskdef
        resource="flexTasks.tasks"
        classpath="${FLEX_HOME}/ant/lib/flexTasks.jar" />
    <target name="build">
        <mxmlc file="${APP_ROOT}/LineAreaSeriesExample.mxml">
            <load-config filename="${FLEX_HOME}/frameworks/flex-config.xml"/>
            <source-path path-element="${FLEX_HOME}/frameworks"/>
            <compiler.include-libraries dir="${FLEX_HOME}/ant/lib/" append="true">
               <include name="DegrafaLibrary.swc" />
               <include name="Axiis_Library.swc" />
            </compiler.include-libraries>
        </mxmlc>
    </target>
</project>

Trying to compile this with ant/mxmlc, produces an error message stating “/home/ger/workspace/axiis/src/Examples/ExampleBackground.mxml(40): Error: unable to resolve ‘Examples/axiis_logo_shadow.png’ for transcoding“. This is because ExampleBackground.mxml is in the same directory as axiis_logo_shadow.png, but the program specifies it is in “Examples/axiis_logo_shadow.png.” Modify ExampleBackground.mxml to just have embedded in line 40 “source=’axiis_logo_shadow.png’”. In other words, do not include the “Examples/” path.

Compile with the ant command:

/home/ger/workspace/axiis/src root# ant
Buildfile: build.xml

build:
    [mxmlc] Loading configuration file /work/software/flex/frameworks/flex-config.xml
    [mxmlc] /home/ger/workspace/axiis/src/LineAreaSeriesExample.swf (547707 bytes)

BUILD SUCCESSFUL
Total time: 10 seconds

From your favorite web browser, open the file LineAreaSeriesExample.swf to see:

Final Thoughts

Examine the LineAreaSeriesExample.mxml file. On line 40, the data used in plotting the graph is specified with “<mx:String id=”payload” source=”data/LineSeriesData.csv”/>.” Examine the data/LineSeriesData.csv file. This would be the data file that gets generated by your security program. You could always modify the Flex program to access your favorite database. Adobe has a “Simple MySQL to Flex” example. There is also AS3FlexDB, which is an open source library that allows Adobe Flex applications to connect to a MySQL server. Alessandro Crugnola has written “Connect to MySQL in Adobe Flex using AS3FlexDB” to step you through the process. The point is, there are many possibilities. Play around with the different examples and start experimenting with different ways to visualize your security data. It is a great way to learn.