If you are going to be using RT, you need to get the “RT Essentials” book written by Jesse Vincent, Robert Spier, Dave Rolsky, Darren Chamberlain, and Richard Foley. It is a good reference and a quick read. For up-to-date information, see the RT Wiki and the Best Practical Solutions blog site.

Prerequisites

To start, please review the following posts:

1. An Apache Implementation
2. Apache and OpenSSL
3. PHP Implementation
• PHP as a Module
• PHP as a CGI
• PHP Configuration Modifications
4. Introduction to MySQL
5. Setting Up and Securing MySQL: References
6. Implementing a Web Application Firewall with ModSecurity

Install Software

With Apache, MySQL, PHP, OpenSSL, and ModSecurity installed, we are now ready to focus on software packages required by RT.

1. Installing expat.

Different operating systems will vary on whether expat, the XML parser, is installed. Expat is needed to complete the cpan install for XML::RSS. Check your particular operating system.

 root# cd /usr/local/src
 /usr/local/src root# wget http://downloads.sourceforge.net/expat/expat-2.0.1.tar.gz
 /usr/local/src root# tar xzf expat-2.0.1.tar.gz
 /usr/local/src root# cd expat-2.0.1
 /usr/local/src/expat-2.0.1 root# ./configure
 /usr/local/src/expat-2.0.1 root# make
 /usr/local/src/expat-2.0.1 root# make check
 /usr/local/src/expat-2.0.1 root# make install

2. Install FastCGI

For RT, you can install mod_perl or mod_fastcgi. In this posting, we are going to walks through the installation of FastCGI. Information concerning mod_perl will be provided below so the reader can chose what fits best in their environment. FastCGI is much simpler to install and allows the core Apache process to stay small in size. With FastCGI, RT runs as a separate process from Apache allowing RT to be stopped and restarted without affecting the Apache server. In general, FastCGI programs are easier to manage.

The Apache module mod_fastcgi allows a web server to run CGI scripts via a separate, persistent program. PHP comes with FastCGI support compiled in by default, so nothing needs to be done to the PHP installation.

You can have the Apache program call FastCGI, and have it run as the same user as the Apache server or use suexec to have FastCGI switch to a different user. Under some operating systems, suexec may not get compiled and installed when installing Apache. Check if suexec is installed, and if not go back to the Apache source, compile it, and install it. Initially, we are not going to use the suexec program. Instead we will create the group “rt”, add user httpd to group rt, and set permissions that way. You may choose later to use suexec.

 root# ls -la /usr/local/apache/bin/suexec
ls: /usr/local/apache/bin/suexec: No such file or directory
 root# cd /usr/local/src/httpd-2.2.8
 /usr/local/src/httpd-2.2.8 root# make suexec
 /usr/local/src/httpd-2.2.8 root# cp ./support/suexec /usr/local/apache/bin/suexec

Now, we are ready to get mod_fastcgi installed.

 root# cd /usr/local/src
 /usr/local/src root# wget http://www.fastcgi.com/dist/mod_fastcgi-2.4.6.tar.gz
 /usr/local/src root# tar xzf mod_fastcgi-2.4.6.tar.gz
 /usr/local/src root# cd mod_fastcgi-2.4.6
 /usr/local/src/mod_fastcgi-2.4.6 root# cp Makefile.AP2 Makefile
 /usr/local/src/mod_fastcgi-2.4.6 root# make top_dir=/usr/local/apache
 /usr/local/src/mod_fastcgi-2.4.6 root# make top_dir=/usr/local/apache install
 /usr/local/src/mod_fastcgi-2.4.6 root# /usr/local/apache/bin/apachectl stop
 /usr/local/src/mod_fastcgi-2.4.6 root# vi /usr/local/apache/conf/httpd.conf

Add the following lines to the Apache httpd.conf file:

# Load the mod_fastcgi module.
LoadModule fastcgi_module modules/mod_fastcgi.so

Check if installation and configuration is working.

 /usr/local/src/mod_fastcgi-2.4.6 root# /usr/local/apache/bin/apachectl configtest
Syntax OK
 /usr/local/src/mod_fastcgi-2.4.6 root# /usr/local/apache/bin/apachectl start
 /usr/local/src/mod_fastcgi-2.4.6 root# cat /var/www/logs/error_log | grep -i fastcgi
[Fri Aug 01 12:17:22 2008] [notice] FastCGI: process manager initialized (pid 15221)
[Fri Aug 01 12:17:22 2008] [notice] Apache/2.2.8 (Unix) mod_ssl/2.2.8
OpenSSL/0.9.7a mod_fastcgi/2.4.6 configured -- resuming normal operations

For in depth coverage of mod_perl, Stas Bekman and Eric Cholet have written the book, “Practical mod_perl.” They have made the complete book available online in both HTML and PDF format under the Creative Commons Attributes Share-Alike License. Stas Bekman and Jim Brandt have also written the “mod_perl2 User’s Guide Book” where 50% of the book’s proceeds go to The Perl Foundation.

If you are installing under Mac OS X, mod_perl may complain about Perl 5.8.8 being built without threads and you will get a message about building perl with -Duserthreads. If you are determined to use mod_perl, consider dropping back to Apache 1.3.x and using mod_perl 1.x. While Apache 1.3.x is legacy code, and I tend to want to use the code that is being actively developed, there is an argument for using Apache 1.3.x. One major feature of Apache 2.x is threading. On Windows, where most basic libraries are and must be threadsafe, Apache 2 is really the only choice. Earlier Mac OS X releases did not include a completely thread-safe libc, so threading is still not fully supported in Perl. This is why the Perl version that comes with Mac OS X is not compiled to use threads. To use Apache2.x, Perl will need to be configured to use threads. The code is available from the Perl web site.

Rather than getting bogged down in compiling Perl to use thread, we will move ahead and use FastCGI. By the time this post, I will have worked on getting RT installed under Linux, Mac OS X, and FreeBSD. Figuring out what software works best in a multi OS environment can be challenging.

3. Configure RT

Let us start by adding the group RT. Under many operating systems, this would be done with the simple command “groupadd rt.” Things are always more interesting under Mac OS X, where you would have to first look at what group ids (gid), choose an unused gid, and then create the rt group using that gid. Under Mac OS X Leopard, group rt would be created with the commands:

 root# dscl . list /groups PrimaryGroupID | sort -k 2,2 -n
 root# dscl . create /groups/rt gid gid-of-rt
 root# dscl . create /groups/rt passwd '*'
 root# dscl . read /groups/rt
AppleMetaNodeLocation: /Local/Default
Password: *
PrimaryGroupID: gid-of-rt
RecordName: rt
RecordType: dsRecTypeNative:groups

RT’s primary maintenance and documentation site is http://www.bestpractical.com. Documentation can be found at the Best Practical Solutions RT Wiki located at http://wiki.bestpractical.com/. The latest TAR/GZ is located at http://download.bestpractical.com/pub/rt/release/rt.tar.gz. The lack of any version numbers means the version can be updated at any time. The latest version, as of this writing, is 3.8.0.

The following are the steps for downloading and configuring RT:

 root# cd /usr/local/src
 /usr/local/src root# wget http://download.bestpractical.com/pub/rt/release/rt.tar.gz
 /usr/local/src root# tar xzf rt.tar.gz
 /usr/local/src root# cd rt-3.8.0
 /usr/local/src/rt-3.6.5 root# ./configure \
  --with-web-user="httpd" \
  --with-web-group="httpd" \
  --with-rt-user="httpd" \
  --with-rt-group="rt"

4. Install Apache::TEST

Perl module Apache::TEST will not allow you to run the test check as root. You can download the module separately as a non root user and after configuring, compiling, and testing the program, you install it as root.

 root# su - goofy
 ~$ cd src
 ~/src goofy$ wget http://search.cpan.org/CPAN/authors/id/P/PH/PHRED/Apache-Test-1.30.tar.gz
 ~/src goofy$ tar xzf Apache-Test-1.30.tar.gz
 ~/src goofy$ cd Apache-Test-1.30
 ~/src goofy$ perl Makefile.PL
 ~/src goofy$ make
 ~/src goofy$ make test
 ~/src goofy$ sudo su root
 root# make instal

5. Run fixdeps Command and Install Perl Modules

Now you are ready to utilize the fixedeps utility that comes with RT to install required Perl modules. There is also the testdeps utility to test if all dependencies are installed and RT is ready to be installed. You may need to run fixdeps multiple times before testdeps reports that you have all required software packages. The first time through, it can take awhile (depending on your installation). Be aware that some perl modules may need to be installed manually. It various depending on OS and your environment. You will be able to tell which modules need manual installation by the final message provided by the fixdeps program.

 root# cd /usr/local/src/rt-3.8.0
 /usr/local/src/rt-3.8.0 root# make fixdeps
 /usr/local/src/rt-3.8.0 root# make fixdeps
 /usr/local/src/rt-3.8.0 root# make testdeps

6. Install RT

The final installation of RT is the easy part.

 /usr/local/src/rt-3.8.0 root# make install

7. Configure RT_SiteConfig.pm

We now will configure /opt/rt3/etc/RT_SiteConfig.pm. In the next step a database user and a database will be setup. We are only adding those values to the configuration file in this step. I am going to set up a hostname (rt.securitymonks.com) for my current machine. Please do not copy blindly. Change this to your environment. We will create the hostname so it only exists locally by adding an entry into the machines /etc/hosts file. Right now, I am only going to access the Apache server from this machine. In other words, the client and server will be on the same box.

 /usr/local/src/rt-3.8.0 root# vi /etc/hosts

Add the following line, adapting it to your organization:

 /usr/local/src/rt-3.8.0 root# vi /etc/hosts
##
127.0.0.1       localhost
10.1.218.202   rt.securitymonks.com

We are now ready to modify the RT_SiteConfig.pm file.

 /usr/local/src/rt-3.8.0 root# vi /opt/rt3/etc/RT_SiteConfig.pm

At minimum, add the following linesto /opt/rt3/etc/RT_SiteConfig.pm:

Set($rtname, 'BRORT');
Set($Organization, 'securitymonks');

Set($CorrespondAddress , 'john@securitymonks.com');
Set($CommentAddress , 'john@securitymonks.com');

Set($Timezone , 'US/Eastern'); # obviously choose what suits you

# THE DATABASE:

Set($DatabaseType, 'mysql'); # e.g. Pg or mysql

# These are the settings we used above when creating the RT database,
# you MUST set these to what you chose in the section above.

Set($DatabaseUser , 'rtuser');
Set($DatabasePassword , 'secret');
Set($DatabaseName , 'rtdb');

# THE WEBSERVER:

Set($WebPath , '');
Set($WebBaseURL , 'https://rt.securitymonks.com');

# Logging
Set($LogToSyslog, '');
Set($LogToFile, 'debug');
Set($LogDir, '/opt/rt3/var/log');
Set($LogToFileNamed, "rt.log");

8. Initialize the Database

RT needs to create the rtdb database, the rt db users, and initialize some tables. This can be done with the command initialize-database, which should be run only once.

 /usr/local/src/rt-3.8.0 root# make initialize-database
 /usr/local/bin/perl sbin/rt-setup-database --action init --dba root --prompt-for-dba-password
In order to create or update your RT database, this script needs to connect to your mysql
instance on localhost as root.  Please specify that user's database password below. If the
user has no database

password, just press return.

Password:
Working with:
Type:   mysql
Host:   localhost
Name:   rtdb
User:   rtuser
DBA:    root
Now creating a mysql database rtdb for RT.
Done.
Now populating database schema.
Done.
Now inserting database ACLs
Granting access to rtuser@'localhost' on rtdb.
Done.
Now inserting RT core system objects
Done.
Now inserting data
Done inserting data
Done.

Check the MySQL database out.

 /usr/local/src/rt-3.8.0 root# mysql -u rtuser -p
mysql> use rtdb;

9. Modify Apache Configuration File

Edit the /usr/local/apache/conf/httpd.conf file.

 /usr/local/src/rt-3.8.0 root# /usr/local/apache/bin/apachectl stop
 /usr/local/src/rt-3.8.0 root# vi /usr/local/apache/conf/httpd.conf

We are going to have the RT server run under our secure web server. Find the “<virtualhost _default_:443>” line, change it to “<virtualhost 10.1.218.202:443>“. Add the following lines to that section (adjusting to your environment):

   ServerName rt.securitymonks.com
   DocumentRoot /opt/rt3/share/html
   ErrorLog /usr/local/apache/logs/rt.error
   LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
   CustomLog /usr/local/apache/logs/rt.access_log combined
   AddHandler fastcgi-script fcgi
   ScriptAlias / /opt/rt3/bin/mason_handler.fcgi/

Add the user the Apache server runs as (httpd by default), to the RT group. For non Mac OS X, modify group membership by editing the file /etc/group (vi /etc/group). Mac OS X users need to user the dscl command.

 root# dscl . append /groups/rt GroupMembership httpd
 root# dscl . read /groups/rt

Change the group and permission on the log area if you have told RT to log to /opt/rt3/var/log.

 root# chgrp rt /opt/rt3/var/log
 root# chmod g+w /opt/rt3/var/log

Test the configuration of the file, and if everything checks out start up Apache.

 /usr/local/src/rt-3.8.0 root# /usr/local/apache/bin/apachectl configtest
Syntax OK
 /usr/local/src/rt-3.8.0 root# /usr/local/apache/bin/apachectl start

Remember there are now three files to check for problems with RT.

• /opt/rt3/var/log/rt.log
• /usr/local/apache/logs/rt.error
• /usr/local/apache/logs/rt.access_log

There are many configuration operations. The options chosen in this post represents only the minimal to get RT running. Please see the RT Wiki’s FastCGIConfiguration page for additional information.

10. Access RT and Change the Default Password

Now it is time to log in and change the default password. Using the entry we made in our /etc/hosts file, we can now access the site by going to https://rt.securitymonks.com. This URL should be different for your site. You will see a login screen similar to the image on the left.

Log in using the username “root” and password “password“. Once logged in, you will see the screen similar to the image below (click on the image if you need to enlarge):

Over on the left menu bar, select “Configuration.” That will bring you to the “RT Administration” screen:

Select, “Users.” That will bring you to the “Select a user” screen:

Select the user “root,” which will bring you to the “Modify the user root” screen. If you look at the lower left of the screen, there is a “Access Control area.” There is a place to enter “New Password.” Do so. The screen looks like:

Make sure to hit the “Save Changes” button at the bottom of the screen. With a working copy of RT, you are not ready to start adjusting configurations and working with the program. For additional information, Please check out the “RT Essentials” and the RT Wiki and the Best Practical Solutions blog site. Look for future posts to build upon the RT installation and database.

A little background. The MacPorts Project is an open-source community initiative to design an easy-to-use system for compiling, installing, and upgrading either command-line, X11 or Aqua based open-source software on the Mac OS X operating system.

Below are some basic instructions on installing and using the ports command. I initially posted this under “Installing Packages under MacPorts.” Later, while wrestling with Leopard, I added some addition information in the post “Backing Up Using Amanda on Mac OS X Leopard Part I.” This post is an attempt to pull together information buried in both posts.

If you upgraded to Leopard, you may run into problems with an the previous copy of MacPorts installed under Tiger. The simplest solution is to do a fresh install of MacPorts. Move /opt/local under another directory (so you have it in case you need to switch back) and reinstall MacPorts. A clean install of Leopard will bypass many problems but for many, upgrades are a fact of life that have to be dealt with.

Installation

As with many things involving computers, to do one thing requires several steps. Installing MacPorts is no different.

1. Install Xcode Tools

Xcode is Apple’s premiere development environment for Mac OS X.

1. Download the latest verson of Xcode Tools from Apple’s developer site —do not install an older version from the install disk or some ports may fail to install.
2. Run the binary installer.

Notes:
Xcode Tools is neither installed nor updated via the Software Update utility. You will want to makre sure you keep it up to date.

2. Install X Windows (X11)

Apple’s X11 is an optional install (MacPorts’ X11 package is not recommended at this time). If the application X11 is not present in /Applications/Utilities:

1. Insert the OS X 10.5 installation DVD and run the package named Additional Software.
2. At the software selection window expand the Applications category and click the checkbox beside X11 (and nothing else).
3. Click Install to install X11.
4. Drag the /Applications/Utilities/X11 icon to your dock —you must open X11 before launching an X11 application.

Notes:

• X Windows (X11) is required for many MacPorts apps and it is highly recommended that you install it even if you don’t plan to run X11 apps immediately.
• The X11SDK is also required to run X11 software, but it is installed by default during the Xcode Tools install above. If you wish to check to see if X11 (X11User) and X11SDK are installed, look in /Library/Receipts for the files X11User.pkg and X11SDK.pkg.
• If you have upgraded to Max OS X Leopard, please check your /usr/lib directory for any libraries ending with a ” 1″ (that is a space followed by 1). For example, “/usr/lib/ruby 1” and “/usr/X11R6 1“. These files are created, possibly as a backup copy, when Leopard or Xcode 3 is installed. If you fail to remove those libraries, you will get an error “i686-apple-darwin9-gcc-4.0.1: 1/lib: No such file or directory.”

3. Set the shell environment

Setting the Unix shell environment is critical for MacPorts to function. In you ~/.profile file, add the MacPorts paths in front of the standard UNIX paths for the default BASH shell. This is done so that if you have utilities or libraries from both MacPorts and OS X’s standard install, the MacPorts libraries will be run instead of the ones provided by Apple. You can change the PATH via the command line with the command:

export PATH=/opt/local/bin:/opt/local/sbin:$PATH

You’ll need to add this variable to your ~/.profile to run X11 applications:

export DISPLAY=:0.0

So a ~/.profile file might look like this:

export PATH=/opt/local/bin:/opt/local/sbin:$PATH
export DISPLAY=:0.0

Note:

• Changes to ~/.profile do not take effect on the current shell session — they take effect on new sessions so open a new shell session and type ‘env’ to see the path and variable changes.
• Other profile files may take precedence over ~/.profile. If you are using one of the following profile files in your home directory, you will need to merge the contents of it into ~./profile, or vice versa because the shell will ignore one of them:
  • ~/.bash_login
  • ~/.bash_profile

To verify that your ~/.profile has set your shell environment:

• Open a new terminal window
• Type the command: env

Check to make sure the reported path begins with the MacPorts’ paths as show above. Sample output of command env:

TERM_PROGRAM=Apple_Terminal
TERM=xterm-color
SHELL=/bin/bash
PATH=/opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin

4. Install MacPorts

To install MacPorts using the binary installer (to /opt/local/ only):

1. Download the latest MacPorts .dmg disk image, currently version 1.6.0.
2. Run the binary installer.
3. Perform the default “easy” install.

A MacPorts’ selfupdate (see below) will automatically update to the latest release if the binary .dmg image is behind the latest released tarball.

To install MacPorts from source (to any path except /usr/local/):

1. Download and unzip the latest MacPorts tarball from the MacPorts download area.
2. Perform these commands in a terminal window.
   

   cd /usr/local/src wget http://svn.macports.org/repository/macports/downloads/MacPorts-1.6.0/MacPorts-1.6.0.tar.gz tar xzf MacPorts-1.6.0.tar.gz cd MacPorts-1.6.0 ./configure make sudo make install

To see why /usr/local is not a viable install location for MacPorts, see the MacPorts FAQ. Developer releases contain untested code and may have bugs. Use at your own risk.

5. Update MacPorts via selfupdate

The port selfupdate command must be run after a new MacPorts install, and frequently thereafter. The command updates MacPorts with the latest port revisions, and, if necessary, updates MacPorts to the latest release.

1. Open a terminal window.
2. Execute the MacPorts selfupdate command.

   sudo port selfupdate
   

   or for verbose output:

   sudo port -d selfupdate
   

6. Installing Packages

Now you are going to start installing packages under MacPorts. A good place to see what packages have been ported over to MacPorts is to go to the main Darwin Ports Repository. It still has the old name, but it is the MacPorts site. It has a search area where you can enter the package name and find installation instructions.

For example, if you wanted to install the GNU multiple precision arithmetic library (GMP), you would execute the MacPorts install command for GMP.

sudo port install gmp

At this point, you should see the following output:

--->  Fetching gmp
--->  Attempting to fetch gmp-4.2.1.tar.bz2 from http://ftp.gnu.org/gnu/gmp
--->  Verifying checksum(s) for gmp
--->  Extracting gmp
--->  Configuring gmp
--->  Building gmp with target all
--->  Staging gmp into destroot
--->  Running ranlib on static libraries
--->  Installing gmp 4.2.1_4+test
--->  Activating gmp 4.2.1_4+test
--->  Cleaning gmp

Firewall Problems

For those trying to use MacPorts from a computer behind a corporate firewall, the firewall may block rsync access. To configure MacPorts to use some other method than rsync, do the following:

1. Check out a working copy of the ports tree to some place on your hard disk, such as the /usr/local/dports directory. Run portindex command so that ports now find these new ports.
   

   root # svn co http://svn.macports.org/repository/macports/trunk/dports \ /usr/local/dports root # cd /usr/local/dports root # portindex /usr/local/dports

2. Edit the file /opt/local/etc/macports/sources.conf. Comment out the line starting with “rsync://” and add a new line pointing to your working copy, in URL form. For example: file:///usr/local/dports

Using MacPorts

Below are a few port commands to help with basic operation and navigation. Please see Mark Duling, Dr. Michael A Maibaum, and Will Barton excellent guide/site, “MacPorts Guide” for more information.

contents

The contents option displays the files that have been installed by a given port. Uninstalled ports will always show no contents.

 root # port contents glib2
Port glib2 contains:
  /opt/local/bin/glib-genmarshal
  /opt/local/bin/glib-gettextize
etc.

deps

The deps option shows you the dependencies of a port.

 root # port deps glib2
glib2 has build dependencies on:
	pkgconfig
glib2 has library dependencies on:
	gettext
	libiconv

install

The option install is used to install a port.

 root # port install glib2
Skipping org.macports.activate (glib2 ) since this port is already active
--->  Cleaning glib2

installed

The installed option displays all installed ports.

 root # port installed

list

The list option is used to get a list of all available ports.

 root # port list

search

The search option is very useful to find port names by partial matches.

 root # port search glib2
glib2   devel/glib2   2.14.0   Library with data structure functions and other constructs

selfupdate

The selfupdate command updates MacPorts with the latest port revisions, and, if necessary, updates MacPorts to the latest release.

 root # port selfupdate

Final Word

MacPorts is a powerful tool that should be used in conjunction with software configuration and installation. As with any skilled craftsperson, use the right tool for the right job. Find a balance between keeping software up-to-date and taking the time to understand the software. To quote Louis Nizer, famous legal wizard, “A man who works with his hands is a laborer; a man who works with his hands and his brain is a craftsman; but a man who works with his hands and his brain and his heart is an artist”. Life is always about using every tool at your disposal, including hands, brain, and heart. Never limit yourself.

Linux creator Linus Torvalds talked with Nick Miller from the Sydney Morning Herald about his work and ambitions for his operating system. Linux expressed his preference for Mac OS X over Windows. The part that I found most entertaining was when he described the Mac OS X file system HFS+ as “complete and utter craps.” When asked, what he thought about the regular hype about the release of a new version of Microsoft Windows or Apple OS X, Linus responded:

An o/s should never have been something that people (in general) really care about: it should be completely invisible and nobody should give a flying f*** about it except the technical people.

It’s stupid – when you make a big deal about something like Vista or Leopard a lot of it is about things I don’t consider to be the operating system. It’s about the visual shell around it. The fact Microsoft tied the two together so much actually caused them problems, not just the legal problems. If you manage a thousand clients, or a hundred thousand clients which is not at all unheard of, you sure as hell don’t want to point and click at them. In many ways Microsoft has had to fix the design mistakes they made when they thought the graphical approach should be a very intimate part of (Windows).

To Microsoft and Apple the o/s is important as a way to control the whole environment, from a marketing and money-making standpoint, to force people to upgrade their applications, and your hardware

When asked, if he had a favorite between Leopard and Vista, Linus responded:

I don’t think they’re equally flawed. I think Leopard is a much better system. On the other hand, (I’ve found) OS X in some ways is actually worse than Windows to program for. Their file system is complete and utter crap, which is scary. I think OS X is nicer than Windows in many ways, but neither can hold a candle to my own (Linux). It’s a race to secondplace!

In my last post “Bro on FreeBSD 7: Should You Care?”, ZFS was listed as one of the improvements of FreeBSD 7. The posts “The Advantages of Sun’s ZFS Filesystem” and “Ten Reasons to Reformat Your Hard Drives” discuss some of the advantages of ZFS. Blogger jamesd_wi posted a very nice collection of links concerning ZFS titled “Interested in ZFS.” Paul Venezia’s article in InfoWorld titled, “Sun ZFS breaks all the rules” does a real nice job of hitting the high points.

To address Linus comment on HFS+, Apple is developing ZFS for Mac OS X. Noel Dellofano, who is part of the ZFS development team, did release binaries and source code of ZFS for Mac OS X on the MacOSForge site. Noel cautioned in his post, “As I mentioned, ZFS is still BETA, so there are (and likely will be) some issues turn up with compatibility with the upper layers of the system if that’s what you’re referring to.” The Mac OS forge has created a “Known Issues and Features in the Works” area to help keep people apprised of the development of ZFS under Mac OS X.

What does Linus have to say about recent file system development, and in particular ZFS? Swapnil Bhartiya talked to Linus in his article,”Linus Torvalds: I Have Never Really Talked To Microsoft!” On the question of ZFS, Linus responed

Actually, just yesterday we had a git performance issue, where ZFS was orders of magnitude slower than UFS for one user (not under Linux, but git is gaining a lot of traction even outside of kernel development). So I think a lot of the ‘new file system’ mania is partly fed by knowing about the issues with old file systems, and then the (somewhat unrealistic) expectation that a ‘new and improved’ file system will make everything perfect.

In the end, this is one area where you just let people fight it out. See who comes out the winner — and it doesn’t need to be (and likely will not) be a single winner. Almost always, the right choice of file system ends up depending on the load and circumstances.

One thing that I’m personally more excited about than any of the file systems you mention is actually the fact that Flash-based hard disks are quickly becoming available even for ‘normal’ users. Sure, they’re still expensive (and fairly small), but Flash-based storage has such a different performance profile from rotating media, that I suspect that it will end up having a large impact on file system design. Right now, most file systems tend to be designed with the latencies of rotating media in mind.

A very interesting point concerning technological innovation. Linus does in a post state that ZFS is “one of their very very few bright spots.” By “their” he means Sun. Linus problem with ZFS is that “ZFS and the other things that people are drooling about in a way that lets Linux use them on an equal footing. I can pretty much guarantee that. They don’t like competition on that level. They’d *much* rather take our drivers and _not_ give anything back, or give back the stuff that doesn’t matter.” Linus goes on to say, “And yes, maybe ZFS is worthwhile enough that I’m willing to go to the effort of trying to relicense the kernel. But quite frankly, I can almost guarantee that Sun won’t release ZFS under the GPLv3 even if they release other parts. Because if they did, they’d lose the patent protection.”

Jonathan Schwartz, Chief Executive Officer and President Sun Microsystems, Inc., responded in his blog post “An OpenSolaris/Linux Mashup.” Jonathan writes, “We want to work together, we want to join hands and communities – we have no intention of holding anything back, or pulling patent nonsense. And to prove the sincerity of the offer, I invite you to my house for dinner. I’ll cook, you bring the wine. A mashup in the truest sense.” We will leave Jonathan and Linus to settle their issues. If you are interested, you can listen to Linus on the Linux Foundation Podcast. They have two part interview (part 1, part 2) where Linus comments on a variety of topics including patents, internal and external competition, the broader adoption of Linux, Microsoft and much more. If you want to know more about GPLv3, Eben Moglen, a leading advocate of open source rights, gives the talk “The Renaissance of Invention: Free Software and the Next American Century” on the IT Conversations podcast. In the talk, Eben addresses the controversy about GPLv3 and Linux, and whether software patents are more dangerous than other patents. To some, licensing issues might not be the most exciting topic, but it is important to be aware of the issues. Development and adoption of projects, such as ZFS, can be affected.

Drew Thaler has a few things to say about ZFS compared to HFS+. Drew worked in Apple’s CoreOS filesystems group. Before that, he worked on DiscRecording.framework, and singlehandedly created the content subframework that streamed out HFS+, ISO-9660, and Joliet filesystems. Before that, he worked on the same thing for Mac OS 9. And before that, he worked on mass storage drivers for external USB/FireWire drives and internal ATA/ATAPI/SCSI drives. Currently he works on a file I/O subsystem for PlayStation 3 games. In short, Drew knows what he is talking about. He wrote the post, “Don’t be a ZFS Hater” and a follow up post “ZFS Hater Redux” which contain some amazing information. Drew’s makes a very important point when he writes:

Some very smart people at Sun started the ball rolling by putting an awful lot of thought into the future of storage, and they came up with ZFS.

After they announced it and started talking about it, other brilliant people at Apple (and FreeBSD, and NetBSD) paid attention to what they were doing. And they listened, and thought about it, and looked at the code, and wound up coming around to the side of ZFS as well.

The original HFS design is twenty years old. It has had a good run, but so much has changed in the world of computers. As Linus pointed out, technology changes and file systems need to be redesigned for these changes. Some brilliant folks have worked on ZFS . It is not a patched file system trying to handle the latest requirements of computers. ZFS is a file system designed with the flexibility to handle data requirements for the next ten years. It will not be the final word in file systems. The world continues to innovate and change. The point is, ZFS is here now after seven years of development. It sets a new standard for future file systems. I look forward to seeing ZFS implemented in the latest Mac and FreeBSD operating systems.