Consider how valuable it would be to combine the monitoring and automation of system tasks with rapid and consistent responses across an organization. Puppet is an automated administrative engine capable of automating nearly every aspect of a system administrator’s job, from user management, to software installation, to even configuring server services. Puppet is written in Ruby, but uses a configuration management abstraction language. This allows for the modeling of resources instead of describing in details how to configure resources. With Puppet you can perform normal administrative tasks (such as adding users, installing packages, and updating server configurations) on any number of systems, even if those systems are running completely different operating systems.
Michael Coté, an industry analyst with RedMonk, interviewed Reductive Lab’s Luke Kaines and Google’s Nigel Kersten on the podcast “Puppet at Google – RedMonk Radio Episode 48.” To quote Coté, “Nigel has been using Puppet to manage ‘many, many thousands’ of Mac desktops used at Google by developers and others. He tells us how he got involved in using Puppet during WWDC last year and quickly applied its use to managing Google Mac desktop.” Pat Eyler also posted, on the On Ruby blog, an interview with James Turnbull. Turnbull is the author of “Pulling Strings with Puppet.” To quote Turnbull on Puppet, “It takes all the pain out of systematically and efficiently managing your host.”
Phil Windley, Founder and Chief Technology Officer (CTO) of Kynetx has been providing some great information on Puppet. Phil recently attended the O’Reilly Velocity 08 conference where he attended a talk given by Luke Kanies. Phil shares his impression in the post “Velocity 08: Puppet In-Depth and Hands-On.” Phil is also the host of IT Conversations’ Technometria podcast. He must have liked what he heard at Velocity 08, for he had Luke Kanies on his show to discuss Puppet. Plus, Phil has been sharing his experience implementing Puppet in his posts “Puppet Fun” and “Using a Pre-Commit Hook to Check Puppet Syntax.”
The point of today’s post is not to discuss how useful Puppet can be. Its worth can be best determined through examining how it is used. I have provided a few links to people making good use of Puppet. I will also provide follow up posts describing integrating Puppet with other security software we have been discussing. Today’s post is simply to set the groundwork by focusing on implementing a basic Puppet setup.
Prerequisites
Most of what Puppet requires will come as part of the Ruby standard library. It is best to use the Ruby that comes with your OS. I will still go through the installation of Ruby. The only other required package is Reductive Lab’s Facter, which is a cross-platform Ruby library for retrieving facts from operating systems.
The Ruby standard libraries that are required by Puppet include:
- base64
- cgi
- digest/md5
- etc
- fileutils
- ipaddr
- openssl
- strscan
- syslog
- uri
- webrick
- webrick/https
- rdoc
- xmlrpc
Installation
In order to install all the required packages, we will start by installing Ruby, followed by checking that the required libraries are installed, and finish with Facter. If any of these software packages are installed by default on your OS, you will likely want to skip the corresponding installation steps.
Ruby
The Ruby website describes Ruby as “a dynamic, open source programming language with a focus on simplicity and productivity. It has an elegant syntax that is natural to read and easy to write.” Below are the steps to do a source code installation. Please note that there are other options listed in the Download Ruby page. I will be going through installation using the source code only because there are some systems that do not have Ruby installed. If you are going to be working with Ruby, you may consider purchasing a great reference on Ruby titled “Programmers’ Guide, 3rd Edition” by Dave Thomas, with Chad Fowler and Andy Hunt.
root# cd /usr/local/src /usr/local/src root# wget ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.7-p72.tar.gz /usr/local/src root# /usr/bin/md5sum ruby-1.8.7-p72.tar.gz 5e5b7189674b3a7f69401284f6a7a36d ruby-1.8.7-p72.tar.gz /usr/local/src root# tar xzf ruby-1.8.7-p72.tar.gz /usr/local/src root# cd ruby-1.8.7-p72 /usr/local/src/ruby-1.8.7-p72 root# ./configure /usr/local/src/ruby-1.8.7-p72 root# make /usr/local/src/ruby-1.8.7-p72 root# make check /usr/local/src/ruby-1.8.7-p72 root# make install |
RubyGems
RubyGems is the premier Ruby packaging system. Quoting the RubyGems website, “it provides a standard format for distributing Ruby programs and libraries, an easy to use tool for managing the installation of gem packages, and a gem server utility for serving gems from any machine where RubyGems is installed.” Using RubyGems makes handling libraries a great deal easier. If you are behind a proxy server, you will either need to set HTTP_PROXY or include the proxy setting in the gem command.
root# cd /usr/local/src /usr/local/src root# wget http://rubyforge.org/frs/download.php/43985/rubygems-1.3.0.tgz /usr/local/src root# tar xzf rubygems-1.3.0.tgz /usr/local/src root# cd rubygems-1.3.0 /usr/local/src/rubygems-1.3.0 root# ruby setup.rb /usr/local/src/rubygems-1.3.0 root# gem --version 1.3.0 /usr/local/src/rubygems-1.3.0 root# gem update --system Updating RubyGems Updating rubygems-update |
Ruby Libraries
Check that the libraries are installed with the commands:
root# ruby -rbase64 -e "puts :installed" root# ruby -rcgi -e "puts :installed" root# ruby -rdigest -e "puts :installed" root# ruby -retc -e "puts :installed" root# ruby -rfileutils -e "puts :installed" root# ruby -ripaddr -e "puts :installed" root# ruby -ropenssl -e "puts :installed" root# ruby -rstrscan -e "puts :installed" root# ruby -rsyslog -e "puts :installed" root# ruby -ruri -e "puts :installed" root# ruby -rwebrick -e "puts :installed" root# ruby -rrdoc/usage -e "puts :installed" root# ruby -rxmlrpc -e "puts :installed" ruby: no such file to load -- xmlrpc (LoadError) |
With RubyGem installed, checking and installing Ruby libraries is fairly easy. For future reference, a gem is a packaged Ruby application or library. Below we are going to look for a xmlrpc packaged library globally and install it locally.
bash-3.00# gem list -b ^xmlrpc *** REMOTE GEMS *** xmlrpcs (0.1.3) bash-3.00# gem install xmlrpcs Successfully installed xmlrpcs-0.1.3 1 gem installed Installing ri documentation for xmlrpcs-0.1.3... Installing RDoc documentation for xmlrpcs-0.1.3... |
Facter
Reductive Labs describes Facter as “a cross-platform Ruby library for retrieving facts from operating systems. Supports multiple resolution mechanisms, any of which can be restricted to working only on certain operating systems or environments. Facter is especially useful for retrieving things like operating system names, IP addresses, MAC addresses, and SSH keys.” To install and confirm it is installed, issue the following commands:
root# cd /usr/local/src /usr/local/src root# wget http://www.reductivelabs.com/downloads/facter/facter-latest.tgz /usr/local/src root# tar xzf facter-latest.tgz /usr/local/src root# cd facter-1.5.2 /usr/local/src/facter-1.5.2 root# ruby install.rb /usr/local/src/facter-1.5.2 root# facter --version 1.5.2 |
There is an alternative method for installing Facter using RubyGem. Currently, it does not appear to be working. I’ll include the instructions with the hope that it will be fixed in the future.
root# gem install --remote --source http://reductivelabs.com/downloads/ facter root# facter --version 1.5.2 |
Puppet
Puppet can be installed via source, packages, or using RubyGem. I’ll demonstrate installation using source and as a gem. Since package installation is dependent on the OS, readers should view the Reductive Labs Wiki.
The source contains both the Puppet server and client code. Download and install Puppet with the following commands:
root# cd /usr/local/src /usr/local/src root# wget http://reductivelabs.com/downloads/puppet/puppet-latest.tgz /usr/local/src root# tar xzf puppet-latest.tgz /usr/local/src root# cd puppet-0.24.5 /usr/local/src/puppet-0.24.5 root# ruby install.rb /usr/local/src/puppet-0.24.5 root# puppet --version 0.24.5 |
As with Facter, using RubyGem to install Puppet does not seem to work. I’ll include the instructions and hopefully it will be fixed in the future.
root# gem install --remote --source http://reductivelabs.com/downloads/ puppet root# puppet --version 0.24.5 |
The Puppet Master
Most of the functionality of Puppet can be run with default configuration. The two things that need to be assigned is a user and a group to run along with a basic configuration to apply to the client.
User and Group
Check if the puppet user and group exist on your system.
root# id puppet uid=477(puppet) gid=149(puppet) groups=149(puppet) |
If that does not work for your OS, you can grep through the password and group files.
root# grep puppet /etc/passwd root# grep puppet /etc/group |
If you need to add the user and group puppet, under most unix version you would do so with the command:
root# groupadd puppet root# useradd -M -g puppet puppet |
Mac OS X Leopard (10.5) makes things a little more complicated if you are going to use the command line. You will first need to look at what group ids (gid) are used, choose an unused gid, and then create the puppet group using that gid. This would be accomplished with the commands:
root# dscl . list /groups PrimaryGroupID | sort -k 2,2 -n | tail -2 rt 503 httpd 10650 root# dscl . create /groups/puppet gid 504 root# dscl . create /groups/puppet passwd '*' root# dscl . read /groups/puppet root# dscl . list /users UniqueID | sort -k 2,2 -n | tail -2 ossecr 505 httpd 47008 root# dscl localhost -create /Local/Default/Users/puppet root# dscl localhost -create /Local/Default/Users/puppet RecordName puppet root# dscl localhost -create /Local/Default/Users/puppet UserShell /sbin/nologin root# dscl localhost -create /Local/Default/Users/puppet RealName "Puppet" root# dscl localhost -create /Local/Default/Users/puppet UniqueID 506 root# dscl localhost -create /Local/Default/Users/puppet PrimaryGroupID 504 root# dscl localhost -create /Local/Default/Users/puppet NFSHomeDirectory /dev/null root# dscl . read /users/puppet AppleMetaNodeLocation: /Local/Default GeneratedUID: B71B9C49-5C59-4B17-B69C-415589832976 NFSHomeDirectory: /dev/null PrimaryGroupID: 504 RealName: Puppet RecordName: puppet RecordType: dsRecTypeNative:users UniqueID: 506 UserShell: /sbin/nologin root# id puppet uid=506(puppet) gid=504(puppet) groups=504(puppet) |
Configuration File
We are going to create a default configuration file /etc/puppet/manifests/site.pp.
root# mkdir -p /etc/puppet/manifests root# vi /etc/puppet/manifests/site.pp |
For a very simple example, we will have Puppet check the user and group ownership of /etc/passwd as well as its permissions. The following would go into site.pp:
file { "/etc/passwd":
owner => "root",
group => "bin",
mode => 644,
}
|
Starting
With a simple configuration file create, we can now start the master daemon.
root# puppetmasterd --verbose --no-daemonize info: Starting server for Puppet version 0.24.5 info: Listening on port 8140 notice: Starting Puppet server version 0.24.5 |
The –verbose option will turn on verbose logging while the –no-daemonize option forces the daemon to run in the foreground. The daemon will start on TCP port 8140.
The Client
The Puppet client will run as root in order to perform the required configuration actions on the node. When you start the client for the first time, it will generate a local self-signed certificate, connect to the specified master, and request the certificate be signed. Puppet relies on SSL when communicating between the server and clients. Once the certificate is signed, the node will request whatever configuration is specified for that node. Clients running on the same host as the server have their certificates automatically signed.
To test Puppet, have the group permission of the /etc/passwd file be “root” and see how Puppet handles it on the client (example: puppet1.securitymonks.com). The client will first require the server (example: puppetmaster.securitymonks.com) to sign its certificate. The client connects to the server when the puppetd command is run. Until the certificate is signed, the group permission on /etc/passwd will not change. From the client, issue the commands:
root# chgrp root /etc/passwd root# ls -l /etc/passwd -rw-r--r-- 1 root root 2800 Sep 28 05:53 /etc/passwd root# puppetd --server puppetmaster.securitymonks.com --verbose --waitforcert 60 notice: Starting Puppet client version 0.24.5 info: No classes to store info: Caching catalog at /var/puppet/state/localconfig.yaml root# ls -l /etc/passwd -rw-r--r-- 1 root root 2800 Sep 28 05:53 /etc/passwd |
The –server option tells the client the name of the server (puppetmaster.securitymonks.com). The –waitforcert tells the client to check every 60 seconds for the server to return signed certificate. Over on the server, we see the message:
info: Caching node for puppet1.securitymonks.com notice: Compiled catalog for puppet1.securitymonks.com in 0.04 seconds |
Certificates
The Puppet master will sign certificates using the puppetca command, which controls the Puppet Certificate Authority. While automatically signing certificates is possible, in the interest of security we will be manually approving requests. Please read more about certificate management and security at Reductive Labs’ Wiki.
From the server, puppetmaster.securitymonks.com, list the waiting certificate and sign the witht he commands:
root# puppetca --list puppet1.securitymonks.com root# puppetca --sign puppet1.securitymonks.com Signed puppet1.securitymonks.com |
Go back to the client and notice that it now has a certificate. Also examine the group permission on the /etc/passwd file.
notice: Got signed certificate notice: Starting catalog run notice: //File[/etc/passwd]/group: group changed 'root' to 'bin' notice: Finished catalog run in 0.08 seconds root# ls -l /etc/passwd -rw-r--r-- 1 root bin 2800 Sep 28 05:53 /etc/passwd |
Curtain Drops
This brings to a close Act One of this Puppet implementation. A very simple Puppet master daemon with one node has been created. In Act Two, we will do a little more advance configuration. If you plan on working with Puppet, buy the book “Pulling Strings with Puppet: Configuration Management Made Easy” by James Turnbull. It is available in print or as a PDF and is the most extensive source of information in print on Puppet. The Reductive Labs Wiki is another great source for up-to-date information. Of course, the ultimate source for information on Puppet are the folks who created it. Consider hiring Reductive Labs to provide training, services, or support.
While I work on writing Act Two, in order to keep you entertained, please enjoy Luke Kanies’ Puppet presentation given at Silicon Valley’s BayLISA event on August 2006. It is a most informative and entertaining video with a run time of about 90 minutes. Enjoy. Exit stage right.
Awesome post! I wrote an intro to Puppet myself last month but yours is much more informative!!