Puppet

Introduction

Links

http://reductivelabs.com/trac/puppet/wiki/StyleGuide
http://reductivelabs.com/trac/puppet/wiki/GlossaryOfTerms
http://reductivelabs.com/trac/puppet/wiki/PuppetIntroduction
http://reductivelabs.com/trac/puppet/wiki/DocumentationStart
http://reductivelabs.com/trac/puppet/wiki/LanguageTutorial
http://reductivelabs.com/trac/puppet/wiki/ConfigurationReference
http://www.linux-mag.com/id/4141/?r=s

Puppet-daemons:

• Client: puppetd (call with -t and -d option to see what it does)
• Server: puppetmasterd

Puppet installation

• see RD-Computefarm-Checklist before starting
• Puppet-Server: puppet.klu.infineon.com (this is a alias that points to kluls231.klu.infineon.com)
• Puppet-rpms:
  • puppet-server-0.23.2-1.el4 (Server package)
  • puppet-0.23.2-1.el4 (client package)
• The client rpm automatically tries to contact the server with the name puppet.<local-domain>

Puppet configuration

Configuration directories and files

(Stored in svn,http://svn.klu.infineon.com/repos/AdminToolKit/trunk/puppet ):

• /etc/puppet:
  • facts (fact definiton, e.g. sitename, which can be used in the rules afterwards)
  • manifests (definitions and rules,...)
  • ssl
  • fileserver.conf (defining where files are served from and to whom)
• /var/lib/puppet: (place where the puppet-files are stored)
  • <site> (Site specific files, place defined in fileserver.conf)
    • <site>/push
    • <site>/static

Some configuration files with short describtion

To get a feeling for the language:

/etc/puppet/manifests/site.pp (read by the puppetmaster to find out what do to for the site)

#import custom facts
file { $factdir: source => "puppet://$servername/facts", recurse => true }

# import the functions
import "functions.pp"

# import site defenitions
import "sitedef.pp"

# import all of the server classes
import "classes/*"

# Common settings
import "common.pp"

#  vim: set syntax=puppet

/etc/puppet/fileserver.conf:

# This file consists of arbitrarily named sections/modules
# defining where files are served from and to whom

# Define a section 'files'
# Adapt the allow/deny settings to your needs. Order
# for allow/deny does not matter, allow always takes precedence
# over deny

[klu]
path /var/lib/puppet/files/klu     # Files specially for the klu site
allow *.klu.infineon.com           # only machines in KLU see them

[kia]                              # same for the other sites
...
[facts]                            # place for the fact definitions
path /etc/puppet/facts
allow *.infineon.com

/etc/puppet/facts (example fact definiton)

# returns site name to facter
# ohad.levy@infineon.com
# Solaris returns the LDAP domain instead of DNS domain
Facter.add("sitename") do
        setcode do
                %x{hostname | cut -c 1-3}.chomp
        end
end

Useful links

Autogenerate puppet pp files.

http://cft.et.redhat.com/

Puppet under solaris(blastwave)

/opt/csw/bin/puppetd --server puppet.klu.infineon.com --test --factsync true --report true --color false

Notes

1. 1 Configuration Language.

Puppet script.

1. 2 Transaction Layer.

Puppet can run multiple time resulting in the same status of handled files. 
Your machines will apply the configuration often (by default, every 30 minutes)
Puppet will only make any changes to the system if the system state does not match the configured state.

1. 3 Resource Abstraction Layer.

Puppet takes care of operating system specific command names. You should only care about configuration. Configuration entries are called resourses. 

--noop no-op, or dry-run

• providers.

makes it possible to use resouce types like file, user, and package.

• Modifying the system.

Taking a couple of lines onto the end of your fstab, you use the mount type to create a new resource that knows how to modify the fstab, or NetInfo, or wherever mount information is kept.

All resources are simple collections of attributes, some of which change how the resource functions and some of which directly manage the resource (these are called properties).

There is another special kind of attribute, those which work on all resources. These are called metaparams, and include things like the log level for the resource or --noop.

• Exec Resources

Sometimes, there will not be a resource type already developed for managing the resource you're interested in; for these cases, we provide an exec resource type, which allows you to run external commands, along with hooks to make the exec behave idempotently. For instance, you can create an exec instance that creates a Subversion repository using the svnadmin command, along with a check that causes the command to only get run if the repository does not already exist

• Classes

The next step up is to start associating resources; you would normally have at least the sudo package installed, and you'd put both the package and the configuration file into a class: Then you can just call include sudo and both resources would get applied

• Inheritance

Puppet supports a limited form of class inheritance, but it's only useful for one thing: Subclasses can override resources defined in parent classes, using a special override syntax

In the sub class above, the resource type (in this case File) is capitalized. This means that we are referring to a type that has already been declared. Using file instead would be illegal since that would result in resource overlap.

• Facter Variables

$operatingsystem variable. This variable is set, along with many others, in the top-level scope by the parser, which gets the values from Facter -- you can get a list of all available variables by just running the stand-alone facter script, but you'll also want to know about $ipaddress and $hostname, just to start.

• Selectors

Another new thing is the ? { ... } syntax, which we call a selector and is somewhat similar to the relatively common trinary operator. This tests the variable before the ?, and picks a matching value from the provided list, or the default value if nothing matches. This allows us to provide different values based on the operating system, host name, or just about anything else we want. Note that it is a parse error if no value matches. Also, selectors (as of 0.22.2) are case-insensitive in their matching.

• resource reference

The File[sshd_config] syntax is called a resource reference, and it's used to uniquely refer to a resource.

• Variables

$myvar = value Puppet will refuse to allow you to assign to the same variable twice in a given class or definition, because it will only ever use the second value and your first value will get thrown away, since all variable assignments are evaluated before any resource specifications.

1. puppet. Stand alone Puppet Site Manifest Script evaluator. Parses, evaluates, and implements a Site Manifest locally.
2. puppetmasterd. Puppet Site Manifest Server Daemon. Runs on each host serving/providing configurations to Puppet client hosts.
3. puppetd. Puppet Client Host Configuration Daemon. Runs on each host whose configuration is managed by Puppet. Requests a Host Specific Manifest from a Puppet Manifest Server and implements it locally.
4. puppetca. SSL Certificate Authority Server used for receiving cerification requests from puppet clients.

Backup location.

# Define the bucket
filebucket { main: server => puppet }
 
# Specify it as the default target
File { backup => main }

create a CNAME/alias called puppet pointing to your puppet server.

install server.

yum -y install puppet-server

install client:

yum -y install puppet ruby-rdoc

Verifying Installation on client.

puppetd --waitforcert 60 --test

Verify cert on server.

puppetca --list
puppetca --sign newmachine.halfface.se

/etc/puppet/puppet.conf

[puppetd]

   # Disable colour output
   color = false

There are two types of logging in that default node: the first, warning, is a server-side function to log the message on the server; the second, notify, looks like a resource and is sent to the client. Whether you're looking at the client or server you'll see the message.

puppetmaster keys.

ls -la /var/lib/puppet/ssl/private_keys/

puppetmaster is listening on port 8140

Verify client key. /usr/bin/openssl verify -CAfile /var/lib/puppet/ssl/certs/ca.pem /var/lib/puppet/ssl/certs/wwwold.halfface.se.pem

1. /var/lib/puppet/ssl/certs/wwwold.halfface.se.pem: OK

Verify server key. openssl verify -CAfile /var/lib/puppet/ssl/certs/ca.pem /var/lib/puppet/ssl/ca/signed/wwwold.halfface.se.pem

1. /var/lib/puppet/ssl/ca/signed/wwwold.halfface.se.pem: OK

Troubleshoot puppetmasterd. /usr/sbin/puppetmasterd --manifest=/etc/puppet/manifests/site.pp --logdest=/var/log/puppet/puppetmaster.log --debug --trace

run puppet once. /etc/init.d/puppet once

If you are just a little paranoid about security, it is much better to generate the certificates on the puppetmaster ahead of time using puppetca --generate HOST and then copy the generated files onto the client during kickstart. To be exact, you need to copy the following files from the puppetmaster to the client (all paths are relative to /var/lib/puppet/ssl on the appropriate machine):

 Puppetmaster          Client HOST
 certs/ca.pem          certs/ca.pem
 certs/HOST.pem        certs/HOST.pem
 private_keys/HOST.pem private_keys/HOST.pem
 ca/signed/HOST.pem    public_keys/HOST.pem

ls -la certs/ca.pem certs/wwwold.halfface.se.pem private_keys/wwwold.halfface.se.pem ca/signed/wwwold.halfface.se.pem ls -la certs/ca.pem certs/wwwold.halfface.se.pem private_keys/wwwold.halfface.se.pem public_keys/wwwold.halfface.se.pem

mv -f /var/lib/puppet/ssl/certs/ca.pem /var/lib/puppet/ssl/certs/ca.pem.bak cp certs/ca.pem /var/lib/puppet/ssl/certs/ca.pem mv -f /var/lib/puppet/ssl/certs/wwwold.halfface.se.pem /var/lib/puppet/ssl/certs/wwwold.halfface.se.pem.bak cp certs/wwwold.halfface.se.pem /var/lib/puppet/ssl/certs/wwwold.halfface.se.pem mv -f /var/lib/puppet/ssl/private_keys/wwwold.halfface.se.pem /var/lib/puppet/ssl/private_keys/wwwold.halfface.se.pem.bak cp private_keys/wwwold.halfface.se.pem /var/lib/puppet/ssl/private_keys/wwwold.halfface.se.pem mv -f /var/lib/puppet/ssl/public_keys/wwwold.halfface.se.pem /var/lib/puppet/ssl/public_keys/wwwold.halfface.se.pem.bak cp ca/signed/wwwold.halfface.se.pem /var/lib/puppet/ssl/public_keys/wwwold.halfface.se.pem

Remove certificate of client. puppetca --clean wwwold.halfface.se Removing /var/lib/puppet/ssl/ca/signed/wwwold.halfface.se.pem Removing /var/lib/puppet/ssl/private_keys/wwwold.halfface.se.pem Removing /var/lib/puppet/ssl/certs/wwwold.halfface.se.pem

Generate cerificate for client. puppetca --generate wwwold.halfface.se

List certificates. puppetca -l -a support

You can use puppetrun to trigger the clients to run on specific hosts and (optionally) for specific classes.

puppetrun --class sendmail --host mx0 --host mx1

Puppetshow

• Install dependencis.

yum install ruby ruby-docs ruby-irb ruby-libs ruby-mode ruby-rdoc ruby-ri rubygems ruby-mysql mysql mysql-server httpd

Setup puppetshow.

• prerequisites.

Install mysql and create a database called puppetdb. Add user puppet with password puppet as admin for that database.
Install httpd
Install ruby on rails.

• Install puppetshow.

cd /var/lib
svn co https://reductivelabs.com/svn/puppetshow puppetshow
cd /var/lib/puppetshow
chown -Rh apache:apache .

• fix ssh certificates.

ln -s /var/lib/puppet/ssl /etc/puppet/ssl

• vi setup.pp

$user = "apache"
$group = "apache"
$clientname = "ip122net"
# Link the libraries in
file { "$base/lib/puppet":
    ensure => "/usr/lib/ruby/site_ruby/1.8"
}

• Edit config file for database setup. /var/lib/puppetshow/config/database.yml

development:
  adapter: mysql
  database: puppetdb
  username: puppet
  password: puppet
  socket: /var/run/mysqld/mysqld.pid

• Run puppetshow installation.

puppet -v setup.pp

• fix links for ruby on rails.

ln -sf /usr/lib/ruby/gems/1.8/gems/rails-1.2.5 /var/lib/puppetshow/vendor/rails/railties
ln -sf /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.4 /var/lib/puppetshow/vendor/activesupport
ln -sf /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.5 /var/lib/puppetshow/vendor/actionpack
ln -sf /usr/lib/ruby/gems/1.8/gems/actionwebservice-1.2.5 /var/lib/puppetshow/vendor/actionwebservice
ln -sf /usr/lib/ruby/gems/1.8/gems/actionmailer-1.3.5 /var/lib/puppetshow/vendor/actionmailer
ln -sf /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.5 /var/lib/puppetshow/vendor/activerecord

• fix path to ruby.

ln -s /usr/bin/ruby /usr/bin/ruby1.8

• fix path. /usr/lib/ruby/site_ruby/1.8/puppet/configuration.rb

    if name != "puppetmasterd" and Puppet::Util::SUIDManager.uid != 0
       conf = "/etc/puppet"
       var = "/var/puppet"

• Configure httpd. Put the following in /etc/httpd/conf.d/puppet.conf

<VirtualHost ip122net2@halfface.se:80>
    ServerAdmin root@localhost
    SetEnv RAILS_ENV development
    ServerName ip122net2.halfface.se
    ServerAlias ip122net2
    DocumentRoot /var/lib/puppetshow/public
    ErrorLog /var/lib/puppetshow/log/apache.log

    <Directory /var/lib/puppetshow/public/>
        Options ExecCGI FollowSymLinks
        AddHandler cgi-script .cgi
        AllowOverride all
        Order allow,deny
        Allow from all
    </Directory>
</VirtualHost>

puppet.conf

[main]
   # Where Puppet stores dynamic and growing data.
   # The default value is '/var/puppet'.
   vardir = /var/lib/puppet

   # The Puppet log directory.
   # The default value is '$vardir/log'.
   logdir = /var/log/puppet

   # Where Puppet PID files are kept.
   # The default value is '$vardir/run'.
   rundir = /var/run/puppet

   # Where SSL certificates are kept.
   # The default value is '$confdir/ssl'.
   ssldir = $vardir/ssl

   # Where Puppet looks for template files.
   templatedir=/etc/puppet/templates

   # An external command that can produce node information.
   external_nodes = /etc/puppet/node.sh

   # Where Puppet should look for facts. Multiple directories should be colon-separated, like normal PATH variables.
   # Section: main
   # Default: $vardir/facts
   factpath = $confdir/facts

   # Whether facts should be synced with the central server.
   # Section: main
   # Default: false
   factsync = true

[puppetmasterd]
   # The list of reports to generate. All reports are looked for in puppet/reports/<name>.rb, and multiple report names should be comma-separated (whitespace is okay).
   # Section: reporting
   # Default: store
   reports = store,rrdgraph,tagmail

   # Whether RRD information should be graphed.
   # Section: metrics
   # Default: false
   rrdgraph = true

   # The directory where RRD database files are stored. Directories for each reporting host will be created under this directory.
   # Section: metrics
   # Default: $vardir/rrd
   rrddir = $vardir/rrd

   # smtpserver
   # The server through which to send email reports.
   # Section: tagmail
   # Default: none
   smtpserver = www.halfface.se

   # Whether to create dot graph files for the different configuration graphs. These dot files can be interpreted by tools like OmniGraffle or dot (which is part of ImageMagick).
   # Section: graphing
   # Default: false
   graph = true

   # Whether to validate types during parsing.
   # Section: parser
   # Default: true
   typecheck = true

   # Whether to validate parameters during parsing.
   # Section: parser
   # Default: true
   paramcheck = true

   # Whether to store each client's configuration. This requires ActiveRecord from Ruby on Rails.
   # Section: puppetmasterd
   # Default: false
   storeconfigs = true

   # The type of database to use.
   # Section: rails
   # Default: sqlite3
   dbadapter = mysql

   # The database server for Client caching. Only used when networked databases are used.
   # Section: rails
   # Default: localhost
   dbserver = localhost

   # The name of the database to use.
   # Section: rails
   # Default: puppet
   dbname = puppetdb

   # The database user for Client caching. Only used when networked databases are used.
   # Section: rails
   # Default: puppet
   dbuser = puppet

   # The database password for Client caching. Only used when networked databases are used.
   # Section: rails
   # Default: puppet
   dbpassword = puppet

[puppetd]
   # The file in which puppetd stores a list of the classes
   # associated with the retrieved configuratiion.  Can be loaded in
   # the separate ``puppet`` executable using the ``--loadclasses``
   # option.
   # The default value is '$confdir/classes.txt'.
   classfile = $vardir/classes.txt

   # Where puppetd caches the local configuration.  An
   # extension indicating the cache format is added automatically.
   # The default value is '$confdir/localconfig'.
   localconfig = $vardir/localconfig

   # Whether to send reports after every transaction.
   # Section: puppetd
   # Default: false
   report = true

Custom Facts

How to create custom fact.

# hardware_platform.rb

Facter.add("hardware_platform") do
        setcode do
                %x{/bin/uname -i}.chomp
        end
end

Verify custom fact.

host:$ mkdir -p ~/lib/ruby/facter ; export RUBYLIB=~/lib/ruby
host:$ cp /path/to/hardware_platform.rb $RUBYLIB/facter
host:$ facter hardware_platform