Puppet - The Linux and Windows Automation Tool.

Preamble:

Puppet is an open source configuration management utility. It runs on many Unix-like systems as well as on Microsoft Windows, and includes its own declarative language to describe system configuration. It is mainly written in Ruby and released under the Apache License. It is also important to note that there is Puppet Community Edition and Puppet Enterprise Edition. The Enterprise and the Community Edition have equally the same features except for the fact that the Enterprise Edition comes with personalized support and maintenance to help Enterprises successfully deploy, develop and maintain their IT infrastructure at scale.
In this article we will deploy a production type Puppet environment where we will set up the Puppet Master, and two Puppet Agents respectively running CentOS 6.5 and Ubuntu 14 LTS. We will be using the Puppet Community Edition, which offers ways to switch to the Enterprise Edition if we decide to subscribe to the personalized support from Puppet Labs. See Pricing.

alt text

Pre-requesites:

For a production ready Puppet System we will use the following technologies:

  • Software Technologies:
    • Puppet Server: Which will run of the Master Node where all the configurations will be set.
    • Apache Web Server: Which will be installed on the Master Node
    • Phusion Passenger: Which will keep the Puppet Server running continuously. It will only be installed on the Master Node.
    • Puppet Agent: Which will run of the Agent Node.
  • Hardware Technologies:
    • The Master Node
      • FQDN: ppmaster.linuxforafrica.org
      • IP Address: 45.55.192.108
      • OS: CentOS 6.5
    • The Agent Node 1
      • FQDN: ppagent1.linuxforafrica.org
      • IP Address: 45.55.192.109
      • OS: CentOS 6.5
    • The Agent Node 2
      • FQDN: ppagent2.linuxforafrica.org
      • IP Address: 45.55.192.110
      • OS: Ubuntu 14. LTS

Note: Please note that one Puppet Master Node can manage one or multiple Puppet Agent Nodes. It this article we will show an example where we will automate the management of 02 Agent Nodes or Servers Agents.

The Big Picture:

alt text

Preparing the Master Server

The Master Node or Puppet Master Server (ppmaster.linuxforafrica.org) will host the Puppet Server, the Apache Web Server, and Phusion Passenger. It will mainly be the central Server where the Agent Servers will pull their configurations from.
So on a freshly minimal installed linux Server (CentOS 6.5) do the following:

  • Create a user with sudo power:
    The following set of commands will create a new user, assign it to the Wheel Group, and give the Wheel Group sudo rights. As root run:
1
adduser mynewuser
1
passwd mynewuser

On the above command you will be prompted to set a strong password.

1
usermod -G wheel mynewuser

The above command assigns mynewuser to the Wheel Group

1
visudo

Then on the above command, uncomment the following line # %wheel ALL=(ALL) ALL by removing the #.

  • Set the hostname: Next we will set the hostname by running the command below:
1
echo 45.55.192.108 ppmaster.linuxforafrica.org >> /etc/hosts
  • Setting SELinux to Permissive Mode: Security-Enhanced Linux is a Linux kernel security module that provides a mechanism for supporting access control security policies. It is used by default by CentOS 6.5. To set it to permissive mode run the following:
1
su mynewuser

The above command will swicth our current user from root to mynewuser.

1
sudo vim /etc/sysconfig/selinux

Then set SELINUX=permissive

Next update and restart the server:

1
sudo yum update
1
sudo shutdown -r now

Installing and Configuring The Puppet Master Server

The following configuration will be done on the Master Node ppmaster.linuxforafrica.org

  • First, install the Puppet Repository: This can be done by running the following commands.
1
sudo mynewuser
1
sudo yum install http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
  • Second, install the Puppet Server: This can be done by running the following commands.
1
sudo mynewuser
1
sudo yum install puppet-server
1
sudo puppet master --version

(The version at the time of writing was 3.7.5)

  • Third, create the Puppet Master Directory Environment: This can be done by running the following commands.
1
sudo mynewuser
1
sudo mkdir -p /etc/puppet/environments/production/{modules,manifests}
  • Next, configure Puppet: This can be done by running the following commands.
1
sudo mynewuser
1
sudo vim /etc/puppet/environments/production/environment.conf

And paste in the following code snippet:

modulepath = /etc/puppet/environments/production/modules
environment_timeout = 5s 

Save and exit, then run the following commands to edit the puppet.conf file.

1
sudo mynewuser
1
sudo cp /etc/puppet/puppet.conf /etc/puppet/puppet.conf.backup
1
sudo vim /etc/puppet/puppet.conf

Then just after the [main] section append the following code snippet:

[master]
    environmentpath = $confdir/environments
    basemodulepath = $confdir/modules:/opt/puppet/share/modules 

Save and exit. Next we will edit again the puppet.conf file.

1
sudo vim /etc/puppet/puppet.conf

Then in the [main] section, at the very bottom, append the following code snippet:

[master]
    dns_alt_names = ppmaster.linuxforafrica.org,ppmaster 

Note: the ppmaster.linuxforafrica.org must be a resolvable Domain Name. Do not forget to change it to your own settings.

  • Finally, Generate the Puppet Server Certificates: This can be done by running the following commands.
1
sudo mynewuser
1
sudo puppet master --verbose --no-daemonize

Then when you see the following output Notice: Starting Puppet master version 3.7.5.
Do Ctl+C to come back to the prompt.

Note: The ssl keys and certificates are store in /var/lib/puppet/ssl
To regenerate new keys and certificates for the Puppet Master Server, just delete the /var/lib/puppet/ssl directory by running sudo rm -rf /var/lib/puppet/ssl. Then rerun sudo puppet master --verbose --no-daemonize

  • Firewall configuration: Please open ports 8140 by running the following commands.
1
sudo mynewuser
1
sudo iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 8140 -j ACCEPT
1
sudo service iptables restart

Installing Apache and Phusion Passenger

The following configuration will be done on the Master Node ppmaster.linuxforafrica.org

  • First, install the needed packages: This can be done by running the following commands.
1
sudo mynewuser
1
sudo yum install httpd httpd-devel mod_ssl ruby-devel rubygems gcc gcc-c++ libcurl-devel openssl-devel

Next check the version of Apache, Ruby, and Gems Installed:

1
sudo httpd -v
1
sudo ruby -v
1
sudo gem -v
  • Next, we can install Phusion Passenger: This can be done by running the following commands.
1
sudo gem install rack passenger
1
sudo passenger-install-apache2-module

Note: After the installation of Phusion Passenger copy the output that shows how to load the passenger module into a notepad. We will be using it in the virtualhost configuration file for the Puppet Master.

  • In the next step, we will configure Phusion Passenger: This can be done by running the following commands.
1
su mynewuser
1
sudo mkdir -p /usr/share/puppet/rack/puppetmasterd/{public,tmp}
1
sudo cp /usr/share/puppet/ext/rack/config.ru /usr/share/puppet/rack/puppetmasterd/
1
sudo chown puppet:puppet /usr/share/puppet/rack/puppetmasterd/config.ru
  • Ensuring the Keys and Certificates generated by the Puppet Server are FQDN: This can be done by running the following commands.
1
su -
1
cd /var/lib/puppet/ssl/certs

Then list the content of the directory to make sure that the key generated is in the form of ppmaster.linuxforafrica.org.pem

1
cd /var/lib/puppet/ssl/private_keys

Then list the content of the directory to make sure that the certificate generated is in the form of ppmaster.linuxforafrica.org.pem

1
cd /var/lib/puppet/ssl/private_keys

Then list the content of the directory to make sure that the key generated is in the form of ppmaster.linuxforafrica.org.pem

  • If the Keys and Certificates are not FQDN then run the following:
1
su mynewuser
1
sudo mv /var/lib/puppet/ssl/certs/ppmaster.pem /var/lib/puppet/ssl/certs/ppmaster.linuxforafrica.org.pem
1
sudo mv /var/lib/puppet/ssl/private_keys/ppmaster.pem /var/lib/puppet/ssl/private_keys/ppmaster.linuxforafrica.org.pem
  • Next Configure Apache:
1
su mynewuser
1
sudo vim /etc/httpd/conf.d/00-puppetmaster.conf

And paste in the following code snippet below. (Check to ensure it reflects your settings).

LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-5.0.7/buildout/apache2/mod_passenger.so
PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-5.0.7
PassengerDefaultRuby /usr/bin/ruby

PassengerHighPerformance on
PassengerMaxPoolSize 12
PassengerPoolIdleTime 1500
PassengerMaxRequests 1000
PassengerStatThrottleRate 120

Listen 8140

        
        SSLEngine on
        SSLProtocol             ALL -SSLv2 -SSLv3
        SSLCipherSuite          EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!IDEA:!ECDSA:kEDH:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
        SSLHonorCipherOrder     on

        SSLCertificateFile      /var/lib/puppet/ssl/certs/ppmaster.linuxforafrica.org.pem
        SSLCertificateKeyFile   /var/lib/puppet/ssl/private_keys/ppmaster.linuxforafrica.org.pem
        SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
        SSLCACertificateFile    /var/lib/puppet/ssl/ca/ca_crt.pem
        SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
        SSLVerifyClient optional
        SSLVerifyDepth  1
        SSLOptions +StdEnvVars +ExportCertData

        RequestHeader unset X-Forwarded-For

        RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
        RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
        RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

        DocumentRoot /usr/share/puppet/rack/puppetmasterd/public/
        RackBaseURI /
        
                Options None
                AllowOverride None
                Order allow,deny
                allow from all
        
        PassengerAppRoot /usr/share/puppet/rack/puppetmasterd

Save and exit, then restart the Apache web server.

1
sudo service httpd start
1
sudo chkconfig httpd on

Now we are almost done with setting up the Puppet Master Node. Passenger and Apache will be responsible for keeping the Puppet Server alive. To complete the setting we will need to install the NTP Server. It is very important to have an accurate and well configured NTP Time Server.

1
sudo yum install ntp
1
sudo chkconfig ntpd on
1
sudo /etc/init.d/ntpd start

Installing and Configuring Puppet on the CentOS 6.5 Agent Server

The Agent Node or Puppet Agent Server will host the Puppet Agent, and will communicate with the Puppet master in order to automatically retrieve its configuration file.
So on a freshly minimal installed linux (CentOS 6.5) do the following:

  • Set the hostname: Next we will set the hostname by running the command below:
echo 45.55.192.108 ppmaster.linuxforafrica.org >> /etc/hosts
echo 45.55.192.109 ppagent1.linuxforafrica.org ppagent1 >> /etc/hosts
  • Install the Puppet Repository: This can be done by running the following commands.
1
su mynewuser
1
sudo yum install http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
  • Install the Puppet Agent: This can be done by running the following commands.
1
sudo yum install puppet

If the Puppet Agent is well installed we can check the version by running the following:

1
sudo puppet master --version

(The version at the time of writing was 3.7.5)

  • Configure the Puppet Agent: This can be done by running the following commands.
1
sudo cp /etc/puppet/puppet.conf /etc/puppet/puppet.conf.backup
1
sudo vim /etc/puppet/puppet.conf

And just in the [agent] section, at the very bottom, append the following code snippet:

server = ppmaster.linuxforafrica.org
  • Initializing and Generating the Puppet Agent Server Certificates: This can be done by running the following commands.
1
sudo puppet agent --verbose --no-daemonize --onetime

Then when you see the following output Notice: Starting Puppet master version 3.7.5.
Do Ctl+C to come back to the prompt.

Now go back to the Puppet Master Server (ppmaster.linuxforafrica.org) and Sign the Certificate requested by the agent:

sudo puppet cert list
sudo puppet cert sign ppagent1.linuxforafrica.org

Next, lets go back to the Puppet Node Server and check:

1
sudo puppet agent --verbose --no-daemonize --onetime

We should now see that the Puppet Agent (ppagent1.linuxforafrica.org) is communicating with the Puppet Master Server (ppmaster.linuxforafrica.org).

Now we are almost done with setting up the Puppet Agent. To complete the settings we will need to install the NTP Server.

1
sudo yum install ntp
1
sudo chkconfig ntpd on
1
sudo /etc/init.d/ntpd start

Installing and Configuring Puppet on the Ubuntu 14 LTS Node Server

The Agent Node or Puppet Agent Server will host the Puppet Agent, and will communicate with the Puppet master in order to automatically retrieve its configuration file.
The following commands will be done on ppagent2.linuxforafrica.org.
So on a freshly minimal Ubuntu system do the following:

  • Set the hostname: Next we will set the hostname by running the command below:
echo 45.55.192.108 ppmaster.linuxforafrica.org >> /etc/hosts
echo 45.55.192.110 ppagent2.linuxforafrica.org ppagent2 >> /etc/hosts
  • Install the Puppet Repository: This can be done by running the following commands.
1
cd /tmp
1
sudo wget http://apt.puppetlabs.com/puppetlabs-release-trusty.deb
1
sudo dpkg -i puppetlabs-release-trusty.deb
1
sudo rm puppetlabs-release-trusty.deb

Update the system and reboot the server

1
sudo apt-get update
1
sudo shutdown -r now
  • Install the Puppet Agent: This can be done by running the following commands.
1
sudo apt-get install puppet

If the Puppet Agent is well installed we can check the version by running the following:

1
sudo puppet master --version

(The version at the time of writing was 3.7.5)

  • Configure the Puppet Agent: This can be done by running the following commands.
1
sudo cp /etc/puppet/puppet.conf /etc/puppet/puppet.conf.backup
1
sudo vim /etc/puppet/puppet.conf

And just in the [agent] section, at the very bottom, append the following code snippet:

server = ppmaster.linuxforafrica.org
  • Initializing and Generating the Puppet Agent Server Certificates: This can be done by running the following commands.
1
sudo puppet agent --verbose --no-daemonize --onetime

Then when you see the following output Notice: Starting Puppet master version 3.7.5.
Do Ctl+C to come back to the prompt.

Now go back to the Puppet Master Server (ppmaster.linuxforafrica.org) and Sign the Certificate requested by the agent:

sudo puppet cert list
sudo puppet cert sign ppagent2.linuxforafrica.org

Then, go back to the Puppet Node Server and check:

1
sudo puppet agent --verbose --no-daemonize --onetime

We should now see that the Puppet Agent (ppagent2.linuxforafrica.org) is communicating with the Puppet Master Server (ppmaster.linuxforafrica.org).

Now we are almost done with setting up the Puppet Agent. To complete the settings we will need to install the NTP Server.

1
sudo yum install ntp
1
sudo chkconfig ntpd on
1
sudo /etc/init.d/ntpd start

Conclusion

We are now done with deploying a production type Puppet environment. In the setting we deployed the Puppet Master (ppmaster.linuxforafrica.org), and 02 Puppets Agents (ppagent1.linuxforafrica.org, ppagent2.linuxforafrica.org). In a later article we will learn how to create Puppet Programs which are manifest files detailing how Agent Servers must be configured.

Comments