home › Adviсe › Master of puppets: Installing and configuring the Puppet remote configuration management system. Master of puppets: Installing and configuring the Puppet remote configuration management system Installing puppet

Master of puppets: Installing and configuring the Puppet remote configuration management system. Master of puppets: Installing and configuring the Puppet remote configuration management system Installing puppet

To use Puppet more effectively, you need to understand how modules and manifests are built. This tutorial will walk you through how these Puppet components work by setting up a LAMP stack on an Ubuntu 14.04 server.

Requirements

Installing Puppet (master and agent). More about this -.
The ability to create at least one Ubuntu 14.04 virtual server to serve the Puppet agent node.

Puppet Code Basics

Resources

Puppet code mainly consists of resources. A resource is a piece of code that describes the state of the system and determines the changes it needs. For example:

user("mitchell":
ensure => present,
uid => "1000",
gid => "1000",
shell => "/bin/bash",
home => "/home/mitchell"
}

The resource declaration has the following format:

resource_type("resource_name"
attribute => value
...
}

To view all Puppet resource types, issue the command:

puppet resource --types

You will learn more about resource types in this guide.

Manifestos

A manifest is an orchestration script. Puppet programs with a .pp extension are called manifests. The default Puppet manifest is /etc/puppet/manifests/site.pp.

Classes

As in any regular programming language, classes are responsible for organizing and reusing parts of the orchestration.

Within a class definition is a block of code that describes how the class works. Once you define a class, you can use it in manifests.

The class definition has the following format:

class example_class(
...
code
...
}

This code defines the example_class class. The Puppet code will be in curly braces.

A class declaration is the place in the code where a particular class is called. With a class declaration, Puppet processes its code.

The class declaration can be ordinary and by resource type.

A regular class declaration is added to the code using the include keyword.

include example_class

When declared as a resource type, the class is declared in resource format:

class("example_class":)

This declaration allows you to add class parameters to your code that override the default values of class attributes. For example:

node "host2" (
class ("apache": ) # use apache module
apache::vhost ( "example.com": # define vhost resource
port => "80",
docroot => "/var/www/html"
}
}

Modules

A module is a group of manifests and other files organized in a predefined way that makes it easy to share and reuse individual parts of the orchestration. Modules help organize Puppet code because they can be used to separate code into multiple manifests.

Puppet modules are stored in the /etc/puppet/modules directory.

Writing a manifesto

You can practice writing Puppet manifests, modules, and classes using the example of installing a LAMP stack on an Ubuntu server (the result will be ).

So, to orchestrate an Ubuntu 14.04 server and install a LAMP stack on it, you need resources for the following actions:

installing the apache2 package.
starting the apache2 service.
package installation MySQL server, mysql-server.
starting the mysql service.
installing php5 package
creating a PHP test script, info.php.
updating the apt index before installing each package.

Below you will find three examples of Puppet code that can be used to achieve such a LAMP stack setup.

The first example will teach you how to write basic manifests in one file. The second example will help you assemble and use a class and module based on previously written manifests. The third example will show you how to use pre-built public modules to install a LAMP stack.

Note: For testing it is better to use a fresh virtual server.

Example 1: Installing LAMP with one manifest

The Puppet manifest can be written on the agent node and then executed using the puppet apply command (you don't need to have a master and agent setup to do this).

In this section, you will learn to write manifests that will use these types of resource declarations:

exec: Execute commands.
package: install packages.
service: service management.
file: file management.

Creating a manifest

Create a new manifest:

sudo vi /etc/puppet/manifests/lamp.pp

Add the following code to it to declare the required resources.

# run "apt-get update" command
exec("apt-update": # resource exec "apt-update"
command => "/usr/bin/apt-get update" # command that this resource will run
}
# install apache2 package
package("apache2":
require => Exec["apt-update"], # request "apt-update" before installing the package
ensure => installed,
}
# start apache2 service
service("apache2":
ensure => running,
}
# install mysql-server
package("mysql-server":
require => Exec["apt-update"], # request "apt-update" by reinstalling
ensure => installed,
}
# start the mysql service
service("mysql":
ensure => running,
}
# install php5 package
package("php5":
require => Exec["apt-update"], # request "apt-update" before installation
ensure => installed,
}
# start the info.php service
file("/var/www/html/info.php":
ensure => file,
content => "", # phpinfo code
require => Package["apache2"], # request for package "apache2"
}

Applying a manifest

To use the new manifest, enter the command:

sudo puppet apply --test

It will display a voluminous result that displays all changes in the state of the environment. If there are no errors in the output, you should be able to open your external IP address or domain name in your browser. A test test will appear on the screen. PHP page with information about the stack. This means that Apache and PHP are working.

Now the LAMP stack is installed on the server using Puppet.

This is a fairly simple manifest since it can be executed on an agent. If you don't have a Puppet master, other agent nodes won't be able to use this manifest.

The Puppet master server checks for server state changes every 30 minutes.

Example 2: Installing a LAMP stack using a module

Now try creating a simple module based on the LAMP manifest you wrote in the previous section.

To create a module, create a new directory in the modules directory (its name must match the name of the module). This directory should contain a manifests directory and an init.pp file. The init.pp file specifies the Puppet class (its name must also match the name of the module).

Creating a Module

Go to the Puppet master server and create a directory structure for the module:

cd /etc/puppet/modules
sudo mkdir -p lamp/manifests

Create and open the init.pp file in the editor:

sudo vi lamp/manifests/init.pp

Insert the lamp class into the file:

class lamp(
}

Copy the contents of the manifest from section 1 and paste it into the lamp class block. You now have a lamp class definition. Other manifests will be able to use this class as a module.

Save and close the file.

Using a module in the main manifest

Now you can configure the main manifest and use the lamp module to install the LAMP stack on the server.

On the Puppet master server, edit the following file:

sudo vi /etc/puppet/manifests/site.pp

Most likely on this moment the file is empty. Add the following lines to it:

node default ( )
node "lamp-1" (
}

Note: Replace lamp-1 with the hostname of your Puppet agent on which to install the stack.

The node block allows you to specify Puppet code that will only apply to some nodes.

The default block applies to all agent nodes that do not have an individual block (leave it empty). The lamp-1 block will be applied to the lamp-1 agent node.

Add the following line to this block, which uses the lamp module:

Save and close the file.

Now the Puppet agent node will be able to download settings from the master server and install the LAMP stack. If you want to make changes right now, run the command on the agent:

sudo puppet agent --test

Modules are the most convenient way to reuse Puppet code. In addition, modules help you logically organize your code.

Example 3: Installing LAMP using public modules

The MySQL module is used in a similar way. Add the following lines to the node block:

class("mysql::server":
root_password => "password",
}

You can also pass MySQL module parameters.

Add a resource that will copy info.php to the desired location. Use the source parameter. Add the following lines to the node block:

file("info.php": # resource file name
path => "/var/www/html/info.php", # target path
ensure => file,
require => Class["apache"], # apache class to use
source => "puppet:///modules/apache/info.php", # location to copy the file to
}

This class declaration uses the source parameter instead of content. This option not only uses the contents of the file, but also copies it.

Puppet will copy the file puppet:///modules/apache/info.php to /etc/puppet/modules/apache/files/info.php.

Save and close the file.

Create an info.php file.

sudo sh -c "echo""> /etc/puppet/modules/apache/files/info.php"

Now the Puppet agent node will be able to download settings from the master server and install the LAMP stack. If you want to make changes to the agent environment right now, run the command on this node:

sudo puppet agent --test

This command will download all updates for the current node and install the stack on it. To make sure Apache and PHP are working, open the node's IP address or domain in a browser:

http://lamp_1_public_IP/info.php

Conclusion

You now have basic knowledge of working with Puppet modules and manifests. Try creating a simple manifest and module yourself.

Puppet is great for managing application configuration files.

Tags: ,
A bit of poetry. It would seem that this article should be the starting point for the entire series, but the target audience is still more experienced users of Open Source Puppet Labs products who are not satisfied with the individual, poorly integrated modules with Puppet Forge. As with any “library vs. framework” case, the price to pay is to follow the worldview of the author of the integrated solution.

A little about how Puppet works

Puppet is primarily a specific language for declaratively specifying the final state of the system. For comparison, the GNU Makefile is extremely suitable, where, in addition to the direct description of dependencies, it is possible to get weird to the fullest.

Puppet's abstraction is something like this ( breaking patterns - forget everything you knew about programming terms!).

Node is a set of configurations for a specific target system. In fact, this is a special case of a class.
Class is a set of declarative logic that is included in the node configuration or other classes. The class has neither instances nor methods, but it has parameters and variables defined within the logic. In fact, it is rather a procedure that can inherit another procedure by simply adding code and having a not-so-banal scope of variables.
Type- but this looks more like a classic class - it assumes instances with a name and definitely given parameters, but nothing more. A specific implementation of a type can be written as a Puppet script via define , which creates instances of other types, or as a Ruby extension with a flight of fancy.
Resource- these are actually named instances of Types. Each resource name is unique within a specific type within the node (directory) configuration.
Variables- well, in short, these are constants... Before Puppet 4, things were even worse with their scope. Now it is adequate: for use from outside the definition location, a fully qualified identifier must be specified, except in the case of class inheritance.

Puppet can be used for local deployment without a network or associated infrastructure. This can be used to create container images. There is even a whole movement advocating the abandonment of a centralized server.

In an ideologically correct manner, the Puppet infrastructure consists of an agent - a privileged service on the target system - and a server that distributes valuable instructions in the form of declarative resource directories upon request from agents. Security is implemented at the level of the private public key infrastructure (X.509). Simply put, the same mechanisms as in HTTPS, but with its own CA and mandatory verification client certificate.

In a simplified form, the deployment procedure looks something like this:

Processing via TLS and X.509 (connection establishment, CRL updating, checking certificate restrictions, etc.)
The agent receives fact generators from the server with caching and all the things (more precisely, everything is pulled from the lib/ folders in the modules). It is not difficult to add your own Ruby script to collect the information of interest.
The agent collects facts about the target system and sends it to the server. All facts can be easily viewed manually via the puppet facts call. These facts are available as global variables.
The server compiles a catalog of resources and sends it to the agent. Underneath this lies a whole layer of different concepts.
The agent pulls everything necessary from the server and brings the system to the specified form. The agent itself does not know what to do with resources; it relies on the implementation of providers (the semantic translation will be “implementer”, not supplier) of specific types of resources. Some providers are standard and are included in Puppet packages, while the rest are pulled from modules.

To enjoy all the delights, there are additional buns in the form of:

Module- a collection of declarative Puppet scripts, Ruby extensions for Puppet, files, file templates, Hiera data and much more. A more correct term would be "package".
Environment- a set of scripts, modules and Hiera data. As the infrastructure became more complex, it inevitably became necessary to divide the configuration further than the standard division by nodes. Basically, this is required for pilot innovations and banal access control (when not all administrators have access to all nodes of the IT infrastructure).
Hiera- hierarchical database. This formulation can be very intimidating. This is probably why it was changed in the documentation of later versions. In fact, this is an extremely simple and convenient mechanism for extracting configuration from YAML or JSON files. Hierarchy is the ability to specify the reading order of multiple configuration files - i.e. hierarchy/priority of these files.
- In addition to fetching data by function call, Puppet pulls default class parameters, which is the main highlight.
- Of course, Hiera supports fact interpolation and even calling special functions.
- In Puppet 4.3, we implemented the same functionality again to support not only the global database, but also the local one for the Environment and Module, although the author has already found several problems in their implementation (PUP-5983, PUP-5952 and PUP-5899), which were instantly fixed by Puppet Labs.
- Several strategies are supported for extracting values from all files in the hierarchy:
  - first - the first value found by priority is returned
  - unique - collects all values into a one-dimensional array and removes duplicates
  - hash - combines all found YAML Hash. Duplicate keys are selected by priority.
  - deep is essentially a recursive version of hash
- The beauty is that the sampling strategy can be specified both when calling the lookup() function, because and in any hierarchy file through the special lookup_options key, which is actively used in the cfnetwork module.
PuppetDB- essentially a layer of business logic around a relational database (PostgreSQL), which allows you to save reports on facts and deployments done and export resources for subsequent import into directories on other nodes or selections via special functions. There is also a web interface in the form of Puppet Dashboard.
X.509 PKI- the already mentioned certificate infrastructure, which is extremely convenient to use for other services without the need to manage a separate infrastructure.
MCollective- seems to be a useful thing for event-based launching of tasks on a server farm, but the author has a certain mistrust in the security of a particular solution.
Puppet Forge- an open platform for publishing and downloading Modules.
some other features in the form of controls external devices type of Cisco equipment and deployment on bare metal, but that's a different story

Notes on security and accessibility

You need to understand that Puppet Server becomes a vulnerable point of the entire IT infrastructure, because... determines the final configuration of all systems. In special cases, it makes sense to do a separation - a separate server for critical infrastructure elements with extremely limited access and manual update and the second for everything else.

The availability of Puppet Server determines the ability to manage the entire infrastructure. It makes sense to host Puppet Server on a virtual machine in a more reliable and quickly recoverable third-party cloud than your own capabilities. Or you should install several servers.

In any case, you should not install other services on the system where Puppet Server with bells and whistles will be deployed. Virtualization and containerization can help you.

Multi-master (several separate Puppet Servers)

In this case, only one server acts as a CA (Certificate Authority) - its unavailability means it is impossible to add new nodes.
- Puppet allows you to use a third-party X.509 infrastructure if the built-in one is not satisfactory.
The entire configuration (Environment) must be stored in a version control system and deployed to each server simultaneously.
The only thing in common is the PostgreSQL database, the organization of high availability of which is beyond the scope of this article.
The cfpuppetserver module supports installations as a primary (with CA) and as a secondary server.

What significant has changed since older versions

The manufacturer has a full description.

All services have moved to JVM, JRuby and Jetty. Despite the obvious advantages of integration, there are also disadvantages in terms of memory consumption.
Lambda functions have been added for processing collections - now there is no need to cut crutches in Ruby or pervert via create_resources()
A tool for processing EPP templates has appeared - essentially the same ERB, but with Puppet DSL instead of Ruby,
The default directory structure of configuration files has changed significantly
Added support for Data Providers for Environments and Modules (hacks are no longer required).
Downplaying the role of the global Hiera. New and related command is puppet lookup.

Installation

This process is quite primitive, but requires following a certain sequence of steps. Since doing this manually is a thankless task, the author will teach you something bad, namely downloading incomprehensible scripts from the Internet and running them as root on your system.

The three main server components are Puppet Server itself, PuppetDB and PostgreSQL. They can all be crammed into one node or divided into two or three systems. Puppet Server and Puppet DB can be run multiple times, but PostgeSQL is a single point of failure. There are various approaches to PostgeSQL replication and clustering. A convenient approach in the case of the main and secondary servers would be Master + Read-Only Slave, which is supported in PuppetDB itself as the main and read-only database node, but automating such a setup takes time and therefore is not yet available included in the cfpuppetserver module.

The configuration itself can simply be stored at least on file system along with Puppet Server, but it's like writing scripts on a production web server. The most suitable solution is a git repository. The r10k utility can pull all branches of the repository and deploy them to Puppet Server as separate Environments. r10k is pretty bad at pulling dependencies, so librarian-puppet is used on top. It's worth noting right away that the main canonical Puppet Environment is "production". Therefore, the configuration repository should use a branch called "production" rather than "master".

System requirements

The hardware is described by the manufacturer himself. The cfpuppetserver module currently only supports Debian Jessie+ and Ubuntu Trusty+.

Configuration in Git

For r10k itself, the location of the repository does not matter much - the main thing is its availability. For example, for testing purposes, the repository could be hosted on the same system and accessed via file:// . A good place to start is the codingfuture/puppet-exampleenv configuration example.

Cloning the repository: git clone https://github.com/codingfuture/puppet-exampleenv my-puppet-conf && cd my-puppet-conf
Install General settings admin access using hints in the comments:
- $EDITOR data/common.yaml
Let's create a node configuration:
- $MY_DOMAIN - root domain name (for example, example.org)
- $HOST_NAME - client host name without domain
- mkdir data/$MY_DOMAIN
- cp data/example.com/puppet.yaml data/$(MY_DOMAIN)/puppet.yaml
- $EDITOR nano -w data/$(MY_DOMAIN)/puppet.yaml - setting up a node with Puppet Server according to suggestions in the comments
- cp data/example.com/host.yaml data/$(MY_DOMAIN)/$(HOST_NAME).yaml
- $EDITOR nano -w data/$(MY_DOMAIN)/$(HOST_NAME).yaml - setting up an arbitrary node based on suggestions in the comments
We push to our own Git server or make it available locally on a node with Puppet Server via rsync or scp. A local repository is convenient as an intermediate step until the Git server is deployed from Puppet itself. In a sense, this is reminiscent of assembling a compiler in several stages.

Installing from scratch on a clean system

The cfpuppetserver module allows you to install everything using Puppet itself, but for the initial installation, the basic operations are duplicated by a Bash script.

On the target system:

Download the installation script: wget https://raw.githubusercontent.com/codingfuture/puppet-cfpuppetserver/master/setup_puppetserver.sh
We look through the script and frown: less setup_puppetserver.sh
Run: bash setup_puppetserver.sh
puppet.$(MY_DOMAIN) .
- Example with a remote repository: bash setup_puppetserver.sh ssh:// [email protected]/puppet-conf
- Example with local: bash setup_puppetserver.sh file:///root/puppetconf/
We see how the system puffs up and doesn’t install everything very quickly.
If the repository is remote:
- Create an SSH key for root: ssh-keygen -t rsa -b 2048
- We register the public key /root/.ssh/id_rsa.pub on the remote Git server...
- ... and there we set up a Git hook by calling the following command: /usr/bin/ssh -T deploypuppet@puppet.$(MY_DOMAIN) ./puppetdeploy.sh
We start deploying the configuration manually: /etc/puppetlabs/deploy.sh
Let's try how it works on the server itself: /opt/puppetlabs/bin/puppet agent --test
Check your network settings, surge protector and SSH access

Adding managed nodes

The Puppet Server's fully qualified name must be available via DNS on the managed host or hardcoded into /etc/hosts.
- Example: echo "128.1.1.1 puppet.example.com" >> /etc/hosts
On the node with Puppet Server, run the following script /root/genclientinit.sh $(HOST_NAME).$(MY_DOMAIN) .
Copy the entire result and paste it into command line on the target system.
We wait for the end of execution and run /opt/puppetlabs/bin/puppet agent --test . At the first launch, a certificate signing request will be generated.
We go to the Puppet Server node to sign the certificate.
- puppet cert list - we check the certificate signature for extra paranoia.
- puppet cert sign $(HOST_NAME).$(MY_DOMAIN) - actually, we sign the certificate.
We return to the managed node and run: /opt/puppetlabs/bin/puppet agent --test` again. This will force the deployment procedure to start.
We are waiting for the deployment to complete via Puppet Agent.
That's it, you have a minimal Puppet infrastructure ready!

Example output from /root/genclientinit.sh

bash</etc/cflocation fi if test ! -z ""; then echo -n >/etc/cflocationpool fi if test ! -z "\$http_proxy"; then export http_proxy export https_proxy="\$http_proxy" export HTTP_PROXY="\$http_proxy" export HTTPS_PROXY="\$http_proxy" fi echo host.example.com > /etc/hostname hostname host.example.com if ! which lsb-release | read; then apt-get install lsb-release fi codename=\$(lsb_release -cs) if test -z "\$codename"; then echo "Failed to detect correct codename" exit 1 fi wget https://apt.puppetlabs.com/puppetlabs-release-pc1-\$(codename).deb dpkg -i puppetlabs-release-pc1-\$(codename) .deb mkdir -p /etc/puppetlabs/puppet cat > /etc/puppetlabs/puppet/puppet.conf<puppet cert sign host.example.com" echo "Use CTRL+C to stop cycle, if fails due to different reasons" sleep 5 done EOT

Module description

Complete list of Bash parameters for the initial installation script

~# ./setup_puppetserver.sh Usage: ./setup_puppetserver.sh [ [ [ [] ] ] ]

r10k_repo_url - URI of the Git repository
certname - fully qualified domain name of the host
cflocation - initialization of the fact cf_location
cflocationpool - initialization of fact cf_location_pool
http_proxy - proxy server for HTTP and HTTPS requests

Complete list of Bash parameters for the Puppet client initialization script

~# /root/genclientinit.sh Usage: ./genclientinit.sh [ [ []]]
The meaning of the parameters is the same as in the previous script.

cfpuppetserver class

deployuser = "deploypuppet" - username for automatically deploying configuration updates
deployuser_auth_keys = undef - list of keys for $deployuser
repo_url = undef - repository URI (example: ssh://user@host/repo or file:///some/path)
puppetserver = true - whether to install the Puppet Server component on this node
puppetdb = true - whether to install the PuppetDB component on this node
puppetdb_port = 8081 - port for PuppetDB
setup_postgresql = true - whether to install the PostgreSQL component on this node (only if PuppetDB installation is enabled)
service_face = "any" - name of the cfnetwork::iface resource for accepting incoming connections
puppetserver_mem = auto - RAM for Puppet Server in megabytes (minimum 192MB)
puppetdb_mem = auto - RAM for PuppetDB in megabytes (minimum 192MB)
postgresql_mem = auto - RAM for PostgreSQL in megabytes (minimum 128MB)

class cfpuppetserver::puppetdb

postgresql_host = "localhost" - database address
postgresql_listen = $postgresql_host - the value goes directly to the listen_addresses PostgreSQL directive
postgresql_port = 5432 - database port
postgresql_user = "puppetdb" - PuppetDB user in the database
postgresql_pass = "puppetdb" - password of the PuppetDB user in the database
postgresql_ssl = false - enable connection encryption based on Puppet PKI certificates

class cfpuppetserver::puppetserver

autosign = false - SHOULD NOT be used in a combat environment, except perhaps in the DMZ. Exists exclusively for test automation.
global_hiera_config = "cfpuppetserver/hiera.yaml" - the path to the default Hiera configuration file according to Puppet canons (the first component is the name of the module, the rest is the path under the files/ folder in the module)

You can help and transfer some funds for the development of the site

Managing a large number of Unix systems cannot be called convenient. To change one parameter, the administrator has to contact each machine; scripts can only partially help, and not in all situations.

It should be recognized that Windows network administrators are still in a more advantageous position. It is enough to change the group policy settings and after a while all computers on the network, including those with a recently installed operating system, will “learn” about the innovation, if it concerns them, of course. Looking back over the long period of Unix development, you will notice that nothing like this ever caught on. There are solutions like kickstart that help with the initial installation operating system, but further refinement will require significant effort. Commercial solutions like BladeLogic and OpsWare only partially solve the problem of automating settings; their main advantage is the availability GUI, and they can only be purchased from large organizations. There are, of course, projects offering free solutions, but throughout their existence they have never been able to create a large community. For example, Cfengine is not very popular among administrators, although in addition to Linux, it can be used in *BSD, Windows and Mac OS X. This may be due to the relative complexity of creating configurations. When describing tasks, you have to take into account the features of each specific system and manually control the sequence of actions when executing commands. That is, the administrator must remember that for some systems you should write adduser for others, useradd, take into account the location of files on different systems, and so on. This complicates the process of writing commands by an order of magnitude; it is very difficult to create the correct configuration on the fly, and it is almost impossible to read the created configurations after a while. Despite the GPL license, Cfengine is actually a one-man project who controls all changes and is not very interested in building an open society. As a result, the capabilities of cfengine are quite satisfactory for the developer, but for other administrators it is rather an extra headache. To improve cfengine, various add-ons were created by third-party developers, which often only made the situation worse. The author of several such modules for cfengine, Luke Kanies, eventually decided to develop a similar tool, but without many of cfengine’s shortcomings.

Puppet Features

Puppet, like cfengine, is a client-server system that uses a declarative, that is, mandatory language for describing tasks and libraries for their implementation. Clients periodically (30 minutes by default) connect to the central server and receive the latest configuration. If the received settings do not match the system state, they will be executed, and if necessary, a report on the operations performed will be sent to the server. The server can save messages to syslog or a file, create an RRD graph, and send them to a specified e-mail. Additional Transactional and Resource abstraction layers provide maximum compatibility with existing settings and applications, allowing you to focus on system objects without worrying about differences in the implementation and description of detailed commands and file formats. The administrator operates only with the object type, Puppet takes care of the rest. Thus, the packages type knows about 17 package systems; the required one will be automatically recognized based on information about the version of the distribution or system, although, if necessary, the package manager can be forced.

Unlike scripts, which are often not possible to use on other systems, Puppet configurations written by third-party administrators will, for the most part, work without problems on any other network. In Puppet CookBook [ http://www.reductivelabs.com/trac/puppet/tags/puppet%2Crecipe] there are already three dozen ready-made recipes. Puppet currently officially supports the following operating systems and services: Debian, RedHat/Fedora, Solaris, SUSE, CentOS, Mac OS X, OpenBSD, Gentoo and MySQL, LDAP.

Puppet language

To move forward, you must first understand the basic elements and capabilities of the language. Language is one of strengths Puppet. With its help, the resources that the administrator plans to manage and actions are described. Unlike most similar solutions, Puppet allows the language to simplify access to all similar resources on any system in a heterogeneous environment. A resource description typically consists of a name, type, and attributes. For example, let's point to the /etc/passwd file and set its attributes:

file("/etc/passwd":

owner => root,

group => root,

Now clients, having connected to the server, will copy the /etc/passwd file and install the specified attributes. You can define multiple resources in one rule, separating them using a semicolon. What to do if the configuration file used on the server differs from the client ones or is not used at all? For example, this situation may arise when VPN settings connections. In this case, the file can be pointed to using the source directive. There are two options here, as usual, to specify the path to another file; two URI protocols are also supported: file and puppet. In the first case, a link to an external NFS server, in the second option, an NFS-like service is launched on the Puppet server, which exports resources. In the latter case, the default path is relative to the puppet root directory - /etc/puppet. That is, the link puppet://server.domain.com/config/sshd_config will correspond to the file /etc/puppet/config/sshd_config. You can override this directory using the filebucket directive, although it is more correct to use the section of the same name in the /etc/puppet/fileserver.conf file. In this case, you can restrict access to the service only from certain addresses. For example, let's describe the config section.

path /var/puppet/config

allow *.domain.com

allow 192.168.0.*

allow 192.168.1.0/24

deny *.wireless.domain.com

And then we turn to this section when describing the resource.

source => "puppet://server.domain.com/config/sshd_config"

Before the colon is the name of the resource. In the simplest cases, you can simply specify an alias or variables as a name. The alias is set using the alias directive. full path to the file. In more complex configurations

file("/etc/passwd":

alias => passwd

Another option for creating an alias is good when you have to deal with different operating systems. For example, let's create a resource describing the sshd_config file:

file(sshdconfig:

name => $operatingsystem ? (

solaris => "/usr/local/etc/ssh/sshd_config",

default => "/etc/ssh/sshd_config"

In this example, we are faced with a choice. The file for Solaris is specified separately, for all others the file /etc/ssh/sshd_config will be selected. Now this resource can be accessed as sshdconfig, depending on the operating system the desired path will be selected. For example, we indicate that if the sshd daemon is running and received new file, you should restart the service.

ensure => true,

subscribe => File

Variables are often used when working with user data. For example, we describe the location of user home directories:

$homeroot = "/home"

Now the files of a specific user can be accessed as

$(homeroot)/$name

The $name parameter will be filled with the user's account name. In some cases it is convenient to define a default value for some type. For example, for the exec type, they often indicate the directories in which it should look for the executable file:

Exec (path => "/usr/bin:/bin:/usr/sbin:/sbin")

If you need to point to several nested files and directories, you can use the recurse parameter.

file("/etc/apache2/conf.d":

source => “puppet:// puppet://server.domain.com/config/apache/conf.d”,

recurse => "true"

Multiple resources can be combined into classes or definitions. Classes are a complete description of a system or service and are used separately.

"/etc/passwd": owner => root, group => root, mode => 644;

"/etc/shadow": owner => root, group => root, mode => 440

As in object-oriented languages, classes can be overridden. For example, on FreeBSD the group owner of these files is wheel. Therefore, in order not to completely rewrite the resource, let’s create a new class freebsd, which will inherit the linux class:

class freebsd inherits linux (

File[“/etc/passwd”] ( group => wheel );

File[“/etc/shadow”] ( group => wheel )

For convenience, all classes can be placed in a separate file, which can be connected using the include directive. Definitions can take multiple parameters as arguments, but do not support inheritance and are used when you need to describe reusable objects. For example, let's define the home directory of users and the commands necessary to create a new account.

define user_homedir ($group, $fullname, $ingroups) (

user("$name":

ensure => present,

comment => "$fullname",

gid => "$group",

groups => $ingroups,

membership => minimum,

shell => "/bin/bash",

home => "/home/$name",

require => Group[$group],

exec("$name homedir":

command => “/bin/cp -R /etc/skel /home/$name; /bin/chown -R $name:$group /home/$name",

creates => "/home/$name",

require => User[$name],

Now to create a new one account just contact user_homedir.

user_homedir("sergej":

group => "sergej",

fullname => “Sergej Jaremchuk”,

ingroups => ["media", "admin]

There are separate descriptions of nodes that support inheritance like classes. When a client connects to the Puppet server, the corresponding node section will be searched and settings specific only to this computer will be provided. To describe all other systems, you can use node default. A description of all types is given in the “Type Reference” document, which must be read in any case, at least in order to understand all the capabilities of the Puppet language. Various types allow you to execute specified commands, including when certain conditions are met (for example, changing a configuration file), work with cron, user credentials and groups, computers, mounting resources, starting and stopping services, installing, updating and removing packages, working with SSH keys, Solaris zones and so on. This is how easy it is to force the list of packages in distributions using apt to be updated every day between 2 and 4 hours.

schedule(daily:

period => daily,

range =>

exec("/usr/bin/apt-get update":

schedule => daily

The update for that period will be performed by each system only once, after which the task is considered completed and will be deleted from the client computer. The Puppet language supports other familiar structures: conditions, functions, arrays, comments, and the like.

Installing Puppet

Puppet requires Ruby (>= 1.8.1) with OpenSSL support and XMLRPC libraries, as well as the Faster library [ http://reductivelabs.com/projects/facter]. The Ubuntu 7.04 repository that was used for the test installation already included the puppy package.

$ sudo apt-cache search puppet

puppet — centralized configuration management for networks

puppetmaster - centralized configuration management control daemon

During installation, all necessary dependency packages will be installed: facter libopenssl-ruby libxmlrpc-ruby.

$ sudo apt-get install puppet puppetmaster

You can check the availability of Ruby libraries with the command.

$ ruby -ropenssl -e "puts:yep"

~$ ruby -rxmlrpc/client -e "puts:yep"

If no errors are received, then everything you need is already included. Files that describe the desired configuration of systems are called manifests in Puppet terminology. When launched, the daemon tries to read the file /etc/puppet/manifests/site.pp; if it is missing, it displays a warning message. When testing, you can tell the daemon to work in offline mode, in which case the manifest is not required

$ sudo /usr/bin/puppetmasterd --nonodes

If necessary, you can connect other files to site.pp, for example with class descriptions. For a test run, you can enter the simplest instructions in this file.

file("/etc/sudoers":

owner => root,

group => root,

All configuration files for both the server and clients are located in /etc/puppet. The fileserver.conf file that we talked about above is optional and is used only if Puppet will also work as a file server. On Ubuntu, this file exports the /etc/puppet/files subdirectory. The ssl subdirectory contains certificates and keys that will be used for encryption when connecting clients. Keys are created automatically the first time you start puppetmasterd; you can create them manually with the command.

$ sudo /usr/bin/ puppetmasterd --mkusers.

The puppetd.conf and puppetmasterd.conf files are similar. They indicate some parameters for the operation of daemons on the client system and server. The client file differs only in the presence of the server parameter, which points to the computer on which puppetmasterd is running.

server = grinder.com

logdir = /var/log/puppet

vardir = /var/lib/puppet

rundir = /var/run

# send a report to the server

To avoid typing everything manually, you can create a template using puppetd itself.

$ puppetd --genconfig > /etc/puppet/puppetd.conf

Similarly, you can create site.pp on the server.

$ puppetd --genmanifest > /etc/puppet/manifests/site.pp

Another file, tagmail.conf, allows you to specify the email addresses to which reports will be sent. In the simplest case, you can use one line.

all: [email protected]

The configuration files are not enough for the client to connect to the server. To do this, you also need to sign the certificates. First, to let the server know about the new computer on the client system, enter the command:

$ sudo puppetd --server grinder.com --waitforcert 60 --test

info: Requesting certificate

warning: peer certificate won’t be verified in this SSL session

notice: Did not receive certificate

If a different line is returned, you should check the operation of the server.

$ ps aux | grep puppet

puppet 5779 0.0 1.4 27764 15404 ? Ssl 21:49 0:00 ruby /usr/sbin/puppetmasterd

The firewall must allow connections on port 8140.

On the server we receive a list of certificates that need to be signed.

$ sudo puppetca --list

nomad.grinder.com

And we sign the client certificate.

$ sudo puppetca –sign nomad.grinder.com

Now the client can freely connect to the server and receive settings.

Unfortunately, it is simply not possible to show all the capabilities of Puppet within the article. But as you can see, this is a functional and flexible tool that allows you to solve most of the problems of simultaneous administration of a large number of systems. If your line of work requires you to set up several systems. And most importantly, the project managed to gather a small but constantly growing community. Therefore, let's hope that a good idea will not be allowed to die or go aside.

Puppet is a cross-platform structure that allows system administrators Perform common tasks using code. The code allows you to perform various tasks from installing new programs to checking file permissions or updating user accounts. Puppet superior not only during the initial installation of the system, but throughout the entire life cycle of the system. In most cases puppet used in client/server configuration.

This section shows installation and configuration Puppet in a client/server configuration. This simple example demonstrates how to install Apache using Puppet.

Installation

For installation Puppet enter in terminal:

Sudo apt-get install puppetmaster

On the client machine(s), enter:

Sudo apt-get install puppet

Settings

Before setting up puppet you might want to add an entry DNS CNAME For puppet.example.com, Where example.com- this is your domain. Default clients Puppet check DNS for puppet.example.com as the puppet server name ( Puppet Master). See Domain Name Service for additional details on using DNS.

If you don't intend to use DNS, you can add entries to the /etc/hosts file on the server and client. For example, in the /etc/hosts file Puppet server add:

127.0.0.1 localhost.localdomain localhost puppet 192.168.1.17 meercat02.example.com meercat02

On every Puppet In the client, add an entry for the server:

192.168.1.16 meercat.example.com meercat puppet

Replace IP addresses and domain names from the example to your current addresses and names of the server and clients.

Now let's set up some resources for apache2. Create a file /etc/puppet/manifests/site.pp containing the following:

Package ( "apache2": ensure => installed ) service ( "apache2": ensure => true, enable => true, require => Package["apache2"] )

Node "meercat02.example.com" ( include apache2 )

Replace meercat02.example.com to your current name Puppet client.

The final step for this simple Puppet server is to restart the service:

Sudo /etc/init.d/puppetmaster restart

Now on Puppet everything is configured on the server and it’s time to configure the client.

First, let's configure the service Puppet agent to launch. Edit /etc/default/puppet, replacing the value START on yes:

Sudo /etc/init.d/puppet start

Let's go back to Puppet server to sign the client certificate using the command:

Sudo puppetca --sign meercat02.example.com

Check /var/log/syslog for any configuration errors. If everything went well, the package apache2 and its dependencies will be installed to Puppet client.

This example is very simple and does not show many of the features and benefits. Puppet. For additional information look

Sergey Yaremchuk

Centralized configuration of UNIX systems using Puppet

Managing a large number of UNIX systems cannot be called convenient. To change one parameter, the administrator has to contact each machine; scripts can only partially help, and not in all situations.

It should be recognized that Windows network administrators are still in a more advantageous position. It is enough to change the group policy settings, and after a while all computers on the network, including those with a recently installed operating system, will “learn” about the innovation, if it concerns them, of course. Looking back over the long period of UNIX development, you can see that nothing like this ever caught on. There are solutions like kickstart that help with the initial installation of the operating system, but further development will require significant effort. Commercial solutions, like BladeLogic and OpsWare, solve the problem of automating settings only partially; their main advantage is the presence of a graphical interface, and only large organizations can afford to purchase them. There are, of course, projects that offer free solutions, but throughout their existence they have not been able to create a large community. For example, Cfengine is not very popular among administrators, although, in addition to Linux, it can be used in *BSD, Windows and Mac OS X. This may be due to the relative complexity of creating configurations. When describing tasks, it is necessary to take into account the characteristics of each specific system and manually control the sequence of actions when executing commands. That is, the administrator must remember that for some systems you should write adduser, for others - useradd, take into account the location of files on different systems, and so on. This complicates the process of writing commands by an order of magnitude; it is very difficult to create the correct configuration on the fly, and it is almost impossible to read the created configurations after a while. Despite the GPL license, Cfengine is essentially a one-man project who controls all changes and is not very interested in building an open society. As a result, the capabilities of Cfengine are quite satisfactory for the developer, but for other administrators it is rather an extra headache. To improve Cfengine, various add-ons were created by third-party developers, which often only made the situation worse. The author of several such modules for Cfengine, Luke Kanies, eventually decided to develop a similar tool, but without many of the shortcomings of Cfengine.

Puppet Features

Puppet, like Cfengine, is a client-server system that uses a declarative language to describe tasks and libraries to implement them. Clients periodically (every 30 minutes by default) connect to the central server and receive the latest configuration. If the received settings do not match the system state, they will be executed, and if necessary, a report on the operations performed will be sent to the server. The message server can save it to syslog or a file, create an RRD graph, and send it to the specified e-mail. Additional Transactional and Resource abstraction layers provide maximum compatibility with existing settings and applications, allowing you to focus on system objects without worrying about differences in implementation and description of detailed commands and file formats. The administrator operates only with the object type, Puppet takes care of the rest. Thus, the packages type knows about 17 package systems; the required one will be automatically recognized based on information about the version of the distribution or system, although, if necessary, the package manager can be set forcibly.

Unlike scripts, which are often impossible to use on other systems, Puppet configurations written by third-party administrators will mostly work without problems on any other network. Puppet CookBook already has three dozen ready-made recipes. Puppet currently officially supports the following operating systems and services: Debian, RedHat/Fedora, Solaris, SUSE, CentOS, Mac OS X, OpenBSD, Gentoo and MySQL, LDAP.

Puppet language

To move forward, you must first understand the basic elements and capabilities of the language. Language is one of Puppet's strengths. It describes the resources that the administrator plans to manage and the actions they take. Unlike most similar solutions, Puppet allows the language to simplify access to all similar resources on any system in a heterogeneous environment. A resource description typically consists of a name, type, and attributes. For example, let's point to the /etc/passwd file and set its attributes:

file("/etc/passwd":

Owner => root,

Group => root,

Mode => 644,

Now clients connecting to the server will copy the /etc/passwd file and set the specified attributes. You can define multiple resources in one rule, separating them using a semicolon. But what if the configuration file used on the server differs from the client ones or is not used at all? For example, this situation may arise when setting up VPN connections. In this case, you should point to the file using the source directive. There are two options here; you can, as usual, specify the path to another file, and also using the two supported URI protocols: file and puppet. In the first case, a link to an external NFS server is used; in the second option, an NFS-like service is launched on the Puppet server, which exports resources. In the latter case, the default path is relative to the puppet root directory – /etc/puppet. That is, the link puppet://server.domain.com/config/sshd_config will correspond to the file /etc/puppet/config/sshd_config. You can override this directory using the filebucket directive, although it is more correct to use the section of the same name in the /etc/puppet/fileserver.conf file. In this case, you can restrict access to the service to only certain addresses. For example, let's describe the config section:

Path /var/puppet/config

Allow *.domain.com

Allow 127.0.0.1

Allow 192.168.0.*

Allow 192.168.1.0/24

Deny *.wireless.domain.com

And then we refer to this section when describing the resource:

source => "puppet://server.domain.com/config/sshd_config"

Before the colon is the name of the resource. In the simplest cases, you can simply specify the full path to the file as the name. In more complex configurations it is better to use an alias or variables. The alias is set using the alias directive:

file("/etc/passwd":

Alias => passwd

Another option for creating an alias is good when you have to deal with different operating systems. For example, let's create a resource that describes the sshd_config file:

file(sshdconfig:

Name => $operatingsystem ? (

Solaris => "/usr/local/etc/ssh/sshd_config",

Default => "/etc/ssh/sshd_config"

service(sshd:

Ensure => true,

Subscribe => File

Variables are often used when working with user data. For example, we describe the location of user home directories:

$homeroot = "/home"

Now the files of a specific user can be accessed as:

$(homeroot)/$name

The $name parameter will be filled with the user's account name. In some cases it is convenient to define a default value for some type. For example, for the exec type it is very common to specify the directories in which it should look for the executable file:

Exec ( path => "/usr/bin:/bin:/usr/sbin:/sbin" )

If you need to point to several nested files and directories, you can use the recurse parameter:

file("/etc/apache2/conf.d":

Source => "puppet:// puppet://server.domain.com/config/apache/conf.d",

Recurse => "true"

Multiple resources can be combined into classes or definitions. Classes are a complete description of a system or service and are used separately:

class linux (

File (

"/etc/passwd": owner => root, group => root, mode => 644;

"/etc/shadow": owner => root, group => root, mode => 440

class freebsd inherits linux (

File["/etc/passwd"] ( group => wheel );

File["/etc/shadow"] ( group => wheel )

For convenience, all classes can be placed in a separate file, which must be included using the include directive. Definitions can take multiple parameters as arguments, but do not support inheritance and are used when you need to describe reusable objects. For example, let's define the user's home directory and the commands needed to create a new account:

define user_homedir ($group, $fullname, $ingroups) (

User("$name":

Ensure => present,

Comment => "$fullname",

Gid => "$group",

Groups => $ingroups,

Membership => minimum,

Shell => "/bin/bash",

Home => "/home/$name",

Require => Group[$group],

Exec("$name homedir":

Command => "/bin/cp -R /etc/skel /home/$name; /bin/chown -R $name:$group /home/$name",

Creates => "/home/$name",

Require => User[$name],

Now, to create a new account, just contact user_homedir:

user_homedir("sergej":

Group => "sergej",

Fullname => "Sergej Jaremchuk",

Ingroups => ["media", " admin]

There are separate descriptions of nodes that support inheritance, as well as classes. When a client connects to the Puppet server, the corresponding node section will be searched and settings specific only to this computer will be provided. To describe all other systems, you can use node default. A description of all types is given in the “Type Reference” document, which must be read in any case, at least in order to understand all the capabilities of the Puppet language. Various types allow you to execute specified commands, including when certain conditions are met (for example, changing a configuration file), work with cron, user credentials and groups, computers, mounting resources, starting and stopping services, installing, updating and removing packages, working with SSH keys, Solaris zones, and so on. This is how you can easily force the list of packages in distributions using apt to be updated daily between 2 and 4 hours:

schedule(daily:

Period => daily,

Range =>

exec("/usr/bin/apt-get update":

Schedule => daily

Installing Puppet

Puppet requires Ruby (version 1.8.1 and above) with OpenSSL support and XMLRPC libraries, as well as the Faster library. The Ubuntu 7.04 repository that was used for the test installation already includes the puppy package:

$ sudo apt-cache search puppet

~$ ruby -rxmlrpc/client -e "puts:yep"

yep

If no errors are received, then everything you need is already included. Files that describe the desired configuration of systems are called manifests in Puppet terminology. When launched, the daemon tries to read the file /etc/puppet/manifests/site.pp; if it is missing, it displays a warning message. When testing, you can tell the daemon to run in standalone mode, which does not require a manifest:

$ sudo /usr/bin/puppetmasterd --nonodes

If necessary, you can connect other files to site.pp, for example, with class descriptions. For a test run, you can enter the simplest instructions in this file.

class sudo (

File("/etc/sudoers":

Owner => root,

Group => root,

Mode => 440,

node default (

Include sudo

All configuration files, both server and client, are located in /etc/puppet. The fileserver.conf file, which we already talked about, is optional and is used only if Puppet will also work as a file server. On Ubuntu, this file exports the /etc/puppet/files subdirectory. The ssl subdirectory contains certificates and keys that will be used for encryption when connecting clients. Keys are created automatically the first time you start puppetmasterd; you can create them manually with the command:

$ sudo /usr/bin/puppetmasterd --mkusers

server = grinder.com

logdir = /var/log/puppet

vardir = /var/lib/puppet

rundir = /var/run

# send a report to the server

report = true

To avoid typing everything manually, you can create a template using puppetd itself:

$ puppetd --genconfig > /etc/puppet/puppetd.conf

Similarly, you can create site.pp on the server:

$ puppetd --genmanifest > /etc/puppet/manifests/site.pp

Another file, tagmail.conf, allows you to specify email addresses to which reports will be sent. In the simplest case, you can use one line:

all: [email protected]

The configuration files are not enough for the client to connect to the server. To do this, you also need to sign the certificates.

First, to let the server know about the new computer, enter the command on the client system:

$ sudo puppetd --server grinder.com --waitforcert 60 –test

The firewall must allow connections on port 8140.

On the server we get a list of certificates that need to be signed:

$ sudo puppetca –list

nomad.grinder.com

And sign the client certificate:

$ sudo puppetca –sign nomad.grinder.com

Now the client can freely connect to the server and receive settings.

Unfortunately, it is impossible to show all the capabilities of Puppet within the article. But, as you can see, this is a functional and flexible tool that allows you to solve most of the problems of simultaneous administration of a large number of systems. And most importantly, the project managed to gather a small but constantly growing community. Therefore, let's hope that a good idea will not be allowed to die or go aside.

Good luck!