A new edition of the bestselling Puppet Beginner’s Guide is now available, covering all the latest features of Puppet 5. Learn Puppet from scratch with renowned consultant and author John Arundel of Bitfield Consulting, from installation to managing complex infrastructures, including chapters on managing Amazon AWS cloud resources and Docker containers with Puppet. Every code example is available right now online in a special GitHub repo just for this book. There is also a complete, working Puppet infrastructure for you to download, adapt, and use on your own servers.
Order your copy now on Amazon.com (Amazon.co.uk)
Reviews of this edition:
“I’ve been using Puppet for some time now, and picked this book up after many great recommendations across the internet. Though this is titled as a Beginner’s Guide, I’ve found that my own Puppet knowledge has increased, and I’ve started rewriting and making use of a lot of the concepts introduced in this book, some of which are idiomatic. I would recommend this book to anyone looking for a guide on how to use Puppet from start to finish, whether you ‘think you know it all’, or whether you are just curious.” - Amazon.co.uk
“Excellent Puppet primer for any SRE. As a managed service provider, we recently migrated a lot of workloads to being managed by Puppet. This includes Amazon instances across multiple regions for several customers. The examples for AWS management provided in the book definitely improved on the structure of the manifests we built from just assembling bits and pieces from Stack Overflow. Another very positive surprise was to discover how Puppet is also able to orchestrate our Docker workloads throughout their lifecycle. We previously relied on a pipeline on GitLab CI, but being able to consolidate those into Puppet streamlines our work further. This book is a phenomenal encyclopedia for looking up the appropriate syntax and tons of best practices for everything you need to know about Puppet. The structure of the book makes digesting such a complex topic extremely tolerable, and I can only give it my highest recommendation for anyone aspiring to learn or improve their Puppet skills.” - Amazon.com
Reviews of previous editions:
“I’ve thoroughly enjoyed it. It explains the subject matter very well and takes a lot of the mystery out of Puppet. If you are looking for a way into Puppet I highly recommend this book.” - Amazon.co.uk
“Great guide that’s easy to read and written well so the concepts are simple to understand.” - Amazon.co.uk
“Very clear explanations.” - Amazon.com
“Well-written… smoothly-structured… John Arundel’s new book is a solid guide to helping you become both a proficient Puppet user and a more efficient, knowledgeable, and versatile system administrator.” — Slashdot
“I can’t imagine having found anything better. This book avoids the stress and boredom of a slow-paced learning session by diving right in to the most important concepts. It was all I needed to feel confident at work, and it only took an hour before I was writing my own code.” — Amazon.com
“John Arundel is both a superb engineer and just as importantly an excellent teacher/communicator. Currently the best Puppet book available and essential for anyone wanting to learn Puppet.” — Amazon.com (Kindle edition)
There’s only so much you can learn from books. As Proust wrote, “We don’t receive wisdom; we must discover it for ourselves after a journey that no one can take for us or spare us.”
It’s good to have a friend point us in the right direction and come with us a little way for moral support, but then we need to walk on by ourselves. I hope that this book will be the beginning of your journey, not the end.
The world-famous classical guitarist John Williams was once asked how long it took him to learn to play the guitar. “I’m still learning,” he said.
This book is designed for those new to Puppet, including system administrators and developers who are looking to manage computer server systems for configuration management. No prior programming or system administration experience is assumed.
The Puppet 5 Beginner’s Guide gets you up and running with Puppet 5 straight away with real-world examples. It will take you from complete beginner to confident Puppet user with a series of clear examples and case studies. By reading this book, you will learn to manage every aspect of your server setup. You will gain expert understanding of Puppet’s latest and most advanced features and put them to work.
This book helps you to:
Install and set up Puppet and discover the latest and most advanced features
Automate your IT infrastructure
Use the latest features in Puppet 5+ and official modules
Manage clouds, containers, and orchestration
Learn best practices to make Puppet more reliable and high performing
Getting started with Puppet
Creating your first manifests
Managing Puppet code with Git
Understanding Puppet resources
Variables, expressions, and facts
Managing data with Hiera
Mastering modules
Classes, roles, and profiles
Managing files with templates
Controlling containers
Orchestrating cloud instances
Putting it all together
A brief introduction to the aims and methods of the book, and how to use it.
In this chapter you’ll learn about some of the challenges of managing configuration on servers, some common solutions to this problem, and how automation tools such as Puppet can help. You’ll also learn how to download the GitHub repository containing all the source code and examples in this book, how to set up your own Vagrant virtual machine to run the code, and how to download and install Puppet.
Why do we need Puppet anyway?
Keeping configuration synchronized
Repeating changes across many servers
Self-updating documentation
Why not just write shell scripts?
Why not just use containers?
Why not just use serverless?
Configuration management tools
What is Puppet?
Resources and attributes
Puppet architectures
Getting ready for Puppet
Getting the Git repo
Installing Virtualbox and Vagrant
Running your Vagrant VM
Alternative Vagrant VMs
Troubleshooting Vagrant
The Puppet manifest is the complete expression, in code form, of how your server should be configured. It consists of a set of resources, each of which models some aspect of system configuration, such as a file, or a software package.
In this chapter you’ll learn how to write your first manifest with Puppet, and how to put Puppet to work configuring a server. You’ll also understand how Puppet reads and applies a manifest. You’ll see how to use Puppet to manage the contents of files, how to install packages, and how to control services.
Hello, Puppet: your first Puppet manifest
Understanding the code
Modifying existing files
Dry-running Puppet
How Puppet applies the manifest
Creating a file of your own
Managing packages
How Puppet applies the manifest
Exercise
Querying resources with puppet resource
Services
Getting help on resources with puppet describe
The package-file-service pattern
Understanding the code
Notifying a linked resource
Resource ordering with require
In this chapter we’ll install Git, create a repo, and import the manifests created in previous chapters. Readers will learn to make some changes to the manifests and commit them to the repo. Readers will learn how to check the revision history of the repo. We will clone our existing repo to a master repo, and deploy it on a new server. We will set up a script and cron job to pull changes from the master repo and apply them automatically.
What is version control?
Tracking changes
Sharing code
Creating a Git repo
Making your first commit
How often should I commit?
Branching
Distributing Puppet manifests
Creating a GitHub account and project
Pushing your repo to GitHub
Cloning the repo
Fetching and applying changes automatically
Writing a manifest to set up regular Puppet runs
Applying the run-puppet manifest
The run-puppet
script
Testing automatic Puppet runs
Managing multiple nodes
We’ve already met three important types of Puppet resource: packages, files, and services. In this chapter we’ll go into a little more detail on these to see how to exploit their power to the full. We’ll also learn about three other important resource types: user
, cron
, and exec
resources. Along the way we’ll talk about:
Files
The path
attribute
Managing whole files
Ownership
Permissions
Directories
Trees of files
Symbolic links
Packages
Uninstalling packages
Installing specific versions
Installing the latest version
Installing Ruby gems
Installing gems in Puppet’s context
Using ensure_packages
Services
The hasstatus
attribute
The pattern
attribute
The hasrestart
and restart
attributes
Users
Creating users
The user
resource
The group
resource
Managing SSH keys
Removing users
Cron resources
Attributes of the cron
resource
Randomizing cron jobs
Removing cron jobs
Exec resources
Automating manual interaction
Attributes of the exec resource
The user
attribute
The onlyif
and unless
attributes
The refreshonly
attribute
The logoutput
attribute
The timeout
attribute
How not to misuse exec
resources
In this chapter you will learn about variables in Puppet, and discover the different types of data structures and data types that Puppet provides. You will find out how to use variables and values in expressions, and how to apply different resources depending on the value of expressions, using conditional statements. You will also learn how Puppet manifests can get data about the server using Facter, find out which are the most important standard facts, and see how to create your own custom facts. Finally, you will see how to use the each
function to loop over arrays and hashes.
Introducing variables
Using Booleans
Interpolating variables in strings
Creating arrays
Declaring arrays of resources
Understanding hashes
Setting resource attributes from a hash
Introducing expressions
Meeting Puppet’s comparison operators
Introducing regular expressions
Using conditional expressions
Making decisions with if
statements
Choosing options with case
statements
Finding out facts
Using the facts
hash
Running the facter
command
Accessing hashes of facts
Referencing facts in expressions
Using memory facts
Discovering networking facts
Providing external facts
Creating executable facts
Iterating over arrays
Using the each
function
Iterating over hashes
In this chapter you will learn why it’s useful to separate program logic from configuration data. You will see how to use Puppet’s built-in Hiera mechanism to store and query config data, including encrypted secrets such as passwords and SSL keys. You will also learn how to use Hiera data to construct Puppet resources directly.
Why Hiera?
Data needs to be maintained
Settings depend on nodes
Operating systems differ
The Hiera way
Setting up Hiera
Adding Hiera data to your Puppet repo
Troubleshooting Hiera
Querying Hiera
Typed lookups
Types of Hiera data
Single values
Boolean values
Arrays
Hashes
Interpolation in Hiera data
Using lookup()
Using alias()
Using literal()
The hierarchy
Dealing with multiple values
Merge behaviours
Data sources based on facts
What belongs in Hiera?
Creating resources with Hiera data
Building resources from Hiera arrays
Building resources from Hiera hashes
The advantages of managing resources with Hiera data
Managing secret data
Setting up GnuPG
Adding an encrypted Hiera source
Creating an encrypted secret
How Hiera decrypts secrets
Editing or adding encrypted secrets
Distributing the decryption key
In this chapter you will learn about Puppet Forge, the public repository for Puppet modules, and you’ll see how to install and use third-party modules from Puppet Forge with the standard module management tool r10k
. You’ll work through examples of using three key Forge modules: puppetlabs-apache
, puppetlabs-mysql
, and puppet-archive
. Finally, you’ll learn how to develop your own Puppet module from scratch, working through a complete example, how to add appropriate metadata for your module, and how to upload it to Puppet Forge.
Using Puppet Forge modules
What is the Puppet Forge?
Finding the module you need
Using r10k
Understanding the Puppetfile
Managing dependencies with generate-puppetfile
Using modules in your manifests
Using puppetlabs/mysql
Using puppetlabs/apache
Using puppet/archive
Exploring the standard library
Safely installing packages with ensure_packages
Modifying files in place with file_line
Introducing some other useful functions
The pry
debugger
Writing your own modules
Creating a repo for your module
Writing the module code
Creating and validating the module metadata
Tagging your module
Installing your module
Applying your module
More complex modules
Uploading modules to the Puppet Forge
In this chapter you will explore the details of Puppet classes, the distinction between defining a class and creating instances of that class, and how to pass parameters to classes. You’ll learn how to create defined resource types, and how they differ from classes. You’ll also see how to organize your Puppet code using the concepts of roles, and profiles, and how to use Hiera data to assign different classes to nodes.
Classes
The class
keyword
Declaring parameters to classes
Automatic parameter lookup from Hiera data
Parameter data types
Available data types
Content type parameters
Range parameters
Flexible data types
Defined resource types
Type aliases
Managing classes with Hiera
Using include
with lookup()
Common and per-node classes
Roles and profiles
Roles
Profiles
In this chapter we’ll learn about an important and powerful feature of Puppet: the template. We’ll see how to use a simple template to interpolate the values of Puppet variables, facts, and Hiera data into a file, and we’ll also introduce more complex templates using iteration and conditional statements to generate dynamic configuration files.
What are templates?
The dynamic data problem
Puppet template syntax
Using templates in your manifests
Referencing template files
Inline templates
Template tags
Computations in templates
Conditional statements in templates
Iteration in templates
Iterating over Facter data
Iterating over structured facts
Iterating over Hiera data
Working with templates
Passing parameters to templates
Validating template syntax
Rendering templates on the command line
Legacy ERB templates
In this chapter we’ll look at the emerging topic of containers and see how it relates to configuration management. We’ll see how to use Puppet to manage Docker itself, as well as images and containers, and explore some different strategies for managing configuration within containers.
Understanding containers
The deployment problem
Options for deployment
Introducing the container
What Docker does for containers
Deployment with Docker
Building Docker containers
The layered filesystem
Managing containers with Puppet
Managing Docker with Puppet
Installing Docker
Running a Docker container
Stopping a container
Running multiple instances of a container
Managing Docker images
Building images from Dockerfiles
Managing Dockerfiles
Building dynamic containers
Configuring containers with templates
Self-configuring containers
Persistent storage for containers
Host-mounted volumes
Docker volumes
Networking and orchestration
Connecting containers
Container orchestration
What is orchestration?
What orchestration tools are available?
Running Puppet inside containers
Are containers mini VMs or single processes?
Configuring containers with Puppet
Containers need Puppet too
In this chapter we will learn how to use Puppet to create and manage AWS cloud resources, such as EC2 instances.
Introducing the cloud
Automating cloud provisioning
Using CloudFormation
Using Terraform
Using Puppet
Setting up an Amazon AWS account
Creating an AWS account
Creating an IAM policy
Creating an IAM user
Storing your AWS credentials
Getting ready to use puppetlabs/aws
Creating a key pair
Installing the puppetlabs/aws
module
Installing the AWS SDK gem
Creating EC2 instances with Puppet
Choosing an Amazon Machine Image (AMI)
Creating the EC2 instance
Accessing your EC2 instance
VPCs, subnets, and security groups
The ec2_securitygroup
resource
The ec2_instance
resource
Managing custom VPCs and subnets
Creating an instance in a custom VPC
The ec2_vpc
resource
The ec2_vpc_internet_gateway
resource
The ec2_vpc_routetable
resource
The ec2_vpc_subnet
resource
Other AWS resource types
Provisioning AWS resources from Hiera data
Iterating over Hiera data to create resources
Orchestrator nodes
Cleaning up unused resources
In this chapter we will apply all the lessons from previous chapters to see what a complete, working infrastructure looks like. This is a complete worked example which readers can use as the basis of their own Puppet codebase, adapting and expanding it as needed.
Getting the demo repo
Copying the repo
Understanding the demo repo
The control repo
Module management
Classes
Roles
Profiles
Users and access control
SSH configuration
Sudoers configuration
Time zone and clock synchronization
Puppet configuration
The bootstrap process
Adapting the repo for your own use
Configuring users
Adding per-node data files and role classes
Modifying the bootstrap credentials
Bootstrapping a new node
Bootstrapping a Vagrant VM
Bootstrapping physical or cloud nodes
Using other distributions and providers
The beginning